Correction at 11:42: When one partner stops taking order, the system is consistent but not available because one of the nodes is not available. For a system to have property of availability, all the nodes(number of partners here) of the system should be available.
this is wrong. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it is available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will not get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency. For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability.
@@laterlater8348 Exactly! Availability means at least one node should be available to process the request, whereas Consistency requires all the nodes to be available.
But if one of the nodes goes down, the system can still be consistent right. Now instead of N nodes we have an N-1 node which is serving the requests and serving the same data throughout. And a system is said to be highly available when we have multiple systems spread across which helps in serving more requests. I think what she said is right here
@@girishanker3796 Talking in context of example she gave. Me and my partner were taking orders and sharing the order details to each other we took to maintain consistency. Now a network Partitioning occurred, I will not be able to access my partner's orders, so if someone asked me info about order 1247 which is not in my list, I won't be able to give. Here we lost Availability. Then I removed my partner, now only I am handling, I have all the data and I am able to address user queries, here Availability is not lost.
Thankyou Sister for making such wonderful playlist.. Your videos were so helpful in preparing intrws.. I think Interviewer asked everything from this playlist, at one point I thought he had also watched your videos.. lol He asked CAP theorm and I vent out everything that I learnt here and at the end he said”very good”.. that was such a lovely feeling
I really admire how clearly you explained, by taking a very practical human-level example, that one could relate to and follow. Keep up the good work. Thank you so much!
Nice examples, theories are available on internet but the examples you provided for CAP & MQs are awesome, that's the quality of goood teacher. Thank you.
Having high availabilty and high consistency at the same time might be possible or not. But in real life scenarios I think, partition tolerance cannot be ignored. Hence, we have to choose either between the availability or consistency with partition tolerance, i.e., AP OR CP.
Hi Yogita thank you for the video. In case of active active we have high availability but it may not be consistent and in case of active stand by it will be consistent but not HA even though active can failover to standby any case of disaster. By consistency do we mean when A receives data copy of It is sent to B and then acknowledgement is sent back to client? I feel consistency and partioning are mutually exclusive Please correct me if I am wrong
HI...Why don't we hash the CustomerphoneNO or CustomerId while receiving order and send it to specific service for that hash...SO that data don't have to be consistent while taking orders...We can sync data in parallel if we need to be...
We have to start with more services...We would be estimating in advance how many customers we would be serving(based on survery's) ....While customer's about to increase...we replicate data to more services with hashfunction( parllely with original system) a month in advance to more machines...and reboot Loadbalancer with new services+old service info...
This is really helpful! Explained it better than any of the explanations I've seen so far as I've been trying to learn this to prepare for interviews. Thank you!
what if the order id is based on customer data like the phone number of customer, then both bar and foo will have consistent data, and if a customer places multiple orders, then we append the order id with _1, _2,... both at foo and bar ? can you please let me know
Hello mam , u said that cap theorem do not support all 3 attributes when we look up on to our modern applications like FB , YT , In shorts where we can see consistency - up to date information , Availability we can use 24/7 , Partition tolerance - Even any one of our node crash it shows us the response due to its replication ..( My question was ) : if any one of the database like cassandra , mongodb , sql , orcale ,do not support how the Application work ??? it will helpful for my Btech project Thank you ...
Network partitions are unavoidable . I don’t think you have a choice of not having partition . The only choice you can make is between consistency and availability, that is, CP or AP
how to handle that scenario if before replicating a particular data channel breaks?eg after writing to A then a will sync it to b but before that a fails?
Partition P is must in real world for sure... now you have trade off between A & C .. Example of of AP - Social Media like Instagram, Facebook, LinkedIn etc. Where Consistence can be given off for eventual consistence or delayed consistency but Availability and Partiotion are preferred. for example, its okay to wait to read the comments or post r like with some eventual consistency. Example of CP - Banking, Finance where transaction needs consistence following the ACID properties. Where A transaction like sending and receiving funds needs constancy. hence C is must else your core service will have no purpose... now you have trade off between P and A but P is must... so most financial sectors follows PC CAP is itself a huge topic
Good explanation...specially analogy with ordering system. In context of CAP , there is one another concept going around PACELC. Hopefully going to see more detail in coming videos. :)
CAP is possible. Let's start to take a order based on decided range of order id by person1 & person2. Whenever customer will ask the status then redirect request to corresponding person's server.
Consistency can only achieved by RDBMS and they rarely goes down so they can also provide availability and High availability can only provided by Cassandra cause of its ring partitioning of the nodes and For Partition i preferred MongoDB, DynamoDB etc.
Ma'm after this video my curiosity levels are going very please kindly upload the next part of the video asap. Giving gratitude for ur excellent explaination skills that made me fall in love with system designs.
That's absolutely correct. CAP theorem talks about distributed systems. Refer page 4 in these slides - www.researchgate.net/profile/Eric_Brewer3/publication/221343719_Towards_robust_distributed_systems/links/09e41511a6f75c93ff000000.pdf
If it's monolithic architecture, then I feel the system will be highly consistent, but there's also a risk of a single point of failure. If the database fails, the system is no longer available.
having partitioning tolerance should always be assumed, so really the only combos would be CP or AP, because partitions will always happen at some point, and having a point of failure due to them is a flaw in design imo
Having consistency without partition tolerance is not possible because if there is no partition tolerance, that means there is no communication between nodes, there is no way for system to be consistent.
We can't achieve perfect CA system which is scalable. It won't be scalable. Hence, most of the systems, achieve Availability with eventual consistency. Means, we prefer to keep the system available all the time and bring the system to consistent state eventually as the time passes by synchronising the data in background. Not immediately.
The example for explaining CAP was too good. I think I should use this example to explain it to others. By the way one can only achieve two properties at a time in CAP. All the three can't be achieved at the same time.
As requests increases in DS the load increases in which A with P is not possible , C without P is possible . C and A without P will only be a good choice.
C and A without P will only be a good choice. and if we have single server then there is no problem of consistency and partitioning - then it won't be a case of CAP theorem since it is applicable to only distributed systems. Besides, availability means a node being available in the system. It does not entails the capacity at which requests are being served. Refer these slides: www.researchgate.net/publication/221343719_Towards_robust_distributed_systems
Thanks a lot for such an amazing video. Request you to please upload video on like Facebook messager, New Feed , Uber , etc full system design video. Thanks a lot for your effort and hard work to deliver this kind of informative contents. :)
Understood the concepts properly 1. System needs to be Consistent as well as Available. Partitions cannot be tolerated because that would make the system inconsistent. Availability does not get affected in this case. This case we have CA. 2. System needs to Tolerate Partition and be Consistent. Then sacrifice one of the partitions by shutting that partition down. The system is Consistent as we chose one of the two partitions. No communication errors will occur. Also the system has just tolerated paritition. But it is not Available since we just shut down an entire partition. This case we have PC. 3. The system needs to Tolerate Paritition and be Available. Do not shut down one of the paritions. This means we do not shut down any partition and keep being available to the customer. But now Consistency is an issue since the partitions have a broken communication channel between them. This case we have PA. So you can have either: CA, PC, PA
Writing this for those who are new to this topic and the video is a bit misleading. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency (AP). For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability (CP). Also the premise here is the system is distributed in nature and a Partition (P )has occured for reasons like network failure or node failure . Finally in case of no partition that is system completely healthy then we get both Consistency and Availability (CA)
I would appreciate if you can mark the time stamp where wrong information is shared from my end. It will help me learn and also publish errata if something is actually wrong.
Correction at 11:42: When one partner stops taking order, the system is consistent but not available because one of the nodes is not available. For a system to have property of availability, all the nodes(number of partners here) of the system should be available.
this is wrong. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it is available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will not get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency. For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability.
@@laterlater8348 Exactly! Availability means at least one node should be available to process the request, whereas Consistency requires all the nodes to be available.
But if one of the nodes goes down, the system can still be consistent right. Now instead of N nodes we have an N-1 node which is serving the requests and serving the same data throughout. And a system is said to be highly available when we have multiple systems spread across which helps in serving more requests. I think what she said is right here
@@girishanker3796 Talking in context of example she gave. Me and my partner were taking orders and sharing the order details to each other we took to maintain consistency. Now a network Partitioning occurred, I will not be able to access my partner's orders, so if someone asked me info about order 1247 which is not in my list, I won't be able to give. Here we lost Availability.
Then I removed my partner, now only I am handling, I have all the data and I am able to address user queries, here Availability is not lost.
The story telling is really fantastic. I was able to recreate the same situation while being interviewed. Thanks alot
Great that someone is not just click-baiting and teaching something that's useful.
Keep up the good work.
I appreciate your honest comment. This is one of the qualities we are focusing on for our channel. Thanks for appreciating.
This is the best video on CAP theorem in RUclips. Thanks for the clear explanation. Looking forward to more of your videos on System Design!
More to come!
lol
Thankyou Sister for making such wonderful playlist.. Your videos were so helpful in preparing intrws.. I think Interviewer asked everything from this playlist, at one point I thought he had also watched your videos.. lol
He asked CAP theorm and I vent out everything that I learnt here and at the end he said”very good”.. that was such a lovely feeling
Lovely playlist ❤❤
I really admire how clearly you explained, by taking a very practical human-level example, that one could relate to and follow. Keep up the good work. Thank you so much!
I read many content on CAP, but my doubt got clarified in this video. Thanks for all effort!
after watching more then 7-8 videos on CAP theorem found this video easy to understand. Thanks for the Food and Bar example
Nice examples, theories are available on internet but the examples you provided for CAP & MQs are awesome, that's the quality of goood teacher. Thank you.
Glad you like them!
Having high availabilty and high consistency at the same time might be possible or not. But in real life scenarios I think, partition tolerance cannot be ignored. Hence, we have to choose either between the availability or consistency with partition tolerance, i.e., AP OR CP.
That's correct
A superb example that makes good sense. Kudos
All your videos are great Yogita. Thanks for sharing your knowledge.
Thanks Ishank.
Another brilliant video!!! - Thanks for making and sharing them.
This is the best tutorial on CAP ever. Really appreciate your efforts. Looking forward to learn from you.
Simple and easy to learn! Great job
Excellent, Great explanation. Thank you very much Yogita
Awesome. Very clean video with clear understanding
Hi Yogita thank you for the video. In case of active active we have high availability but it may not be consistent and in case of active stand by it will be consistent but not HA even though active can failover to standby any case of disaster.
By consistency do we mean when A receives data copy of It is sent to B and then acknowledgement is sent back to client?
I feel consistency and partioning are mutually exclusive
Please correct me if I am wrong
What if we keep a pub sub model, do we still need to care about partition tolerence?
HI...Why don't we hash the CustomerphoneNO or CustomerId while receiving order and send it to specific service for that hash...SO that data don't have to be consistent while taking orders...We can sync data in parallel if we need to be...
We have to start with more services...We would be estimating in advance how many customers we would be serving(based on survery's) ....While customer's about to increase...we replicate data to more services with hashfunction( parllely with original system) a month in advance to more machines...and reboot Loadbalancer with new services+old service info...
This is really helpful! Explained it better than any of the explanations I've seen so far as I've been trying to learn this to prepare for interviews. Thank you!
You're very welcome!
what if the order id is based on customer data like the phone number of customer, then both bar and foo will have consistent data, and if a customer places multiple orders, then we append the order id with _1, _2,... both at foo and bar ? can you please let me know
Crystal clear explanation
Great example of partner and orders. Thank you!
Great explanation Yogita
CAP is not possible, CA is RDBMS, AP such as Cascanda or CouchDB, CP such as MongoDB, HBase, BigTable
Hello mam , u said that cap theorem do not support all 3 attributes when we look up on to our modern applications like FB , YT , In shorts where we can see consistency - up to date information , Availability we can use 24/7 , Partition tolerance - Even any one of our node crash it shows us the response due to its replication ..( My question was ) : if any one of the database like cassandra , mongodb , sql , orcale ,do not support how the Application work ??? it will helpful for my Btech project Thank you ...
Beautifully explained... Loved it 👍🏻
As easy as it can get - this explanation is really the most easily understood one for CAP. Thanks.
You're very welcome Shyam :)
Firstrate explanation 👌👌
Network partitions are unavoidable . I don’t think you have a choice of not having partition . The only choice you can make is between consistency and availability, that is, CP or AP
That's almost correct. More details in next video!
how to handle that scenario if before replicating a particular data channel breaks?eg after writing to A then a will sync it to b but before that a fails?
The whole write is failed/rejected in that case.
Really mam this example of foo and bar is nicely explained.
Thanks Harsh. Glad you found it useful.
Partition P is must in real world for sure... now you have trade off between A & C ..
Example of of AP - Social Media like Instagram, Facebook, LinkedIn etc. Where Consistence can be given off for eventual consistence or delayed consistency but Availability and Partiotion are preferred. for example, its okay to wait to read the comments or post r like with some eventual consistency.
Example of CP - Banking, Finance where transaction needs consistence following the ACID properties. Where A transaction like sending and receiving funds needs constancy. hence C is must else your core service will have no purpose... now you have trade off between P and A but P is must... so most financial sectors follows PC
CAP is itself a huge topic
Very well explained. 🙏
Good explanation...specially analogy with ordering system. In context of CAP , there is one another concept going around PACELC. Hopefully going to see more detail in coming videos. :)
Thanks for sharing Jinal :)
Was looking for this... Thank you
😀
CAP is possible.
Let's start to take a order based on decided range of order id by person1 & person2. Whenever customer will ask the status then redirect request to corresponding person's server.
Thanks for the both videos. It was really good. Thank You
Glad you enjoyed it!
ONe of the Best explanation
this was a great explanation! thank you :)
Consistency can only achieved by RDBMS and they rarely goes down so they can also provide availability and High availability can only provided by Cassandra cause of its ring partitioning of the nodes and For Partition i preferred MongoDB, DynamoDB etc.
Ma'm after this video my curiosity levels are going very please kindly upload the next part of the video asap. Giving gratitude for ur excellent explaination skills that made me fall in love with system designs.
Thanks a lot Kunal. Next video will be up soon!
Why we need partition?
very good explanation.. Thanks :)
Thanks a lot for such quality content
C and A is possible in case of non-distributed(single instance of system handling everything) systems but not in distributed.
That's absolutely correct. CAP theorem talks about distributed systems. Refer page 4 in these slides - www.researchgate.net/profile/Eric_Brewer3/publication/221343719_Towards_robust_distributed_systems/links/09e41511a6f75c93ff000000.pdf
Hi, awesome video as always.
If it's a monolithic database application then can we call that it's Highly Consistent and Highly Available?
If it's monolithic architecture, then I feel the system will be highly consistent, but there's also a risk of a single point of failure. If the database fails, the system is no longer available.
possibility is CAP, CP, AP is it true??
having partitioning tolerance should always be assumed, so really the only combos would be CP or AP, because partitions will always happen at some point, and having a point of failure due to them is a flaw in design imo
Awesome work .Pls start distributed systems series
We will, very soon :)
Having consistency without partition tolerance is not possible because if there is no partition tolerance, that means there is no communication between nodes, there is no way for system to be consistent.
We can't achieve perfect CA system which is scalable. It won't be scalable. Hence, most of the systems, achieve Availability with eventual consistency. Means, we prefer to keep the system available all the time and bring the system to consistent state eventually as the time passes by synchronising the data in background. Not immediately.
maam where have you read such things !!
i also want to read please suggest!
A video on books on system design is coming soon Yogesh!
same example is share by SCALER Academy by Ansuman..
Nice❤️❤️
Thank you so much for this Mam!!
The example for explaining CAP was too good. I think I should use this example to explain it to others. By the way one can only achieve two properties at a time in CAP. All the three can't be achieved at the same time.
As requests increases in DS the load increases in which A with P is not possible , C without P is possible . C and A without P will only be a good choice.
C and A without P will only be a good choice. and if we have single server then there is no problem of consistency and partitioning - then it won't be a case of CAP theorem since it is applicable to only distributed systems. Besides, availability means a node being available in the system. It does not entails the capacity at which requests are being served. Refer these slides: www.researchgate.net/publication/221343719_Towards_robust_distributed_systems
Oh my bad.🤦
very well explained
check for CA (not very high because there could be network issues) AP CP
Thank you very much
Thank you.
Thank you for your videos
Glad you like them!
Thanks a lot for such an amazing video. Request you to please upload video on like Facebook messager, New Feed , Uber , etc full system design video. Thanks a lot for your effort and hard work to deliver this kind of informative contents. :)
Sure I will. Thanks a lot for appreciation!
Great work Madam . Pls start low level design in java from scratch . Like ticket booking system etc .
Thanks Sanath. Will try!
I would say, it can be implemented but it's not guarantee , there will be always a possibility for a failure.
Ma'am please make videos of system design of facebook, twitter etc.
We will in the future Shivam. 🙂
Understood the concepts properly
1. System needs to be Consistent as well as Available. Partitions cannot be tolerated because that would make the system inconsistent. Availability does not get affected in this case. This case we have CA.
2. System needs to Tolerate Partition and be Consistent. Then sacrifice one of the partitions by shutting that partition down. The system is Consistent as we chose one of the two partitions. No communication errors will occur. Also the system has just tolerated paritition. But it is not Available since we just shut down an entire partition. This case we have PC.
3. The system needs to Tolerate Paritition and be Available. Do not shut down one of the paritions. This means we do not shut down any partition and keep being available to the customer. But now Consistency is an issue since the partitions have a broken communication channel between them. This case we have PA.
So you can have either: CA, PC, PA
Writing this for those who are new to this topic and the video is a bit misleading. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency (AP). For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability (CP). Also the premise here is the system is distributed in nature and a Partition (P )has occured for reasons like network failure or node failure . Finally in case of no partition that is system completely healthy then we get both Consistency and Availability (CA)
I would appreciate if you can mark the time stamp where wrong information is shared from my end. It will help me learn and also publish errata if something is actually wrong.
@@sudocode I have replied to your other comment , you can get the time stamp there
We can achieve only two of them at a time.
♥️🌠🎆
W O W
I think only possible combinations would be
1. CA - partially.
2. AP
3. CP
#50k soon ❤️ sudocode thanks
no cap
You are so beautiful, its really difficult to concentrate on the technical thinks :)
Thank you
Thank you 🌹
nicely explained