Head to database.husseinnasser.com for a discount coupon to my Introduction to Database Engineering course. Link redirects to udemy with coupon applied.
does CockroachDB handle all that even if the coordinator node goes down? i searched your youtube channel but found no video about CockroachDB.. i wonder why.. is there something wrong with CockroachDB ?
As a student, for my project, I had to deploy my application on multiple servers for scaling (horizontal scaling). Since I was using docker and docker swarm to help in containers and orchestrestration, came the time to think 🤔 on how to handle the database. Question was : How do I horizontally scale the database while keeping its state across these servers? Ans : Distributed Transactions. After going deep in it and understanding the level of complexity behind scaling a database horizontally, my prof advised me to just use a single database, cuz this problem is still, as of date not yet completely solved. Distributed Databases is still an active area of research and development. He/She who solves the problem is set to impact the industry massively. Just food for thought and Thanks for the video. :)
It depends on your system, I don't believe absolute consistency exist in distributed databases. But eventual consistency is pretty much solved and used in any large scale system.
We can maintain two leaders, both of them accessing some database like etcd which is an inmemory distributed key value store. So if one goes down, the other is there. Or we can just keep one and spin up another when one goes down.
Upon revisiting the video a year later, I discovered that Raft implements a 2-phase commit protocol. If the leader fails, a random leader election occurs with the highest term index.
What would be disadvantages of 1 phase commit, say where each node sends ready to commit message, as soon as transaction work is done and forcing it on stable storage. The coordinator then only sends commit/abort to all the nodes?
We could've done other way right ? Like as soon as co ordinator realized that he didn't got response from one DB , It will ask other DB's to rollback that commit and don't accept Tx's anymore until that offline node comes back online and then all of them will commit tx. But ofcourse , As CAP theorem says we are trading consistensy for availability then.
I have a small doubt regarding the Three-Phase Commit (3PC) protocol: If the coordinator fails after sending the 'CanCommit' requests in 3PC, how is this scenario effectively managed differently from the Two-Phase Commit (2PC) protocol, where a similar failure leads to a blocking issue?
Is it moments like these when I should start considering NoSql for microservices rather than trying to achieve a fake sense of acidity through XA and Saga implementations of DT?
Hussein you're a GOD, you literally published this video 2h before my exam and now I've understood that! Thank you very much! Hope to see more videos about DB stuff 😉
It is not necessary to synchronize the distributed databases. There are two arguements for this. 1. All human are distributed, our memories are not synchronized. What is important is just network communications to pass messages. 2. If you synchronize the databases, it is the same as having one database. This is wasteful and should be avoided. Only minimal backup as 'disk' is necessary.
Head to database.husseinnasser.com for a discount coupon to my Introduction to Database Engineering course. Link redirects to udemy with coupon applied.
does CockroachDB handle all that even if the coordinator node goes down? i searched your youtube channel but found no video about CockroachDB.. i wonder why.. is there something wrong with CockroachDB ?
As a student, for my project, I had to deploy my application on multiple servers for scaling (horizontal scaling). Since I was using docker and docker swarm to help in containers and orchestrestration, came the time to think 🤔 on how to handle the database.
Question was : How do I horizontally scale the database while keeping its state across these servers?
Ans : Distributed Transactions.
After going deep in it and understanding the level of complexity behind scaling a database horizontally, my prof advised me to just use a single database, cuz this problem is still, as of date not yet completely solved.
Distributed Databases is still an active area of research and development. He/She who solves the problem is set to impact the industry massively.
Just food for thought and Thanks for the video. :)
It depends on your system, I don't believe absolute consistency exist in distributed databases. But eventual consistency is pretty much solved and used in any large scale system.
horizontal scaling of databases means? creating a new database when the existing database is full?
Please talk about Saga Design Pattern in the BE Show.
What is BE show
@@arenazo the better question - what is sega design pattern
@@filcondrat it's about coordination of microservices
@@JohnZakaria I was somewhat ironic
I heard MAGA.
We can maintain two leaders, both of them accessing some database like etcd which is an inmemory distributed key value store. So if one goes down, the other is there. Or we can just keep one and spin up another when one goes down.
We try to solve one issue and get a new one. 😅
Great video though even though I am not a dev.
Great Video!
Can you please discuss about Saga Implementation details.
Upon revisiting the video a year later, I discovered that Raft implements a 2-phase commit protocol. If the leader fails, a random leader election occurs with the highest term index.
What would be disadvantages of 1 phase commit, say where each node sends ready to commit message, as soon as transaction work is done and forcing it on stable storage. The coordinator then only sends commit/abort to all the nodes?
We could've done other way right ? Like as soon as co ordinator realized that he didn't got response from one DB , It will ask other DB's to rollback that commit and don't accept Tx's anymore until that offline node comes back online and then all of them will commit tx. But ofcourse , As CAP theorem says we are trading consistensy for availability then.
Very good video, very well explained, Thnx !
I have a small doubt regarding the Three-Phase Commit (3PC) protocol: If the coordinator fails after sending the 'CanCommit' requests in 3PC, how is this scenario effectively managed differently from the Two-Phase Commit (2PC) protocol, where a similar failure leads to a blocking issue?
We mostly use Paxos or Raft in distributed databases these days.
Thanks for informative video . Would love to hear from you about 3 -Phase and SAGA .
2 phase, 3 phase and saga differences would be great. how does it change consistency etc.
Very well explained!
Is it moments like these when I should start considering NoSql for microservices rather than trying to achieve a fake sense of acidity through XA and Saga implementations of DT?
Hussein you're a GOD, you literally published this video 2h before my exam and now I've understood that! Thank you very much! Hope to see more videos about DB stuff 😉
Maybe you could tackle transactional outbox next? I think you could apply the same even though it's meant for databae+messaging.
who wants book shelf tour
Go Saga next
Make a video on mnesia database from erlang ecosystem
We used two phased commits
Database 3 : Ohh what happened, O was in a coma 😂😂😂
It is not necessary to synchronize the distributed databases.
There are two arguements for this.
1. All human are distributed, our memories are not synchronized. What is important is just network communications to pass messages.
2. If you synchronize the databases, it is the same as having one database. This is wasteful and should be avoided. Only minimal backup as 'disk' is necessary.
Ladees vashrum
Feeling , I am able to understand arabic language
First
I am hurt!🤣
🥇
Just want to ask what about timestamp protocol compared to 2 phase protocol couldnt understand their respective drawbacks🥲