Видео

In the graphs, are we assuming that a message is delivered immediately after is it received(arrives) by a node? Because you are using the word deliver and receive interchangebaly

Consensus algorithms rely on a leader (paxos, raft) to sequence messages 2:50 when using leader based replication, need to select a new leader when the leader crashes or becomes unavailable This is where consensus comes in Consensus is nodes agree on a value 5:10 paxos is for single value consensus and multi paxos is for consensus on multiple values (total order broadcast) Consensus and total order broadcast formally equivalent because if you can form consensus on the next message to deliver then you have total order broadcast Consensus algorithms (paxos, raft) use a partially synchronous, crash recovery model Partial synchrony means that we will use timeout failure detection to assume that a node failed Asynchronous model impossible to achieve consensus (FLP impossibility result)

Beautiful playing!

2:15 read after write consistency (client sees the data it wrote, even if not reading from same replica) Read and write quorum can guarantee this type of consistency if r+w>n r is number of replicas used for reads w number of replicas used for writes n total replicas Example values r = w = n+1/2 write to majority and read from majority Can have n-r replicas unavailable and still get read after write consistency and n-w replicas unavailable for writes and still get read after write consistency 4:50 2 out of 3 quorum when reading, timestamp is compared to read the latest 6:50 diagram of read write quorum 8:40 the client can keep the data consistent with read repair, updating replicas with latest value when it gets the response from each node Note: client uses the existing timestamp of the value so that if another client updated it the database replica with latest value will reject the write from the client (it’s identical to a retry of the write) Replicas can stay in sync using anti entropy and read repair

Fun to watch this series.

Thank you Sir Kleppmann.

7:50 I’m confused about the need for impotence if using tcp. If the ack gets lost, client will re-send that segment, in tcp protocol, shouldn’t the server disregard that duplicate segment because it’s a stateful protocol?

@martin at 22:30 these are two different clients. In lecture of logical clock, counter/vector was managed at server side. How this will done at client side?

At 17:53 . Client is generating logical timestamps. Lets say vector one. What if there are multiple clients. 2 instance of a service which is connected with the database. How that is managed?

Such practical things to learn tysm @martin

Lets say there are three Nodes A, B and C with A as a leader. If there is partition such as (A, B) and (C). C will keep incrementing its term number without becoming a leader whereas (A, B) will keep commiting the log entries. What happens when the partition resolve => Since C term is higher, A will become a follower. Does it mean A will lost its commited entries? I cant find the answer from the codebase, can someone pleaes point out?

Where lamport and vector timestamp used in real-time application or distributed system?

thanks

@kleppman Hi Martin! Great engineering lecture with great background. It's a real pleasure to hear what's going on behind just measuring time in software. Thanks a lot!

Only Chill Guys here.😀

I've looked at several resources on explaining this concept of RPC and yours is the only one which I can say I finally understand it after watching it. Great explanations!

Like that T-shirt color

Say Hi to MIT distributed systems students 😂

I guess there is a problem in Causal broadcast. On the receiving side, always we have deps>delivered at first place( for example, message with SeqNum=2 is received now, message with SeqNum=1 has been delivered before. Therefore we will have deps[i]=2, delivered[i]=1), I guess the it should be : deps< delivered --> ignore the message deps > delivered --> broadcast. Moreover, when deliver[i] has more value than, by further increase we even make it more, then the condition would be always true. Please let me know if I am mistaken.

This subject could not have been explained better Martin. Thank you for your service to the computer science community. This is true gold!

Great analysis, thank you! I need some advice: I have a SafePal wallet with USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). How can I transfer them to Binance?

In the last slide, who do we return to?

I think professor forgot to clarify why clock synchronization using NTP will still fail to order messages. My understanding here is as follows: if clocks on nodes are skewed then the ordering fails, Even if we try to syncronize the clock using NTP, ordering will still fail because of variable network latency, network congestion etc. Due to this, physical clocks cant be perfectly synchronized using NTP and hence will lead to inconsistencies during message ordering and so we use logical clocks. Please correct me if im wrong @kleppmann

I love you Martin

Please correct me if i missed something..Before client sends commit message to coordinator for 2 phase commit start..it perform normal transaction on replicas but my confusion is if the problem happens during that write i.e. one replica performed write but another fails then how two phase will help...i thought that the entire reason of two-phase commit is to perform write in prepare phase and the commit in commit phase...why we are allowing normal update to both replicas before 2pc can start the process.

Thank you so much for creating these succint ,logical , detalied content .it really pave the way for me to get to know distributed system !!

Amazing video!

C first delivers m1, then m2 and then m3 - here I am confused by the diagram and the verb deliver, because the arrow head points to C but you mention delivers. Do you mean C allows m1 to be accepted/delivered to itself by A ?

Thank you for this videos.

Bi-jan-tyne

wow great teaching, thank you

I remember this font from a famous programming book just not quite sure what book it is. Design Patterns?

Thank you sir, a grateful new subscriber.

videos are clearer than the book 😅

I have an ultimate question that puzzles me for a long time since I first went thru these courses: If we can simply use vector clock or even lamport clock (that provides TOB)+ read repair to achieve linearizability in a leaderless replication, why do we bother having complicated algorithms such as Raft or Paxos to only achieve linearizability in a single leader replication?

I don't know what I'm missing but AppendLog seems to be a bit problematic since a replica will deliver a log to its application that potentially can changed the storage state? Imagine if there's n replica and leader is sending message to them, and after 1st and 2nd replica successfully append log, suddenly all other n-2 replica plus the leader die, then the message that was delivered to application in the 1st and 2nd node would be invalid right?

Amazing!!

After watching many videos, I finally found the explanation of an understandable protocol given in a very understandable way. Thank you very, very, very much

It is a gem 💎

i wondering about version vector, too. !!

Pretty much all this concepts are just confused on what to name an request standard. Let's call it RPC, no REST no gRPC, how does AJAX sound... It good that I now know the difference tho.

This video is so clear and informative. It answered most of the questions I had.

can we use this algorithm in wireless sensor networks?

What an excellent video - thank you!

For me the best course for DS. Thanks Martin

11:10 wow that's the ultimate definition of concurrency right there

Thank you for this amazing free series, you are the best.

Big fan of his thought process. I wish I could do phd under him.

I am little confused on your statement on "logical timstamps/lamport timestamps provides total order". if they provide total order why even bother about vector clocks? you clearly say that lamport timestamp cannot distinguish concurrent events so how can you call that a "total order"? Also when I put in "does lamport timestamp gives partial order or total order ?" in chatGPT it clearly says "partial order". What am I missing?

I have one doubt. In one of the slide , you mentioned ..in other parts of the computer science Synchronous and Asynchronous are used in other aspects or differently. Can someone please help me to understand that ? and great video series and thanks a lot Martin!