2:53 Just one correction: that last packet should be ACK, not FIN. The same side doesn't send another FIN (unless the first one gets lost), but it does need to ACKnowledge the other side's FIN. (Also, a clarification. This is a common way to label handshakes when talking about TCP. Really, SYN, FIN, and ACK are flags. Only the packets labeled as such have the SYN and FIN flags, because they are used only for those purposes. But ACK is in very nearly every packet, because you always need to acknowledge the other side's state, except with the very first packet that opens a connection, which will only have SYN set. But for purposes of talking about handshakes, we only mark it where it relates to SYN and FIN packets.)
I can't relate to this, friend of mine used to hang up during saying goodbye. So I always received a partial goodbye and never a chance to say goodbye myself.
My idea is just as unsatisfactory and would add more noise, but an "Attack" with a second delayed "Attack" by some arbitrary amount of time and same for "Received" and delayed "Received." Unsatisfactory because both can still be lost, so it just kicks the can down the road with "What if more redundancy?" Sometimes though with these problems one redundant measure is safe enough, maybe two, rarely three, but analyses are done for safety engineering for how much redundancy is statistically enough so that we don't go out of budget or over materials or some other criteria they need to remain below...
Can't either/both sides ping each other periodically and if there's no response in a timeout of 60 seconds to consider the connection closed? I think it's called check-alive heartbeat or something.
@@thekaxmax They already _do_ have timers, though. As long as the connection is open, there's always a need to wait for more packets and time it out if nothing is received for too long. (No need for pinging. Just sending or acknowledging data is enough to prove aliveness. But if no data is sent for a while, there are so-called keep-alive packets, which just repeat acknowledgment of the other side. It's like a response to a ping, but without the ping itself, because they aren't trying to measure their latency.) And even more, when sending a steady stream of data, the sender needs to regulate (with the receiver's help) the rate at which it sends, to avoid network congestion.
What's the problem of having the message be "Attack at [time] if [2 or more] messages come. Also don't stop sending messages until say... 4 has been recieved?"
My guess is that how would first general know that they received 2 messages? First general sends a bunch and assume they got it, but if they were all intercepted the first general would be charging in alone. Or the second general receives it and sends a bunch back but they all get intercepted, so same problem as the original 2 generals problem.
@@ilikekitty10 simply send the entire state of the system. for each message show if it's OK or ?? for each of the generals. After some messages you have enough history to confirm that both you and the other party have received enough messages. Problem is that the payloads gets bigger and bigger.
So do they even send a "FIN" message at all or was that just a hypothetical to demonstrate the problem? I understand that it cannot be verified due to the problem being unsolvable but is that packet even sent at all just to save on server "resources" or smth?
I imagine you still send a FIN just to be courteous - you just limit how many times you resend it without receiving a response before giving up and dropping the connection without a confirmed termination.
You should really explain the SYN flood attack and ACK bugs. This was more of the old school "You can't trust the computers". Bugs are found everyday, but are also fixed everyday.
And peers can stop responding without sending a FIN in any case. One reason keepalive and other pings exist is to check that a connection is still alive. There's never a guarantee of an orderly close and an orderly close should never be relied upon.
0:59 - Why does Professor Clegg call this a three-way handshake? Who is the third party in this handshake? It's not in any of the diagrams he draws, which have only "Client" and "Server".
It's not that he, specifically, calls it that. _Everyone_ familiar with TCP does, including the people who wrote the RFC, and no doubt plenty of people before the RFC existed.
@@fllthdcrb Yes, but maybe my point is that someone who is NOT "familiar with TCP" is going to have this question. EDUCATION consists of telling people things they don't already know. So you have to assume that someone doesn't know something.
@@jaspermcjasper3672 True. I think he just assumed that people seeing this video may already have known these basic things about TCP. Computerphile likes to teach things sometimes by building up knowledge over multiple videos. I would have thought they had a video about it, but it looks like they didn't cover handshakes. Whoops.
Computers need "attack at dawn" message and "approved, attacking at dawn" and then waiting until dawn unless the message wasn't acquired and you send "attack at dawn" again and they reply again until no more "attack at dawn" message.
Why even wait for another response? Just assume the connection was terminated after receiving the first "fin", respond once and don't care about it anymore :D Also timeouts help in real life of course.
Just as an idea, TCP reliability relies on the sender to assume lost packets and resend. What it if was the other way around? What if instead, the receiver was in charge of handling reliability? I.e. instead of: Sender - Msg2 Receiver - ACK Msg2 Sender - Msg3 Receiver - ACK Msg3 ... Do Receiver - REQ Msg2 Sender - Msg2 Receiver - REQ Msg3 Sender - Msg3 ... On the surface they seem the same, but it does allow for a slight modification at the end. Receiver - REQ Msg101 Sender - Msg101 Receiver - REQ Msg102 Sender - Msg102 FIN That way the FIN is in the final message itself. Given that the receiver is in charge of reliability, the receiver will send another REQ if it didn't receive the packet, so a simple timeout after last message would suffice. I know the immediate point is that the sender doesn't KNOW the last message was received, but as this whole video points out, that's already an issue with the current system. As a benefit though, the server that has many simultaneous connections isn't sending out unnessessary duplicate messages because it didn't receive an ACK. I would assume this would reduce line congestion coming out of the server given that the potential unnecessary duplicate messages would be on the client side.
Well that's the crutch he mentioned at the end of the video, the sender has to wait a bit on the line to see if the receiver actually got the final message, before hanging up. So it's essentially the same issue and same solution.
Say you want to reset. A tells B to reset. B sends ack back, but the ack gets lost. B got reset, and send ack. They did do the reset. A never got an ack, and thinks B never got the message. A is not reset, but B is.
The animation from 'user with a laptop' to 'general with a sword' gave me some huge Macromedia Dreamweaver Flash animations throwback.
Macromedia Director was probably the application I first used for animation! -Sean
2:53 Just one correction: that last packet should be ACK, not FIN. The same side doesn't send another FIN (unless the first one gets lost), but it does need to ACKnowledge the other side's FIN. (Also, a clarification. This is a common way to label handshakes when talking about TCP. Really, SYN, FIN, and ACK are flags. Only the packets labeled as such have the SYN and FIN flags, because they are used only for those purposes. But ACK is in very nearly every packet, because you always need to acknowledge the other side's state, except with the very first packet that opens a connection, which will only have SYN set. But for purposes of talking about handshakes, we only mark it where it relates to SYN and FIN packets.)
I was surprised when this video ended.
I can't relate to this, friend of mine used to hang up during saying goodbye. So I always received a partial goodbye and never a chance to say goodbye myself.
UDP friend ?
@@Ediolot123 I laughed way too hard at this, thanks!
My idea is just as unsatisfactory and would add more noise, but an "Attack" with a second delayed "Attack" by some arbitrary amount of time and same for "Received" and delayed "Received." Unsatisfactory because both can still be lost, so it just kicks the can down the road with "What if more redundancy?"
Sometimes though with these problems one redundant measure is safe enough, maybe two, rarely three, but analyses are done for safety engineering for how much redundancy is statistically enough so that we don't go out of budget or over materials or some other criteria they need to remain below...
Loving the snazzy animations, thank you 👍🏼
Fascinating
Can't either/both sides ping each other periodically and if there's no response in a timeout of 60 seconds to consider the connection closed? I think it's called check-alive heartbeat or something.
Computationally intensive cos both need to run and monitor timers
@@thekaxmax They already _do_ have timers, though. As long as the connection is open, there's always a need to wait for more packets and time it out if nothing is received for too long. (No need for pinging. Just sending or acknowledging data is enough to prove aliveness. But if no data is sent for a while, there are so-called keep-alive packets, which just repeat acknowledgment of the other side. It's like a response to a ping, but without the ping itself, because they aren't trying to measure their latency.)
And even more, when sending a steady stream of data, the sender needs to regulate (with the receiver's help) the rate at which it sends, to avoid network congestion.
Wonderful explanation. I will use this
Nice with a video about (negative/positive) proofs, and why some things are unavoidable ugly.
Just described a denial of service attack. Send 1000s of syn messages and ignore the syn ack measages.
Those attacks can be mitigated with syn cookies. SCTP solves it by using a 4-step handshake.
What's the problem of having the message be "Attack at [time] if [2 or more] messages come. Also don't stop sending messages until say... 4 has been recieved?"
My guess is that how would first general know that they received 2 messages? First general sends a bunch and assume they got it, but if they were all intercepted the first general would be charging in alone. Or the second general receives it and sends a bunch back but they all get intercepted, so same problem as the original 2 generals problem.
@@ilikekitty10 simply send the entire state of the system. for each message show if it's OK or ?? for each of the generals. After some messages you have enough history to confirm that both you and the other party have received enough messages. Problem is that the payloads gets bigger and bigger.
@@Nope-w3c and you still just end up waiting around....
...man that sounds familiar, doesn't it? 😆
0:31 hey dont call me out like that
So do they even send a "FIN" message at all or was that just a hypothetical to demonstrate the problem?
I understand that it cannot be verified due to the problem being unsolvable but is that packet even sent at all just to save on server "resources" or smth?
I imagine you still send a FIN just to be courteous - you just limit how many times you resend it without receiving a response before giving up and dropping the connection without a confirmed termination.
You should really explain the SYN flood attack and ACK bugs. This was more of the old school "You can't trust the computers". Bugs are found everyday, but are also fixed everyday.
Were explained elsewhere
is this somehow connected with the two-phase-commit protocol for distributed transactions? or analog kind of problem?
Yes, 2-phase commit is vulnerable in the same way. 3-phase commit is the kludge.
And peers can stop responding without sending a FIN in any case. One reason keepalive and other pings exist is to check that a connection is still alive. There's never a guarantee of an orderly close and an orderly close should never be relied upon.
Connection timeouts are the final orderly finish
0:59 - Why does Professor Clegg call this a three-way handshake? Who is the third party in this handshake? It's not in any of the diagrams he draws, which have only "Client" and "Server".
It's not a third party, it's a third message. Syn, Syn/Ack, Ack.
It's not that he, specifically, calls it that. _Everyone_ familiar with TCP does, including the people who wrote the RFC, and no doubt plenty of people before the RFC existed.
@@fllthdcrb Yes, but maybe my point is that someone who is NOT "familiar with TCP" is going to have this question. EDUCATION consists of telling people things they don't already know. So you have to assume that someone doesn't know something.
@@jaspermcjasper3672 True. I think he just assumed that people seeing this video may already have known these basic things about TCP. Computerphile likes to teach things sometimes by building up knowledge over multiple videos. I would have thought they had a video about it, but it looks like they didn't cover handshakes. Whoops.
I don't think his General was carrying a sword...
To be fair, If your willing to hand wave away the wait at the beginning then it's fair to hand wave away the wait at the ending.
polite protocol 😂
Computers need "attack at dawn" message and "approved, attacking at dawn" and then waiting until dawn unless the message wasn't acquired and you send "attack at dawn" again and they reply again until no more "attack at dawn" message.
Why even wait for another response? Just assume the connection was terminated after receiving the first "fin", respond once and don't care about it anymore :D Also timeouts help in real life of course.
I saw tom scotts video about this problem and he talked about idempotency tokens. Does that not solve the issue or is this in a different context?
Just as an idea, TCP reliability relies on the sender to assume lost packets and resend. What it if was the other way around? What if instead, the receiver was in charge of handling reliability? I.e. instead of:
Sender - Msg2
Receiver - ACK Msg2
Sender - Msg3
Receiver - ACK Msg3
...
Do
Receiver - REQ Msg2
Sender - Msg2
Receiver - REQ Msg3
Sender - Msg3
...
On the surface they seem the same, but it does allow for a slight modification at the end.
Receiver - REQ Msg101
Sender - Msg101
Receiver - REQ Msg102
Sender - Msg102 FIN
That way the FIN is in the final message itself. Given that the receiver is in charge of reliability, the receiver will send another REQ if it didn't receive the packet, so a simple timeout after last message would suffice.
I know the immediate point is that the sender doesn't KNOW the last message was received, but as this whole video points out, that's already an issue with the current system.
As a benefit though, the server that has many simultaneous connections isn't sending out unnessessary duplicate messages because it didn't receive an ACK. I would assume this would reduce line congestion coming out of the server given that the potential unnecessary duplicate messages would be on the client side.
Well that's the crutch he mentioned at the end of the video, the sender has to wait a bit on the line to see if the receiver actually got the final message, before hanging up. So it's essentially the same issue and same solution.
g..g..granville get yer mop!
for me personally my hardest fin was when my dad died recently :(
no ack at all..
If you have a standing order to attack at a future time which resets if it is cancelled requiring an ack.
Say you want to reset.
A tells B to reset.
B sends ack back, but the ack gets lost.
B got reset, and send ack. They did do the reset.
A never got an ack, and thinks B never got the message.
A is not reset, but B is.
Doesn't block chain solve this problem?
Fork handles !?!
For the almighty algorithm and assorted generals
Like for first comment!
Can you rename this to byzantine general's problem? This is the official name of it. What you called it makes no sense.
well, there's 10 minutes wasted!
Bye then
Were you waiting for an ACK?
Felt like that 20 sekonds in, scrolled the comments, yeah right...
Not their fault you watched it
The title of the video is the "goodbye problem", not "the solution to the goodbye problem".