This is exactly the type of videos I was hoping to see after I saw you left Netflix. Your frontend masters DSA course showed you're a great teacher and obviously from watching your stream you're super knowledgable so it's the perfect storm for this type of content
Finally, no random 1-2 year ex-F or tech leads teaching some random leetcode, useless non-context system design question, written from their official pass-interview books, or tutorial stuffs, and "constantly unintentionally" mention it can get you into Big tech like them. I was expecting someone from Big tech to fill the gap for this type of contents, and truly show the ability of Big tech engineer (like how we admire their technical ability in the old time besides salary), but didn't expect it took such a long time to have one like Prime.
I took networks in uni last semester and made a couple of tcp and udp programs for class. this clears up the gap in my understanding a lot. Keep these videos up ex faang king
What I love about this is the intersection between theory and practice. I’ve learned bits about networking and specifically TCP before - but didn’t really know how it would work in practice. Just seeing how you turn the theory into code really demystifies it. Great content!
Loving these videos. I don't have the time/focus to tune in to the livestream so would sometimes skim the vods for these insights, but this is everything I could wish for.
This was exactly what I was looking for, this was great. I'd take even longer form content on this info, like 20-30 mins and more working examples. Still this was awesome thank you for sharing all this info
I usually prefer to create a state machine to parse each byte received and generate a callback when the data is finished/available. Therefore working more asynchronous
I'm actually about to publish a Go package to simplify (?) coloring -and formatting - text. Basically you toss a string and a hex color code and get back the string with the ANSI codes necessary for the formatting based on system support (true color of xterm). Don't know if that would help you but anyway. I just need to make a tiny change to one of the functions and I'll be done and hopefully published by tomorrow
so if I understand, TCP handles the sequence of the data, HTTP handles the version number and the length of the data, and HTTPS makes it so, even if someone is listening to the messages that are passed around, even if he can make a guess what the contents of the packets are, he still can't tell if he guessed right. Right?
There are some things that are unencrypted when it comes to TLS and HTTP There are some things that are not. There are now more and more secure algorithms that are being used. Secure DNS as an example
TCP handles the guarantees that you'd want to have to deem your communication is reliable. It guarantees in-order delivery, retransmission of dropped packets, and de-duplication of packets. HTTP is concerned with the contents of the communication between two parties. Like, if I introduce myself "what's your name", convention is to respond with your name and maybe a gesture like a handshake. In the same vein, there are always required headers from the client like the HTTP method, and even if the server doesn't respond with a body, it still needs to respond with a status code like 200, 303, 404, 500 so the client ( usually a web browser ) knows how to process that. In order for clients and servers to be able to understand each other, there have to be some defined elements that are always present in each message, and HTTP governs what those parts are. In other words, HTTP defines how what plain text communication should be sent between two parties in order to have a meaningful conversation. As for HTTPS, that just means your HTTP connection is using TLS ( transport layer security). One of the 3 reasons is the one that you mentioned; if someone is packet sniffing it doesn't matter because the message has been encrypted. The other two parts is for clients to be able to trust that the server they're making a connection with IS ACTUALLY the entity that they say they are ( what good is encrypting messages if I'm unknowingly connected to a bad website. The last part of TLS is that you have guarantees that your message wasn't tampered with ( think of the tamper seals on food, for example ). The short version of this is: TCP: how do we deliver data "reliably" over an unreliable medium HTTP: what kind of structure / form should our text messages between us take? HTTPS (TLS): how do we make this data transfer secure?
i mean technically speaking, the purpose of conway's game is that the next state in the game can always be calculated given the previous state. so you dont need to send the entire world state of the game to every person, just the starting state. You could probably find a way to only send 'update' bits telling each connected client to update their game at the same time.
I'm in a similar funny scenario: the professor told us to make a matrix solver, so I started by making a program that takes a solution, generates a matrix, and feeds it to the other program to solve it, and then I check if I came full circle. I like how paradoxical it sounds from afar
Wow, thank you! It's really impressive not only that you can explain the structure of TCP packets, but that you can just whip together an API that implements it
great stuff! I have just one question: why put f.previous into the FrameReader struct as opposed to stroring it in a local variable in the Read method just outside of the loop, since you're not using it anywhere else?
@@TheVimeagen In other words this means that a command may span over multiple frames (i.e. TCP packages), right? You also said this in the video. What is confusing to me is that inside of conn_Next() (one call of which corresponds to one command) you're only calling c_Read once. Because of that I'm not exactly sure I understand how this works in case you need to do multiple calls to read for one command. The only potential explanation I have is that the TCP Reader abstracts away from you the actual structure of TCP packets and just Reads every time you ask it to, so you effectively: -hand a 1024 byte scratch to FrameReader_reader (which is net_Listen behind all the interfaces) -check if what it wrote there combined with t_previous resembles a complete VimWithMe Packet -if yes (n>0) copy the result to data bytes[] -othewise just do another iteration of f_reader_Read But in this case, again, I'm not sure why would you need to access t_previous from a different FrameReader_Read() execution.
@@TheVimeagen dang it, no matter what I do, it seems that youtube deletes my reply every time :( it's only visible with the "sort by newest first" view
God damn. I implemented the exact same simple message structure a month ago but didn't put a version number at the front... I also put a one byte addative checksum at the end, was that redundant with TCP?
gonna be that guy...but aCkChYuAlLy it's TCP's MSS that determines the segment size...still very much subject to MTU at the L2/Ethernet layer and any other encapsulation headers involved though
Less overhead if you don't need the features of HTTP. More freedom to implement your own communication protocol if your application doesn't behave like a normal http web application.
TCP is a terrible protocol to use for real time applications such as games. Tho with version control it may work a little better, but what happens when your packet version goes 1,2,3,5,8,6,4,7? Will it play 8 and know to toss everything before or will it wait a X number of ticks before tossing? The reason UDP is used is because relying on the internet for stability is stupid think, and a lot of developers say "well we're in 20xx the internet should be fine I'll use tcp" Wrong... internet sucks and need to build your code to handle missing, slow, jumbled packets of data and to do that in real time its easier and faster to do it with UDP.
@@TheVimeagen I agree with the time statement. I'm not a full on developer so I don't have a direct solution. I am an SRE by trade and just know from studying this issue for performance reasons, that TCP protocol if missing packets will start to snowball into a starve situation. I really like your packet version control, i think that will significantly improve your efficiency. I would say don't change anything just think through the issues that might arise if 4000 clients miss 1 packet or are out of order what does your code do. And we'll see how it works day 1. Not to play puns but a quick solution to this could be implementing communications via QUIC-go which is googles UDP implementation. Again i don't know from the developers side how hard/quick (*tish*) that would be. Or I'm just an idiot and nothing matters anyway, keep up the great work prime!
quic is very difficult to implement and remember (which i didn't explain well) that i have neovim as my client. which means i _just have_ tcp or udp i would have to build parts of quick into neovim via lua. no thanks
Ah yes, I did not realize you were using neovim as your client. My mistake. Integrating quic into neovim sounds nightmareish. Quic is UDP based at least 😄
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
This is exactly the type of videos I was hoping to see after I saw you left Netflix. Your frontend masters DSA course showed you're a great teacher and obviously from watching your stream you're super knowledgable so it's the perfect storm for this type of content
YESSSSS
Where is his course?
Look at that, an actual Ex-FAANG Ex-Netflix engineer teaching actual concepts and good materials at that
Finally, no random 1-2 year ex-F or tech leads teaching some random leetcode, useless non-context system design question, written from their official pass-interview books, or tutorial stuffs, and "constantly unintentionally" mention it can get you into Big tech like them.
I was expecting someone from Big tech to fill the gap for this type of contents, and truly show the ability of Big tech engineer (like how we admire their technical ability in the old time besides salary), but didn't expect it took such a long time to have one like Prime.
Netflix btw
Modern computer network protocols and infrastructure is one of the greatest achievements of the human race.
agreed
Looking foward for videos like this
Banger content. Learned heaps in under 9 minutes, you don't get that everywhere. Keep up the good work!
I took networks in uni last semester and made a couple of tcp and udp programs for class. this clears up the gap in my understanding a lot. Keep these videos up ex faang king
What I love about this is the intersection between theory and practice. I’ve learned bits about networking and specifically TCP before - but didn’t really know how it would work in practice. Just seeing how you turn the theory into code really demystifies it. Great content!
You teach exactly how my brain needs to receive it to learn apparently. SO glad you’re doing this full time now
Loving these videos. I don't have the time/focus to tune in to the livestream so would sometimes skim the vods for these insights, but this is everything I could wish for.
This was exactly what I was looking for, this was great. I'd take even longer form content on this info, like 20-30 mins and more working examples.
Still this was awesome thank you for sharing all this info
Love this deeper explanation style of video beside the main channel type of videos!
Low Level Learning looking good with that mustache
I like this new prime. Coconut oil shampoo is doing this man a favour.
does this work with coconut shower gel, too? i have some of that..and need to learn more about coding! :)
Please continue doing these deeper and more technical videos, I'm learning so much!!
Love this, Prime. More of this kind of thing please, it's excellent.
Great explanation about the tcp package, I watch on streaming and watch again to understand more, really clever!!
love this sort of video from you
That’s huge! Nice content, Prime!
Videos like this make my drive hard
This is bloody great!
great stuff! I like this format a lot
I usually prefer to create a state machine to parse each byte received and generate a callback when the data is finished/available.
Therefore working more asynchronous
I was thinking about this earlier today. Man the algorithms are good!
this is what i wanted to see. love too know more about protocols
Love this type of content from you
these kind of videos > netflix btw
I'm actually about to publish a Go package to simplify (?) coloring -and formatting - text. Basically you toss a string and a hex color code and get back the string with the ANSI codes necessary for the formatting based on system support (true color of xterm). Don't know if that would help you but anyway. I just need to make a tiny change to one of the functions and I'll be done and hopefully published by tomorrow
Oh yeah, this format is great, are you still keeping the vods up on youtube as well?
Great video!
Quick question, would it be possible for you to upload coding related vods here for members only?
Yoo this format is actually fire, but is there a place where I can watch the entire stream as you code ?
so if I understand, TCP handles the sequence of the data, HTTP handles the version number and the length of the data, and HTTPS makes it so, even if someone is listening to the messages that are passed around, even if he can make a guess what the contents of the packets are, he still can't tell if he guessed right. Right?
There are some things that are unencrypted when it comes to TLS and HTTP
There are some things that are not.
There are now more and more secure algorithms that are being used. Secure DNS as an example
TCP handles the guarantees that you'd want to have to deem your communication is reliable.
It guarantees in-order delivery, retransmission of dropped packets, and de-duplication of packets.
HTTP is concerned with the contents of the communication between two parties. Like, if I introduce myself "what's your name", convention is to respond with your name and maybe a gesture like a handshake. In the same vein, there are always required headers from the client like the HTTP method, and even if the server doesn't respond with a body, it still needs to respond with a status code like 200, 303, 404, 500 so the client ( usually a web browser ) knows how to process that. In order for clients and servers to be able to understand each other, there have to be some defined elements that are always present in each message, and HTTP governs what those parts are. In other words, HTTP defines how what plain text communication should be sent between two parties in order to have a meaningful conversation.
As for HTTPS, that just means your HTTP connection is using TLS ( transport layer security). One of the 3 reasons is the one that you mentioned; if someone is packet sniffing it doesn't matter because the message has been encrypted. The other two parts is for clients to be able to trust that the server they're making a connection with IS ACTUALLY the entity that they say they are ( what good is encrypting messages if I'm unknowingly connected to a bad website. The last part of TLS is that you have guarantees that your message wasn't tampered with ( think of the tamper seals on food, for example ).
The short version of this is:
TCP: how do we deliver data "reliably" over an unreliable medium
HTTP: what kind of structure / form should our text messages between us take?
HTTPS (TLS): how do we make this data transfer secure?
I reckon I'd watch the 3 x longer version of this
i mean technically speaking, the purpose of conway's game is that the next state in the game can always be calculated given the previous state. so you dont need to send the entire world state of the game to every person, just the starting state. You could probably find a way to only send 'update' bits telling each connected client to update their game at the same time.
Yes, but we are solving it generally. Sending diffs down
I'm in a similar funny scenario: the professor told us to make a matrix solver, so I started by making a program that takes a solution, generates a matrix, and feeds it to the other program to solve it, and then I check if I came full circle. I like how paradoxical it sounds from afar
Wow, thank you! It's really impressive not only that you can explain the structure of TCP packets, but that you can just whip together an API that implements it
wow, I was looking for sending binary data in websockets, and this is great!
please more videos like this
So hyped about this content!!
Cool stuff, thanks for sharing your knowledge.
great stuff! I have just one question: why put f.previous into the FrameReader struct as opposed to stroring it in a local variable in the Read method just outside of the loop, since you're not using it anywhere else?
Because you may read more than one frame, I need to keep storing whatever remaining data is left until the next call to read
@@TheVimeagen In other words this means that a command may span over multiple frames (i.e. TCP packages), right? You also said this in the video. What is confusing to me is that inside of conn_Next() (one call of which corresponds to one command) you're only calling c_Read once. Because of that I'm not exactly sure I understand how this works in case you need to do multiple calls to read for one command.
The only potential explanation I have is that the TCP Reader abstracts away from you the actual structure of TCP packets and just Reads every time you ask it to, so you effectively:
-hand a 1024 byte scratch to FrameReader_reader (which is net_Listen behind all the interfaces)
-check if what it wrote there combined with t_previous resembles a complete VimWithMe Packet
-if yes (n>0) copy the result to data bytes[]
-othewise just do another iteration of f_reader_Read
But in this case, again, I'm not sure why would you need to access t_previous from a different FrameReader_Read() execution.
@@TheVimeagen dang it, no matter what I do, it seems that youtube deletes my reply every time :( it's only visible with the "sort by newest first" view
God damn. I implemented the exact same simple message structure a month ago but didn't put a version number at the front... I also put a one byte addative checksum at the end, was that redundant with TCP?
In tcp yes
@@TheVimeagen Many consumer lever routers fuck up the checksum though.
videos like this = good stuff.
So here is my question: Du you send the whole board every time, or just on connect and after the only the diffs?
Michigan Tech mentioned!
This is amazing. I'm stuck in CRUD land and wanna learn something new.
this is awesome prime
Hell ya thanks for explaining
this is prime content
love this, do more!
Am i getting educated here? noice
I think I accidentally may have made an educational video
gonna be that guy...but aCkChYuAlLy it's TCP's MSS that determines the segment size...still very much subject to MTU at the L2/Ethernet layer and any other encapsulation headers involved though
isn’t there the handshake?
Looking at TCP packets as an ex-FAANG engineer.
Where to get its source code?
dumb question - can you explain the benefit of working with TCP directly instead of making HTTP requests?
HTTP is the application layer. TCP is the underlying protocol that allows HTTP function. So basically HTTP is TCP with extra steps
Less overhead if you don't need the features of HTTP. More freedom to implement your own communication protocol if your application doesn't behave like a normal http web application.
This is great!
love this
So cool to see how excited you are to build this. But why not UDP? Kappa
Nice video!
how much toilet paper rolls have you stock piled?
Why would you send color ? If you send the type of object in the board. the game already knows the color of that object.
Just curious. Shouldn't videos like these go into the PrimeTime channel?
PrimeTime is more for his reactions videos and such. This is more of a coding showcase. I can see why he wouldn't want to upload this on PrimeTime.
Well this vid was epic
Good video!
what drawing software is this?
Gimp
Flip forget to zoom in
Can't believe this is free
the teacheagen
Now i need to make my ownt TCP protocol :v
Oh boy dis is gut
Ok I won't unsubscribe
Get a graphic tablet!
i understood it even though am dumb
TCP is a terrible protocol to use for real time applications such as games. Tho with version control it may work a little better, but what happens when your packet version goes 1,2,3,5,8,6,4,7? Will it play 8 and know to toss everything before or will it wait a X number of ticks before tossing?
The reason UDP is used is because relying on the internet for stability is stupid think, and a lot of developers say "well we're in 20xx the internet should be fine I'll use tcp"
Wrong... internet sucks and need to build your code to handle missing, slow, jumbled packets of data and to do that in real time its easier and faster to do it with UDP.
No one is arguing with you, I just have limited time
@@TheVimeagen I agree with the time statement. I'm not a full on developer so I don't have a direct solution. I am an SRE by trade and just know from studying this issue for performance reasons, that TCP protocol if missing packets will start to snowball into a starve situation. I really like your packet version control, i think that will significantly improve your efficiency.
I would say don't change anything just think through the issues that might arise if 4000 clients miss 1 packet or are out of order what does your code do. And we'll see how it works day 1.
Not to play puns but a quick solution to this could be implementing communications via QUIC-go which is googles UDP implementation. Again i don't know from the developers side how hard/quick (*tish*) that would be.
Or I'm just an idiot and nothing matters anyway, keep up the great work prime!
Looooooove it
But why not UDP?
he said on stream that udp takes too long to code and he wants the game to be up soon and tcp is just easier
he literally says why at 1 minute into the video 🤦♂
@@JustSomeAussie1 it was a joke
@@darouigougoui2969 it was a joke
@@JoshuaMoreno yeah I thought so, but I wanted to leave my comment just in case
Butchered tcp
Ok sure by why not try QUIC?
Quic-go for Go .. or Quiche for rust
quic is very difficult to implement and remember (which i didn't explain well) that i have neovim as my client. which means i _just have_ tcp or udp
i would have to build parts of quick into neovim via lua.
no thanks
Ah yes, I did not realize you were using neovim as your client. My mistake.
Integrating quic into neovim sounds nightmareish.
Quic is UDP based at least 😄