I Wrote HTTP "From Scratch" (It Was Easy)

Поделиться
HTML-код
  • Опубликовано: 21 ноя 2024

Комментарии • 115

  • @lukeh990
    @lukeh990 4 месяца назад +346

    You know it’s going to be a good video when it starts out with: “We’re going to assume the presence of a TCP/IP implementation on your system”

    • @Ginto_O
      @Ginto_O 4 месяца назад +1

      well i mean it is preinstalled on windows but you know how to install it right? in network properties

    • @0w784g
      @0w784g 3 месяца назад +1

      Dag nabbit, here's me still using token ring networking.

  • @Q_20
    @Q_20 4 месяца назад +178

    I read that as C written in HTTP, and was hoping for next level black magic shit.

    • @seanbix5366
      @seanbix5366  4 месяца назад +47

      If anybody is determined to bless the world with this, I'll help where I can!

    • @vpd825
      @vpd825 4 месяца назад +1

      Same here 😂 This is still cool.

  • @shinej11
    @shinej11 4 месяца назад +35

    I spend most of my time with Python, with all the layers of abstraction. This video definitely rekindled my love for C. Thank you.

    • @seanbix5366
      @seanbix5366  4 месяца назад +15

      @@shinej11 Python's is imo the least evil scripting language, and can do some low level stuff too.
      I once implemented some serial networking protocols in Python on a microcontroller.
      Use the right tool for the job and have fun!

    • @gregh5061
      @gregh5061 3 месяца назад

      @@seanbix5366 You probably won't see this, but what C projects would you recommend a Python dev work on to get a good grasp over C? I have nothing specific I want to do with C in mind, which is why I ask this.

    • @seanbix5366
      @seanbix5366  3 месяца назад +6

      ​@@gregh5061Something that motivates or excites you personally.
      If I worked primarily with Python and didn't hate what I do, I would try rewrite some of that work in C.
      As an IT person by trade, I've rewritten high level scripts (including python) that could benefit from a performance boost in C just to learn and I think it's a fun, practical and quick way to pick up the nuances.
      If that's too daunting,
      then it's back to basics with data structures, algorithms (and books) imo.
      Maybe in a library you
      can reuse and improve!

  • @shreeyashpandey3530
    @shreeyashpandey3530 4 месяца назад +40

    That's a very creative way to use gimp as a presentation tool!
    hella based video!

  • @syedsaifuddin1413
    @syedsaifuddin1413 4 месяца назад +20

    Ahh very good explanation good sir.
    I was getting into c socket programming without understanding the underlying fundamentals to implement this protocol.
    Gained a lot of information from this one.

  • @felixkuhlmann9985
    @felixkuhlmann9985 3 месяца назад +1

    Very concise and thorough introduction to the application of those theoretical ideas and protocols, most college classes lack this practical approach. Thank you!

  • @DailyFrankPeter
    @DailyFrankPeter 4 месяца назад +5

    In the worlds of Linus Torvalds - let's write it from scratch; how hard can it be!

  • @davidrieger3160
    @davidrieger3160 2 месяца назад

    Dude this is such a good video. I hope you are a uni professor.
    Would love to see an implementation of TCP as well

  • @AngelsFishingRod
    @AngelsFishingRod 2 месяца назад

    Taking Fundamentals of Data Comm rn and this video is so goated to actually understanding the shit im learning

  • @JeremyAndersonBoise
    @JeremyAndersonBoise 4 месяца назад

    This is a great practice project for programmers. Very cool moves.

  • @darkshoxx
    @darkshoxx 4 месяца назад +7

    F*ck Powerpoint, we're using GIMP 😆! Excellent content

  • @impaglg
    @impaglg 4 месяца назад

    Thank you for this video Sean! it really was eye opening.

  • @michelhertwig
    @michelhertwig 3 месяца назад

    very good video! please go on with your important and solid work. good job! very good!

  • @elgs1980
    @elgs1980 3 месяца назад

    What can be more satisfying!

  • @DanielPopsuevich
    @DanielPopsuevich 4 месяца назад +2

    Execllent video! Just what I was looking for. I recommend thinking about a better title in future in order to attract more viewers to your content since this video is a fantastic deep dive, you explained protocols in less than a minute visually.

  • @androu-f3b
    @androu-f3b 2 месяца назад

    Nice video and very clear explanation! For some reason I thought http is much more complicated.
    One question I have, is there a specific reason you search for "GET /" in the request and not just "GET"?

    • @seanbix5366
      @seanbix5366  2 месяца назад +1

      @@androu-f3b Because here I'm checking if the client is specifically getting "/", the root directory.
      Its the GET request the server receives when they (the client) just type in " IP_ADDRESS/".
      Its rudimentary but eh it works.
      Otherwise it's "GET /SPECIFIC_RESOURCE".
      Thank you!

  • @AndrewNijmeh
    @AndrewNijmeh 4 месяца назад +1

    this is how real men program 🙏🏼

  • @kanha12297
    @kanha12297 3 месяца назад

    Very informative video ❤ loved every bit of it

  • @artaxerxes360
    @artaxerxes360 3 месяца назад

    For a need like me this is treasure.

  • @SalirDeMatrix
    @SalirDeMatrix 4 месяца назад +2

    You are a good developer. 👍

  • @xcelestialdemon8245
    @xcelestialdemon8245 3 месяца назад

    Every time you said "haych" and angel lost its wings

    • @seanbix5366
      @seanbix5366  3 месяца назад +1

      You couldn't fathom the width of the brush I sweep them up with.

  • @utuberrocks-q5j
    @utuberrocks-q5j 4 месяца назад +4

    How different is this from the binary HTTP protocol? And can you leverage your code into that easily?

    • @seanbix5366
      @seanbix5366  4 месяца назад +7

      I literally googled HTTP RFC and immediately started writing the code. I was roughly today years old when I saw there was http 2 and 3 so I don't know yet 👉👈
      I've heard they're the same protocol, but with a binary format and extra features, so it should be doable!

    • @petertriller8343
      @petertriller8343 4 месяца назад

      @@seanbix5366 http2 and 3 are a lot more complex, you can have several transfers simultaneously over the same TCP connection and weird stuff like that and http3 uses UDP / Quick as underlying network protocol. So don't count on being able to reuse too much

  • @whonehuljain
    @whonehuljain 4 месяца назад +3

    Great content, also amazing explaination!

  • @SamirPatnaik
    @SamirPatnaik 4 месяца назад +10

    S+ tier content

  • @ahmadgazzz
    @ahmadgazzz 3 месяца назад

    This is awesome, but i kept getting thrown off because of the audio

  • @s14vc
    @s14vc 3 месяца назад +1

    How did you were able to actually implement the theory of networks? I just saw a course in college about OSI model but it was almost entirely theorical, what did you have to learn?

    • @seanbix5366
      @seanbix5366  3 месяца назад +4

      TCP/IP is typically implemented as a part of the OS kernel.
      If you're not using an OS, e.g. in embedded (very very cheap) systems, software libraries will implement the lower level protocols for you (up to level 4 is common).
      Here, I've used the TCP/IP implementation in Linux ("Berkeley Sockets API") to implement HTTP (although I could have used a library for that too).
      TL;DR: I didn't!

  • @WomboBraker
    @WomboBraker 2 месяца назад

    ok ok at 2:30 coup de grace, i subscribed gj

    • @seanbix5366
      @seanbix5366  2 месяца назад

      Thank you, more stuff on the way

  • @TodorTsankov-vz3sd
    @TodorTsankov-vz3sd 4 месяца назад +1

    writing HTTP 😇 vs TCP 👹

  • @fabienWendpanga
    @fabienWendpanga 4 месяца назад +1

    Very interesting! Now, i am wondering as a c# developer is it something i cam replicate? Also do you have it on a repository ? Thank you very much

    • @seanbix5366
      @seanbix5366  4 месяца назад

      Yes, and yes.
      This is "socket" programming, the operating system's abstraction of TCP/IP which you can do in C#.
      That's how you would send the HTTP header string and data.
      This is purely for learning purposes though, it's not massively practical or idiomatic.
      C# has easy HTTP interfaces.
      The project is linked in the description!

  • @prashlovessamosa
    @prashlovessamosa 4 месяца назад

    Wow great learnt a lot keep uploading good stuff.

  • @justcurious1940
    @justcurious1940 4 месяца назад +1

    Is the operating system involved in the TLS protocol or the OpenSSL library will take care of encryption and the decryption of the data and then just use the send and recv functions provided by the OS ?

    • @seanbix5366
      @seanbix5366  4 месяца назад +1

      That's abstracted away by the library such that it's simple.
      The operating system is involved in the protocol, such that the library that implements the protocol, interfaces the operating system for various utilities to assist it in doing so.
      Encryption algorithm: library code
      Reading a certificate file: OS code
      So yeah its more than likely that it (openssl code) encrypts the data before just sending it over send().

    • @justcurious1940
      @justcurious1940 4 месяца назад +1

      ​@@seanbix5366 Thank u for clarifying this. Good luck with what ever u are doing today or u are planing to do tomorrow 🙃.

  • @minma02262
    @minma02262 4 месяца назад +2

    Try http2 and http3. Very nice content 👍👍👍

  • @bennguyen1313
    @bennguyen1313 4 месяца назад +3

    Any thoughts what would be involved in writing an ESP32 application, that takes data from the uart, and passes it wirelessly either bluetooth (SPP) or via Wifi. For Wifi, I imagine the ESP32 could act as a web server (TCP) and display the RS232 data, but maybe there's a UDP way it could send data to a PC on the same network?

    • @seanbix5366
      @seanbix5366  4 месяца назад +3

      (Edit: I got a job writing ESP32 firmware)
      Yeah totally, you can do a webserver.
      Arduino libraries for ESP32 have you mostly covered either way; UDP, TCP or HTTP.

  • @s1ckret
    @s1ckret 4 месяца назад +2

    Great work, man!

  • @singhyuvraj122
    @singhyuvraj122 4 месяца назад +3

    Great Video, Please more tutorials on Live coding of Email Server, HTTP and TCP /IP in C from scratch

    • @seanbix5366
      @seanbix5366  4 месяца назад +1

      @@singhyuvraj122 I won't be doing any TCP/IP implementation until I end up working more in embedded and need to make one. An email server sounds rad though!

    • @MarkHall-cf6ji
      @MarkHall-cf6ji 4 месяца назад

      ​@@seanbix5366i think sockets allow you to send raw link-layer packets (ethernet frames) so you can probably implement TCP/IP from userspace

  • @Alfred-Neuman
    @Alfred-Neuman 4 месяца назад +1

    Next step is to write HTML from scratch, and then Javascript...
    Good luck! 😀👍

  • @Mikee512
    @Mikee512 4 месяца назад +2

    Easy in C*
    *Except for dynamically sized strings. :P

  • @MadMatty72
    @MadMatty72 4 месяца назад

    La-mans talk, live it

  • @Gersberms
    @Gersberms 2 месяца назад

    Thanks for saying TCP/IP doesn't follow the OSI model. So many people get this wrong!

    • @seanbix5366
      @seanbix5366  2 месяца назад

      @@Gersberms I was taught in school that TCP+IP+HTTP = OSI but saw in books that its distinct!

  • @andregeraldo6556
    @andregeraldo6556 4 месяца назад

    just got smarter watching this lol, good video

  • @bladekiller2766
    @bladekiller2766 4 месяца назад

    Isn't this simple HTTP server, that parses the requests and returns some response?
    Most of the backend frameworks already have this functionality implemented.

    • @seanbix5366
      @seanbix5366  4 месяца назад +17

      @@bladekiller2766 Oh for sure.
      Dead simple, its in the title.
      If you want to actually be productive, you can always grab a library/framework for whatever.
      But now I don't have the third party dependencies, and I learned the protocol inside and out!
      From my experience- there's always an underestimated leap from thinking you understand a technology, to actually being able to implement it which can be fun to discover.
      Plus this is targeted more towards IT people, who often overestimate a lot of complexity in the tech they're using.

    • @bladekiller2766
      @bladekiller2766 4 месяца назад

      @@seanbix5366 Couldn't agree more.
      All the tech stacks and networking complexity looks extremely complex, but when you try to implement it from scratch by yourself you will suddenly realize that it's doable and it will give you insights on why some things are the way they are.
      Props for doing it in C :)

    • @ゾカリクゾ
      @ゾカリクゾ 3 месяца назад

      @seanbix5366 now implement ipv6 😛

  • @iSam36O
    @iSam36O 4 месяца назад

    moonfly gang 🔥 great video btw, keep them coming!

  • @energy-tunes
    @energy-tunes 3 месяца назад

    how long did this take you? is the original spec long?

    • @seanbix5366
      @seanbix5366  3 месяца назад

      @@energy-tunes original spec is linked.
      I implemented a working subset I needed in like 2-3 coding sessions.
      The hardest part was probably MIME types and understanding + writing CORS.
      What took longer was just making a useful interface to the Linux socket API...

  • @grenadier4702
    @grenadier4702 4 месяца назад

    Maybe I didn't notice but have you tried using non-blocking event-driven i/o? As far as I remember it's the fastest way to handle requests

    • @seanbix5366
      @seanbix5366  4 месяца назад

      @@grenadier4702 All of my sockets are "non blocking", but they block anyways (lmao) because each connection is accepted in its own thread. TCP multicasts make full use of non blocking IO on one thread through.
      All file reads are lazy loaded and cached into memory with mmap so its eh, good enough on that front I guess.
      Not a hash map or event queue in sight, just a simpleton, liberally mutex locking.

    • @grenadier4702
      @grenadier4702 4 месяца назад

      ​@@seanbix5366 Yeah, I see now. Here, right?
      while (er == RECV_TRYAGAIN) {
      er =
      tryAcceptConnection (localhost, remotehost);
      }
      You're basically looping until an error or a connection arrival?
      And here in each thread you just check for blocking and repeat the same loop again?
      else {
      if (errno == EAGAIN || errno == EWOULDBLOCK) {
      continue;
      }
      fprintf(stderr, "
      Socket error, closing connection...
      ");
      closeConnections(args->remotehost);
      }
      But why do you need non-blocking sockets here at all? When you accept a connection, if it would block, you repeat the same loop again so it's the same as a blocking socket. In second code block, it;'s the same thing: even if it's non-blocking, you do the same loop, i.e. waiting for data arrival, blocking other tasks in your thread pool
      Maybe I misunderstood your code?
      UPD: fuck, i realised you wrote "but they block anyways", sorry :) But the question is why did you use non-blocking sockets at all, then?

    • @grenadier4702
      @grenadier4702 4 месяца назад

      @@seanbix5366 oh gosh, freaking youtube deleting comments
      I checked out your code.Why did you use non-blocking sockets in the first place? Because I see even if errno is EWOULDBLOCK, you still iterate over and over waiting for data to arrive or send

  • @timemanager3239
    @timemanager3239 Месяц назад

    Goat

  • @jeffsad8391
    @jeffsad8391 Месяц назад

    I have a question:is good c++ or python for begginners?

    • @seanbix5366
      @seanbix5366  Месяц назад

      @@jeffsad8391 I made the mistake of starting with C++. Useful language but bad for learning imo. Python is good to become productive quickly, but plain C has been the most fun for learning for me. Knowing memory management well is key to avoiding stress when learning programming concepts imo.

    • @jeffsad8391
      @jeffsad8391 Месяц назад

      @@seanbix5366 is good like moving from python to c++?

    • @jeffsad8391
      @jeffsad8391 Месяц назад

      @@seanbix5366 but is good like to move from python to c++?

  • @truthybyte
    @truthybyte 2 месяца назад

    HTTP: ☺
    HTTPS: 💀

  • @ArtieOddity
    @ArtieOddity 4 месяца назад

    Please Do Not Throw Sausage Pizza Away

  • @xorxpert
    @xorxpert 4 месяца назад

    TCP/HTTP is easy, then I went too hell with WebSocket but I pulled through lol, but I hate those frames.

    • @seanbix5366
      @seanbix5366  4 месяца назад

      @@xorxpert Nice!
      I have a websocket deepdive on this channel too- the variable length length encoding and payload masking were wicked, but worth it in the end!

    • @xorxpert
      @xorxpert 4 месяца назад

      @@seanbix5366 It was mostly challenging for me at the time. I usually like to learn how things work for experience but mostly avoiding third party or standard libraries. Whilst working on a multiplayer game, I built my own websocket server & client in the process along with a protocol library, for handling sending packets efficiently properly writing/reading and parsing data (binary), while converting between respectable objects. latency was very important, all unmanaged code. Performance, speed, and efficiency was the focus. Per protocol standard for large payloads, you have to deal with splitting the data into frames, and you know TCP, you got to handle waiting and acknowledging packets - multi (safe) threaded 😅
      Took me about a mouth until i was entirely finished it, though was fun and worth learning!

    • @seanbix5366
      @seanbix5366  4 месяца назад +1

      @@xorxpert So far none of my websocket payloads are larger than a few dozen bytes... But I was under the impression that I could simply cram an arbitrary amount of data after a websocket header and TCP would eat it?
      I don't remember, need to look at the code! I fetch all the BIG data with http from JS anyways.... Good work!
      I couldn't even find a simple websocket library anyways.

  • @kamertonaudiophileplayer847
    @kamertonaudiophileplayer847 3 месяца назад

    Cool, thank you.

  • @KangJangkrik
    @KangJangkrik 4 месяца назад

    Now try SSL implementation and let's see if u can handle it

    • @seanbix5366
      @seanbix5366  4 месяца назад +2

      Hah!
      Nice try, Sense Of Child-like Wonder!
      Unfortunately, I want to finish the project and get it working.
      SSL/TLS is a whole new can if worms for another time...

  • @michaelrenper796
    @michaelrenper796 4 месяца назад

    I wrote HTTP from scratch in 1997 in Java. I was even easier.

  • @shiyunze7157
    @shiyunze7157 2 месяца назад

    gold

  • @gatogordo4131
    @gatogordo4131 3 месяца назад

    I'm a fraud as a programmer, that's a real pro

    • @seanbix5366
      @seanbix5366  3 месяца назад +2

      Oh no, I'm very much just a guy.
      You'll ask me what an abstract factory is and I'll say "sorry I don't listen to stoner rock".
      Go spend a few days writing some C or ASM, read some books, convene with the machine spirits, and you'll be way ahead of me for sure if you weren't already!

  • @TomZeta-ik8tb
    @TomZeta-ik8tb 2 месяца назад

    same bro

  • @magibai
    @magibai 4 месяца назад

    You Sir, deserve a subscribe

  • @petertriller8343
    @petertriller8343 4 месяца назад +1

    http is so easy, and that is why I hate http2 and http3. they ruin the elegance of http/1.1

  • @bonbonpony
    @bonbonpony 4 месяца назад

    Dude! Ever heard of scope? As for someone who claims he wrote this code himself, you talk surprisingly few about the code, and surprisingly lot about unrelated deeper layers of protocols that you don't even attempt to implement yourself, and which are therefore totally outside of the scope.

    • @seanbix5366
      @seanbix5366  4 месяца назад +2

      Fair.
      I suspected the overlap between between people who don't know HTTP and how the TCP/IP stack works was near 1:1, and targeted that audience.
      The implementation itself is not impressive (it's in the title) but it seems to have served it's purpose in giving lots of people a deeper intuition for the entire system (and powering my webserver)!
      I'm sure we can agree that conventional explanations (like wrapping a parcel) fall short.

  • @Awwe12675
    @Awwe12675 3 месяца назад

    Zoom

  • @kaushikkundu
    @kaushikkundu 4 месяца назад

    Cfbr

  • @ReligionAndMaterialismDebunked
    @ReligionAndMaterialismDebunked 4 месяца назад +2

    First. Hehe. Shalom. Yeah, my nose is congested, too. I barely get sick as a vegan that exercises often, meditates, hydrates enough (unlike 75% of Americans and so forth), sleeps enough, good amounts of nutrients, breath work, fasting, etc.

    • @seanbix5366
      @seanbix5366  4 месяца назад +2

      I'm sorry to hear that

    • @MercerBray
      @MercerBray 4 месяца назад

      I have a soup recipe that'll fix you right up if you wanna hear it!

    • @stefanalecu9532
      @stefanalecu9532 4 месяца назад

      You forgot this isn't Reddit

    • @FRANKONATOR123
      @FRANKONATOR123 3 месяца назад

      @@stefanalecu9532yeah lmao like who asked 😂

  • @atomictraveller
    @atomictraveller 4 месяца назад

    holmes, i'm asking you now, to stop doing anything from scratch. don't you know?

  • @swojnowski453
    @swojnowski453 4 месяца назад

    It does not matter what you build, but what gets adopted ... Today you can build almost everything but who cares if nobody touches or builds on top of it.

    • @seanbix5366
      @seanbix5366  4 месяца назад +6

      @@swojnowski453 I just want a cool video game for my friends and I, and to learn these technologies along the way.
      I'm building on top of it, I care, and its pretty chill!

  • @MercerBray
    @MercerBray 4 месяца назад +1

    You cover nothing I wanted to know. Your grasp on the subject matter is below that of a hobbyist. This is harmful to the community, basically misinformation. Could you maybe sound less enthusiastic??

    • @seanbix5366
      @seanbix5366  4 месяца назад +1

      Well, what would you like to know my guy?

    • @MercerBray
      @MercerBray 4 месяца назад +1

      @@seanbix5366
      When I made chicken pot pie, the filling was fine but the puff pastry on top didn't puff up though it did cook through. How can I remedy this? The pastry was the store-bought frozen variety. The temperature was 375 °C and I used and egg wash for the glaze.

    • @seanbix5366
      @seanbix5366  4 месяца назад +4

      @@MercerBray 1. Thawing - Avoid an IP Collision
      Proper Thawing: Make sure your puff pastry isn't experiencing an IP (Instantly Pulled) collision by giving it time to thaw in the refrigerator, ensuring no network (temperature) congestion.
      2. Temperature - Keep Your Layer 4 Connections Strong
      Correct Heat: Your oven needs to be set to 375 °F (190 °C). Think of it as keeping your TCP (Thermally Controlled Puff) connections strong. Too hot or too cold and you'll have trouble establishing that reliable connection for puffing!
      3. Baking Setup - Packet Delivery
      Preheated Dish: Preheat your baking dish to prevent any packet loss, ensuring a more reliable delivery of puffiness. And remember, the filling should be hot, so the top layer gets a good handshake with the crust.
      4. Egg Wash - Smooth Communication
      Application: Apply your egg wash gently, just like a smooth handshake protocol, to establish a golden connection without network (pastry) congestion.
      5. Pastry Handling - Avoid Fragmentation
      Thickness: Roll out the puff pastry to about 1/4 inch, keeping it uniform to avoid data fragmentation in the layers, ensuring all packets (layers) arrive intact and puffed.
      6. Pastry Resting - Time for ARP Resolution
      Cooling: Let your pastry rest for a bit in the fridge after rolling it out, allowing the Address Resolution Protocol (puff pastry layers) to map the correct layer addresses (butter) before baking.
      7. Avoid Overhandling - Prevent DDoS (Dough Denial of Service)
      Minimal Handling: Don’t overhandle the dough or you’ll experience a DDoS attack on your puff pastry’s ability to rise!
      8. Ventilation - Manage Network Traffic
      Steam Release: Make small slits in the pastry to manage network traffic (steam), ensuring no data packets (steam) get lost, causing network (pastry) puff loss.
      9. Cover the Edges - Implement Firewall Rules
      Edge Protection: If the edges are browning too fast, cover them with foil as a firewall rule, letting the rest of the network (pastry) puff up securely without packet loss (burning).
      Troubleshooting Tips - Ping Your Pastry
      Temperature Checks: Make sure the oven temperature is accurate, much like pinging to check network latency.
      Proper Layering: Ensure the pastry's layers aren’t merging like a VLAN (Virtual Layers Absent Network).
      Steam Release: Remember, too much trapped steam can cause a denial of service on your puff pastry’s puffing ability!
      Summary
      Thaw properly: In the refrigerator, avoiding an IP collision.
      Correct temperature: 375-400 °F (190-200 °C) for solid TCP connections.
      Preheated dish: Ensures no packet loss.
      Gentle egg wash: For smooth handshake protocol.
      Minimal handling: Prevents DDoS on dough.
      Steam vents: Manage network traffic effectively.
      Edge protection: Implement firewall rules with foil.
      Let these tips serve as your protocol stack for perfect puff pastry puffing!

    • @MercerBray
      @MercerBray 4 месяца назад

      @@seanbix5366 Thanks but could you give the instructions for a 300w microwave oven? Its all I got.

    • @seanbix5366
      @seanbix5366  4 месяца назад +1

      @@MercerBray Ah, you got me.
      The only thing I'll never open source.

  • @timemanager3239
    @timemanager3239 4 месяца назад

    Goat 🦾🦾

  • @ROBOTRIX_eu
    @ROBOTRIX_eu 2 месяца назад