- Reminding that the code is just for fun and learning purposes and not suitable for production :) - As mentioned in the comments here, it is indeed a better practice and more correct/portable to use htons() for the port number rather than swapping the bytes manually like I did in the video - For some reason modern web browsers do not handle header less HTTP responses nicely like wget does (which I used to demonstrate the web server in the video). If you want to adapt the code in the video to work with modern web browsers the fix is simple, change the last argument of sendfile to 47 (length of index.html) and add the following into index.html file: "HTTP/1.0 200 OK Content-Type: text/html Hello" To make this dynamic to file size you can use the stat system call to get the file size before calling sendfile
If I remember correctly, HTTP requires that lines should be terminated with " ", which is Windows line endings, not Unix, so probably this file should be transformed using dos2unix
@@zxenon555 man is literally abreviation of manual. what he is looking at is section 2 of the manual, where is system calls documentation, system calls are subroutines handled by te operative system.
Honestly though, this is an excellent example of security through attack surface minimization. It's totally fine to use linux file permissions as the access control mechanism as long as you use them correctly. Having the program just crash and drop the connection when a file is missing or forbidden is at least a _safe_ behavior and in some applications it's perfectly fine to limit the scope of correct protocol-compliant behavior to just the subset your use-case needs. After all, there's a whole class of exploits this avoids by not engaging in any kind of complex logic to formulate what the 'correct' error code should be for different failure cases.
@@AJMansfield1 I could be wrong, but isn't the idea of making things stateless exactly for this, it's better to avoid managing state with errors, it's better to just let the process crash, and if it was a malicious attempt you won't have to deal with repercussions related to that.
It's a really interesting how modern programmers do coding. Old school programmers took sockets as the last step of programming, but the new generation do the opposite. Thank you for the video, it's really educational.
@@kamertonaudiophileplayer847 I wrote my own http server when I was 18 in cpp to serve my own desktop clients in a cached and non standard way. It was basically a tcp server with http headers to pass through firewalls and proxies at that time and consume minimum bandwidth and hardware back in 2001. It was serving around 4000 concurrent connections at the same time, we just didn’t have more customers to join and test.
While I rarely ever write C or anything low-level tbh, but I wanted to show appreciation for the video format. Is it a tutorial? I don't know, it didn't quite have a "how to do X" vibe, but more of a this is something you can do and here's where you find the docs to do it - I like that. The fact that you show exactly where to find information about what you're using and also explain what you're doing as you're doing it is the perfect tutorial style video I'd like to see. It was also both informative and interesting. So I'd like to say great job! It's a sub from me, I can't wait to see more cool stuff 🙂
Basically its not a "step by step", that's ok because they also included how to look for information, like the manual, usually from there we can figure out how walk, but if someone is looking more in depth, I don't believe this is the video for them.
Given how bloated current software is, how many layers of abstractions on top of each other, writing webserver in C makes lot of sense. Serving a million requests out of one Raspberry Pi would make cloud services cry.
this is impressively descriptive. i really enjoy reading the docs alongside you. i like how you walk us through each step of the script. and i really enjoy the demo at the every end! i could very easily see deploying some cron job to a small server to host a little static website. cheers for capturing my imagination!!
One of the best ways to learn programming is just to roll up your sleeves and sling some code! Make mistakes, fix compiler warnings, learn a new function or build system, etc… Love to see it 👍🏻
i think when you were setting the port in the struct, you can use the htons() function. The flipping of order depends on your processor so if someone with an AMD flipped the bytes manually it would give a different port.
It's nice to see the bare minimum it takes to get a web server to work. It really makes you think about all the other features, why they're there, and how they could be implemented.
this was so good. i have great respect for C, not that i can write it cuz C is scary but i love writing servers and seeing one get written with C in such a rustic way felt wonderful. great vid man.
C is actually pretty simple. "very little" abstraction wich forces u to understand every step made rather than (node example) http.createServer() and thats it. C is a very good language to learn if u want to build a robust knowledge. u can read Dennis Ritchie ANSI C book to get the basics and then go with Linux Programming Interface, wich will show deep on linux and u will be able to see a lot of good C script examples
thats really cool. keep it simple as possible. this is a good perfect base for programming an own webserver. thank you. thats the most minimalistic best example i ever seen.
I program web servers in Golang, and also my first language C++, but I didn't get too far with it. I can't believe how similar this is to Golang, or just how much of this I understand. First of all, I had no idea that terminal programs could be included in C programs, but that makes sense now. And I had no idea Open() was a Linux terminal program that, even more so, functions exactly like it does in Golang! In Golang it's the function os.OpenFile(). I just had no idea I was using what are practically Linux terminal programs and just making calls to them. I don't really have a point and this whole comment might not be very cohesive, but I'm just amazed at what it's like to do this in C.
Indeed the socket API in many languages is quite similar :) notice that those functions are actually technically system calls and not terminal programs, for more information check out my video about system calls explained.
@@nirlichtman I did notice that when I looked at the Golang documentation in the middle of watching your video! Thanks! I subscribed btw and will probably watch more of your videos 😊
Fun fact, C was developed by Dennis Ritchie, for the explicit purpose of creating a language to code UNIX. UNIX was created in collaboration with Ken Thompson. Guess who designed Golang ? Yes, Ken Thompson.
When I was in college I played around with sockets and had a lot of trouble transferring binary files. My code was freeing the socket and moving it to another descriptor, but I wasn't using the sendfile() function. Instead, I was trying to write directly to the socket 😂 Nice channel! Subscribed!
I just discovered you channel and I subscribed immediately! I didn't even know one could man stuff from stdlib! Please keep posting video, it's truly amazing!
OMG I love this channel, especially the vim tips. The video is wonderful. I have never used the "Thank You" button on YT to pay somebody, but I really wish you were monetized so I could do it for the first time.
Lmao this sounded so boring from the thumbnail I enter out of curiosity and boom he gets straight right to the point CHAD, respect to whoever considers others' time 👐👐
Nice concise video. I'm glad you emphasised that this is a LONG way from being production ready though :) For example not checking for NULL from strchr() means if a client sends a really long filename, over 256 chars in this case, it'll crash attempting to assign 0 to the address NULL, obviously also not checking if the file exists. Also the client could request ANY file the user has access to on the entire filesystem, but I understand this is just to show the basic concept :)
Crashing is the least of problems when you develop tools like that. It's basically a netcat that speaks HTTP. You can fail also if file does not exist, but it's all checks for return values of functions that you are using and making it "crash" not horribly, but only for one client request.
in bash/zsh, one can do straight at the prompt: $ *printf "%xn" 8080* to get the hex of 8080 this was a super video, I remember learning C from man pages back in 91 when I was put in charge of a new SunOS Sparc Network... this brought back many happy memories of doing a very similar client/server system for a tokenised command protocol... it was purely for my own brain r&d, but it taught me how useful man pages are, an art I feel is forgotten now just searching for an extant solution is the more lazy way.
(2:40) Why is the byte order reversed for the port? I've seen this a couple times, for example when programming the CANBUS for an arm chip. I never really looked deeper into why that is. Does it have something to do with the endianness of the library? Btw keep the networking videos coming, I love it!
I believe it has to do with the actual CPU architecture that the system is using. When reading bytes and writings bytes from/to memory, CPU uses a specific instruction from its instruction set that depends on the way the CPU was designed. For example, most of ARM CPU's use little endian when working with bytes. If you want a deeper understanding I would suggest you watch Mr. Ben Eater's series on building a 6502 chip-based computer, but specifically this video: ruclips.net/video/yl8vPW5hydQ/видео.html. Around minute 8:10 onwards you can watch a demonstration of little endian and gain an understanding of how bytes and programs are stored on a computer's rom.
I believe the reason is that the TCP packet header (which contains the port number) specifies it to be in network byte order which is big endian. Usually you would use a function like htons to convert an integer from the host platform endianness to network byte order
Thanks @Its_JustNeto. @Mike-gs7eo that's what i thought. Had to double check though, I'm still not really "fluent" in the matter. Getting there though :)
CANBUS is a whole separate CAN of worms when it comes to endianness! The on-the-wire byte order is defined individually for each message type, and they do vary. Strictly speaking CAN messages aren't even a sequence of bytes/octets. They are just a sequence of bits not necessarily a multiple of 8. However, it appears most libraries seem to present the API using some sequence of byte buffers anyway.
This is good for someone who wants to learn networking even tho in reality you will never code C in your job, it gives you a hacker mindset and a way to like a complicated thing for a starter journey throughout the IT world. if you want to be an ethical hacker you have no choice but to like this, even in reality you will code it in RUST or python.
C is still very useful for many things nowadays as well and a lot of people still like using it since it is close to the operating system and has a familiar syntax that inspired many other languages. Even when programming in higher level languages like Python, there are cases in which you may want to write some of the logic in C and call it from Python.
If the GET request has more than 256 chars but no space, then buffer will not be \0 terminated and strchr() will keep reading way beyond the buffer to the first space it can find and write \0 to that address, which will be some stack address, since the buffer is located on the stack. Overriding stack data is one of the most common attacks of all times, about every 2nd vulnerability in existence does that in some form. Avoiding that attack would have been very simple: Just make recv() only read 255 bytes, as then buffer is for sure \0 terminated (it was zero'ed before) and strchr() will never read beyond \0. In that case it would return NULL and the program would crash.
I was following along nicely and then line 24 (at 6:17) blew my mind wide open. Clever way to not assign a variable that will only be used once, never seen that syntax before.
It is cool syntax :) Notice that because the code is for fun and poc and not for production I am skipping many checks so in this specific case I am not handling the case in which the call for strchr would fail - since I am dereferencing the pointer straight away
my compiler takes it as a warning, *incompatible pointer type*, bind should have the parameter 2 as struct sockaddr_in *.... so i think its REALLY a problem :D update: i need to cast struct sockaddr_in* to struct sockaddr* and set len argument for the bind to the size of my struct sockaddr
That is a good point, thought about it after the video, since the port is hardcoded I just did it manually but indeed a better practice is to use htons
Awesome, It would be great to make another one, but this time by managing, for example, the listening and sending parts simultaneously. (I think a simple fork should do the trick ?)
I think it that poll(2) would be better suited for this purpose (it blocks until on of the specified fd is available). But I guess that forks or pthreads work as well
great but how do u know the steps to achieve like from open then close etc. I started programming in c but i want to start beginning projects like these any advice would be greatly appreciated
anyone know how to make the server not end when you connect? I've got mine working through chrome, but I wanna be able to peek at hte index.html file in chrome inspect element, but chrome can't get it because the server has already died :(
Nice way you read the man pages. Btw, can you explain why your system looks to be using dwm but you have a windows-like terminal? Are you using Linux or Windows with WSL?
3 points: 1. This wont work with file names that include spaces. I personally think that allowing spaces in file names should never ave happened. and searching for " ." wont work either as a workaround. Allowing spaces and other characters like newlines in file names makes things very messy. 2. When you compile there are warnings. You need to take care of those and get a clean compile 3. buffer ought to be one byte longer than what you have if you tell recv to get 256 chars then the buffer needs to be 257 chars in length in order to prevent unterminated strings
ive done minimalist web server before. all you really need is sockets and a handshake. then just handle the requests. used to do this using older cellphones to get around the blocks and control my computer. do stupid stuff like download camera photo or send text to speech commands to pc. instead of just handling the requests normally and provide web content back you can just use it as an input and serve back what ever you want.
@@nirlichtman not on phone. this was done in like early 2000's in' visual' basic. phone browsing was very limeted at the time bocking a lot of content. i used the faux webserver to get around those limitations. the host was at home and used the phone browser.
I feel like the sort of example you're going for (basically a SOCKS tutorial) would be better demonstrated with an echo server or something. Same information would have been covered. Just, with a server of any kind, you need to use multiple threads, otherwise, your server will only handle one client at a time.
Excellent video. You could have kept it clean, and saved that hex conversion hack for the port with the relevant call (htons), and that would have been in line with the spirit of the video.
Thanks! Yah it was indeed a mistake to not use htons for the port (I mentioned on the pinned comment), htons is also more portable for other CPUs that order bytes differently.
Cool tip nobody asked for: if you have GNU Make installed you can run `make server` and it will infer that you are trying to compile `server.c` and will run `cc server.c -o server` so you will have your executable actually named after your source code file instead of "a.out".
In this case, I started by reading the overview of the Socket API on Wikipedia and after I got an idea of the general functions and flow I read about the individual functions on the man pages, BTW I also like using the man -k and -K flags for searching through all the man pages, I plan on making a video about tips for searching man pages.
I'm trying to use man like you do but when I use "man socket" the output is different than yours. I'm using ubuntu on windows could this be the issue? I'm quite green about c and linux environment so maybe this will sound such a silly question. Thanks
@@nirlichtman so I did reinstall WSL and everything looks normal now. Unfortunately I did it before read your msg so I'm not able to produce the output in question. Your channel is incredible, resourceful and educational. Thanks for doing this
http 3 actually uses UDP I am actually surprised the http client accepted a response without status code or the HTTP version. I didn't know you can just send the data.
You can't. You need the HTTP version and a status code for the response to be valid per the standard, but browsers may still not accept it unless you include the date, content-length, and either transfer-encoding or connection: close And if you don't include content-type, browsers will have inconsistent behaviour ( Safari for example will assume it's a file download, but Firefox will just render it as plain text )
It's GET since that is the http request type that the browser sends when it retrieves a web page, the buffer + 5 skips over the first 5 characters of the request buffer (which in the case of a GET request, starts with "GET /"), so we can go ahead and extract the path that is requested
Looks like you may be missing the man pages for C Lib (man 3) does "man scanf" work for example? Make sure you have the manpages-dev package (name of package on Debian/Ubuntu)
@@nirlichtmanVery strange, `man scanf` works, but `man sockaddr_in` does not. Just to make sure, I reintalled manpages-dev with the same result. I also get the same result on my mac. Did you perhaps install additional manpages?
@@nirlichtmanI figured out the issue! My manpages-dev was version 5.10 (current is 6.03). Fixed by... adding the debian mirror to my /etc/apt/sources.list allowing insecure installing from the mirror (edit sources.list again) running `apt-get update` running `apt-get install manpages-dev`
I am dereferencing the pointer returned from strchr with * and assigning the location in memory to 0, notice that because the code is for fun and not for production use I am skipping many checks and in this case I am not handling the case strchr fails
@@nirlichtman I think your best option is to record a voiceover afterwards. Do the video first, then narrate it. The bonus there is your voice will also sound better, and you already have a very pleasant speaking voice. Some people love this ASMR style of sound, but a lot of us find it quite stressful/distracting. Thanks for considering it!
- Reminding that the code is just for fun and learning purposes and not suitable for production :)
- As mentioned in the comments here, it is indeed a better practice and more correct/portable to use htons() for the port number rather than swapping the bytes manually like I did in the video
- For some reason modern web browsers do not handle header less HTTP responses nicely like wget does (which I used to demonstrate the web server in the video).
If you want to adapt the code in the video to work with modern web browsers the fix is simple, change the last argument of sendfile to 47 (length of index.html) and add the following into index.html file:
"HTTP/1.0 200 OK
Content-Type: text/html
Hello"
To make this dynamic to file size you can use the stat system call to get the file size before calling sendfile
it makes more sense to send headers separately with send()
@@soniablanche5672 Then we would have to put the headers and the contents of the file in the same buffer and then use send() to transmit it ?
@@tibo1671 I think you do a send() followed by a sendfile() (I found this answer on stackoverflow)
and you need to add Content-Length header. if you don't add this header browsers will say "Connection was reset"
If I remember correctly, HTTP requires that lines should be terminated with "
", which is Windows line endings, not Unix, so probably this file should be transformed using dos2unix
I love this way of showing by opening manual entries of each functions you use in a quick manner! Its very speedy and keeps it interesting ❤❤
Totally agree.
My favorite part, too.
There is standard way of working with linux functions.
I was here to make the same comment
@@zxenon555 man is literally abreviation of manual. what he is looking at is section 2 of the manual, where is system calls documentation, system calls are subroutines handled by te operative system.
Looks bulletproof, let’s ship it! Kidding aside, great tutorial. Nice and concise with just the right level of detail. Keep them coming!
try something like: wget 0.0.0.0:8080//home/$USER/flag.txt
Honestly though, this is an excellent example of security through attack surface minimization. It's totally fine to use linux file permissions as the access control mechanism as long as you use them correctly. Having the program just crash and drop the connection when a file is missing or forbidden is at least a _safe_ behavior and in some applications it's perfectly fine to limit the scope of correct protocol-compliant behavior to just the subset your use-case needs.
After all, there's a whole class of exploits this avoids by not engaging in any kind of complex logic to formulate what the 'correct' error code should be for different failure cases.
@@AJMansfield1 I could be wrong, but isn't the idea of making things stateless exactly for this, it's better to avoid managing state with errors, it's better to just let the process crash, and if it was a malicious attempt you won't have to deal with repercussions related to that.
Looks done on a rush, just like real software patches or production software in general, let’s actually ship it
Just don't GET request any filenames with spaces
It's a really interesting how modern programmers do coding. Old school programmers took sockets as the last step of programming, but the new generation do the opposite. Thank you for the video, it's really educational.
@@kamertonaudiophileplayer847 I wrote my own http server when I was 18 in cpp to serve my own desktop clients in a cached and non standard way. It was basically a tcp server with http headers to pass through firewalls and proxies at that time and consume minimum bandwidth and hardware back in 2001. It was serving around 4000 concurrent connections at the same time, we just didn’t have more customers to join and test.
Great tutorial, no buz words, not trying to sell something, overall just less noise, that's the part of the internet that i love
While I rarely ever write C or anything low-level tbh, but I wanted to show appreciation for the video format.
Is it a tutorial? I don't know, it didn't quite have a "how to do X" vibe, but more of a this is something you can do and here's where you find the docs to do it - I like that.
The fact that you show exactly where to find information about what you're using and also explain what you're doing as you're doing it is the perfect tutorial style video I'd like to see.
It was also both informative and interesting. So I'd like to say great job! It's a sub from me, I can't wait to see more cool stuff 🙂
it's a stealth variation of RTFM
Basically its not a "step by step", that's ok because they also included how to look for information, like the manual, usually from there we can figure out how walk, but if someone is looking more in depth, I don't believe this is the video for them.
I'd definitely say it's a tutorial for someone who's self taught like me. He's included everything I need to read the rest of the information myself.
Given how bloated current software is, how many layers of abstractions on top of each other, writing webserver in C makes lot of sense. Serving a million requests out of one Raspberry Pi would make cloud services cry.
Now THIS is the kind of tutorials I want to see, sooo good
This is an excellent tutorial. No bs, no message from our sponsor, just straight to the point,
it would be great to build this series up to include TLS in C
Nice that's a cool idea, can be an interesting challenge to make an https server in C
it would be a great educational piece there is nothing out there with TLS and C that is comprehensive@@nirlichtman
@@nirlichtman Please do make a video like that if you get the chance! I've been wanting to learn how to implement HTTPS manually for years lol
YESSS please :D
this is impressively descriptive. i really enjoy reading the docs alongside you. i like how you walk us through each step of the script. and i really enjoy the demo at the every end! i could very easily see deploying some cron job to a small server to host a little static website. cheers for capturing my imagination!!
One of the best ways to learn programming is just to roll up your sleeves and sling some code! Make mistakes, fix compiler warnings, learn a new function or build system, etc… Love to see it 👍🏻
i think when you were setting the port in the struct, you can use the htons() function. The flipping of order depends on your processor so if someone with an AMD flipped the bytes manually it would give a different port.
yeah not sure why he didnt use htons
Amd is little endian but network order is big endian so it will still work
It's nice to see the bare minimum it takes to get a web server to work. It really makes you think about all the other features, why they're there, and how they could be implemented.
this was so good. i have great respect for C, not that i can write it cuz C is scary but i love writing servers and seeing one get written with C in such a rustic way felt wonderful. great vid man.
C is actually pretty simple. "very little" abstraction wich forces u to understand every step made rather than (node example) http.createServer() and thats it. C is a very good language to learn if u want to build a robust knowledge. u can read Dennis Ritchie ANSI C book to get the basics and then go with Linux Programming Interface, wich will show deep on linux and u will be able to see a lot of good C script examples
The best introduction to a video on the entire RUclips. I liked the video 1 second in.
You're so quick and you explain it so clearly. That was a really great video! cheers!
thats really cool. keep it simple as possible. this is a good perfect base for programming an own webserver. thank you. thats the most minimalistic best example i ever seen.
I program web servers in Golang, and also my first language C++, but I didn't get too far with it. I can't believe how similar this is to Golang, or just how much of this I understand. First of all, I had no idea that terminal programs could be included in C programs, but that makes sense now. And I had no idea Open() was a Linux terminal program that, even more so, functions exactly like it does in Golang! In Golang it's the function os.OpenFile(). I just had no idea I was using what are practically Linux terminal programs and just making calls to them. I don't really have a point and this whole comment might not be very cohesive, but I'm just amazed at what it's like to do this in C.
Indeed the socket API in many languages is quite similar :) notice that those functions are actually technically system calls and not terminal programs, for more information check out my video about system calls explained.
@@nirlichtman I did notice that when I looked at the Golang documentation in the middle of watching your video! Thanks! I subscribed btw and will probably watch more of your videos 😊
Fun fact, C was developed by Dennis Ritchie, for the explicit purpose of creating a language to code UNIX. UNIX was created in collaboration with Ken Thompson. Guess who designed Golang ? Yes, Ken Thompson.
This makes me want to write a web server in C 😂 very nice and concise tutorial, thanks!
THANK YOU SO MUCH FOR JUST JUMPING IN I LOVE THAT
This is just brilliant, in this short video there is so much to learn.
When I was in college I played around with sockets and had a lot of trouble transferring binary files.
My code was freeing the socket and moving it to another descriptor, but I wasn't using the sendfile() function.
Instead, I was trying to write directly to the socket 😂
Nice channel! Subscribed!
I just discovered you channel and I subscribed immediately! I didn't even know one could man stuff from stdlib! Please keep posting video, it's truly amazing!
OMG I love this channel, especially the vim tips. The video is wonderful. I have never used the "Thank You" button on YT to pay somebody, but I really wish you were monetized so I could do it for the first time.
Thanks! :)
Very nicely done, straight to the point, subscribed after the first minute.
Lmao this sounded so boring from the thumbnail
I enter out of curiosity and boom he gets straight right to the point
CHAD, respect to whoever considers others' time 👐👐
I used this as a reference for making my own small HTTP server, thank you :)
As someone coming from us, all I can say is “amazing!”
Wow. Now _that_ is a good tutorial: not merely telling people what to do, but also showing them how you learned what to do.
Straight to the point, really simple to follow. Thank you.
Nice concise video. I'm glad you emphasised that this is a LONG way from being production ready though :) For example not checking for NULL from strchr() means if a client sends a really long filename, over 256 chars in this case, it'll crash attempting to assign 0 to the address NULL, obviously also not checking if the file exists. Also the client could request ANY file the user has access to on the entire filesystem, but I understand this is just to show the basic concept :)
Crashing is the least of problems when you develop tools like that. It's basically a netcat that speaks HTTP. You can fail also if file does not exist, but it's all checks for return values of functions that you are using and making it "crash" not horribly, but only for one client request.
In 10 minutes you taught me something my professor couldn’t teach our class in 3 weeks
I'm sure he spent more time preparing.
It has been a while since I haven't seen a clean C code like this 😊
If you are in neovim (idk about vim) you can use `Man` (capitalized) command to open manfiles in vim buffers. I hope this can be helpful.
Awesome didn't know about this, thanks! Vim supports it as well :)
I’m going to use this for sure
shift+k to open doc under cursor
wow. i didn't know it was so easy!!!!!! So very cool! Also didnt know that man pages had all this info! Truly amazed!!!
Love it. Reminds me of doing Beej’s tutorials back in the day
This is amazing. Love that you showed how to use manpages
Woah. This video is really helpful. Love u, man.
Amazing way of explaining using manual and step by step approach without any unnecessary stuffs.
Subscribed with All notifications :)
Did i just watched a video writing of a server in C 🥴.
Anyhow i loved the process tmux, how using man pages and all that premium content. Goldmine ❤
Thanks! Actually, I am not using tmux in this video, but rather the built in Vim window splitting.
Love your approach to coding and use of man pages. Thanks for sharing have a great day :-)
i like how you used manuals, never saw anyone else do that in a tutorial lol, and it never occured to me personaly to even try and use them
How do you program in c without man pages ? I have to constantly open them because I don't remember everything
This video taught me two things - making a web server in c, and how little I actually know.
You should have titled it “Writing a Webserver in C: Speedrun”
Great work
in bash/zsh, one can do straight at the prompt:
$ *printf "%xn" 8080*
to get the hex of 8080
this was a super video, I remember learning C from man pages back in 91 when I was put in charge of a new SunOS Sparc Network... this brought back many happy memories of doing a very similar client/server system for a tokenised command protocol... it was purely for my own brain r&d, but it taught me how useful man pages are, an art I feel is forgotten now just searching for an extant solution is the more lazy way.
Very cool! Is it possible to use htons() or a similar function to flip the port number byte order instead of doing it by hand?
Thanks! Yah it is indeed a better practice to use htons for that
Everybody loves hearing your keyboard typing noises!
Yes!
More videos about network programming please
(2:40) Why is the byte order reversed for the port? I've seen this a couple times, for example when programming the CANBUS for an arm chip. I never really looked deeper into why that is. Does it have something to do with the endianness of the library?
Btw keep the networking videos coming, I love it!
I believe it has to do with the actual CPU architecture that the system is using. When reading bytes and writings bytes from/to memory, CPU uses a specific instruction from its instruction set that depends on the way the CPU was designed. For example, most of ARM CPU's use little endian when working with bytes. If you want a deeper understanding I would suggest you watch Mr. Ben Eater's series on building a 6502 chip-based computer, but specifically this video: ruclips.net/video/yl8vPW5hydQ/видео.html. Around minute 8:10 onwards you can watch a demonstration of little endian and gain an understanding of how bytes and programs are stored on a computer's rom.
I believe the reason is that the TCP packet header (which contains the port number) specifies it to be in network byte order which is big endian. Usually you would use a function like htons to convert an integer from the host platform endianness to network byte order
Thanks @Its_JustNeto. @Mike-gs7eo that's what i thought. Had to double check though, I'm still not really "fluent" in the matter. Getting there though :)
CANBUS is a whole separate CAN of worms when it comes to endianness! The on-the-wire byte order is defined individually for each message type, and they do vary. Strictly speaking CAN messages aren't even a sequence of bytes/octets. They are just a sequence of bits not necessarily a multiple of 8. However, it appears most libraries seem to present the API using some sequence of byte buffers anyway.
This is good for someone who wants to learn networking even tho in reality you will never code C in your job, it gives you a hacker mindset and a way to like a complicated thing for a starter journey throughout the IT world.
if you want to be an ethical hacker you have no choice but to like this, even in reality you will code it in RUST or python.
C is still very useful for many things nowadays as well and a lot of people still like using it since it is close to the operating system and has a familiar syntax that inspired many other languages.
Even when programming in higher level languages like Python, there are cases in which you may want to write some of the logic in C and call it from Python.
@@nirlichtman yes, that is true
If the GET request has more than 256 chars but no space, then buffer will not be \0 terminated and strchr() will keep reading way beyond the buffer to the first space it can find and write \0 to that address, which will be some stack address, since the buffer is located on the stack. Overriding stack data is one of the most common attacks of all times, about every 2nd vulnerability in existence does that in some form.
Avoiding that attack would have been very simple: Just make recv() only read 255 bytes, as then buffer is for sure \0 terminated (it was zero'ed before) and strchr() will never read beyond \0. In that case it would return NULL and the program would crash.
Good never imagined someone could do this in the age of express and django
Amazing knowledge mate, thanks for sharing.
I was following along nicely and then line 24 (at 6:17) blew my mind wide open. Clever way to not assign a variable that will only be used once, never seen that syntax before.
It is cool syntax :) Notice that because the code is for fun and poc and not for production I am skipping many checks so in this specific case I am not handling the case in which the call for strchr would fail - since I am dereferencing the pointer straight away
Is that a computer running linux with i3 remote desktoping to a windows computer that is also running linux? xD Great video, enjoyed!
😂 My setup is Windows 10 with a port of dwm for Windows, in this video I am using WSL :) For more information check out the video about my setup
Shall i call you the real Mr.MINIMALIST ?
love your content
Thanks!
Oh yeah fun fact about vim and man pages you can do shift+k in normal mode while your cursor is hovering over a function name to open its man page
This was well presented and extremely helpful, thank you for sharing! :D
This tutorial format is amazing. It teaches you how to do the necessary research yourself too.
@9:12 Will this cause any potential problems?
expected ‘const struct sockaddr *’ but argument is of type ‘struct sockaddr_in *’
my compiler takes it as a warning, *incompatible pointer type*, bind should have the parameter 2 as struct sockaddr_in *.... so i think its REALLY a problem :D
update: i need to cast struct sockaddr_in* to struct sockaddr* and set len argument for the bind to the size of my struct sockaddr
Why not use htons to handle the port number endianity?
That is a good point, thought about it after the video, since the port is hardcoded I just did it manually but indeed a better practice is to use htons
Nice! Well explained, clear and easy to understand. A sub from me!
Awesome, It would be great to make another one, but this time by managing, for example, the listening and sending parts simultaneously. (I think a simple fork should do the trick ?)
That is a good idea, thanks! maybe making a fork for every client connection
I think it that poll(2) would be better suited for this purpose (it blocks until on of the specified fd is available). But I guess that forks or pthreads work as well
Was a fun one! thanks for the video
Great tutorial Sir, need more like this. Thank You
Him setting it for 10 conns, me: Oooh, that's why slowloris works!!!! 😂😂😂
Jokes aside, nice job.
great but how do u know the steps to achieve like from open then close etc. I started programming in c but i want to start beginning projects like these any advice would be greatly appreciated
anyone know how to make the server not end when you connect? I've got mine working through chrome, but I wanna be able to peek at hte index.html file in chrome inspect element, but chrome can't get it because the server has already died :(
You can add a loop on the accept and client handling logic
Nice way you read the man pages.
Btw, can you explain why your system looks to be using dwm but you have a windows-like terminal? Are you using Linux or Windows with WSL?
Thanks!
I am using Windows 10 with WSL and a port of dwm for Windows, more info on the video about my setup.
So nice, I was looking for exactly this
2:16 c'mon man 🤙🏿
*$ printf %x 8080*
also,
*$ man byteorder*
3 points:
1. This wont work with file names that include spaces. I personally think that allowing spaces in file names should never ave happened. and searching for " ." wont work either as a workaround. Allowing spaces and other characters like newlines in file names makes things very messy.
2. When you compile there are warnings. You need to take care of those and get a clean compile
3. buffer ought to be one byte longer than what you have if you tell recv to get 256 chars then the buffer needs to be 257 chars in length in order to prevent unterminated strings
ive done minimalist web server before. all you really need is sockets and a handshake. then just handle the requests. used to do this using older cellphones to get around the blocks and control my computer. do stupid stuff like download camera photo or send text to speech commands to pc. instead of just handling the requests normally and provide web content back you can just use it as an input and serve back what ever you want.
That's cool! What kind of phone and programming languages did you use and how did you put the code on the phone?
@@nirlichtman not on phone. this was done in like early 2000's in' visual' basic. phone browsing was very limeted at the time bocking a lot of content. i used the faux webserver to get around those limitations. the host was at home and used the phone browser.
I feel like the sort of example you're going for (basically a SOCKS tutorial) would be better demonstrated with an echo server or something. Same information would have been covered. Just, with a server of any kind, you need to use multiple threads, otherwise, your server will only handle one client at a time.
libc/man just made a whole lot more sense to me.
Excellent video.
You could have kept it clean, and saved that hex conversion hack for the port with the relevant call (htons), and that would have been in line with the spirit of the video.
Thanks! Yah it was indeed a mistake to not use htons for the port (I mentioned on the pinned comment), htons is also more portable for other CPUs that order bytes differently.
You're wonderful. Thanks!
I love the word MINIMALIST
man if i knew i can use the man function to literally get every
detail, i would have aced my OS course in uni
Cool tip nobody asked for: if you have GNU Make installed you can run `make server` and it will infer that you are trying to compile `server.c` and will run `cc server.c -o server` so you will have your executable actually named after your source code file instead of "a.out".
Cool, thanks for sharing!
Neat! How did you FIRST learn about the various C library functions you used?
In this case, I started by reading the overview of the Socket API on Wikipedia and after I got an idea of the general functions and flow I read about the individual functions on the man pages, BTW I also like using the man -k and -K flags for searching through all the man pages, I plan on making a video about tips for searching man pages.
I'm trying to use man like you do but when I use "man socket" the output is different than yours. I'm using ubuntu on windows could this be the issue? I'm quite green about c and linux environment so maybe this will sound such a silly question. Thanks
What is the difference in the output? I use Debian on WSL so the environment should be pretty similar
@@nirlichtman so I did reinstall WSL and everything looks normal now. Unfortunately I did it before read your msg so I'm not able to produce the output in question. Your channel is incredible, resourceful and educational. Thanks for doing this
Interesting. Might have to try this.
http 3 actually uses UDP
I am actually surprised the http client accepted a response without status code or the HTTP version. I didn't know you can just send the data.
You can't. You need the HTTP version and a status code for the response to be valid per the standard, but browsers may still not accept it unless you include the date, content-length, and either transfer-encoding or connection: close
And if you don't include content-type, browsers will have inconsistent behaviour ( Safari for example will assume it's a file download, but Firefox will just render it as plain text )
I think I'm switching from code golf to C speed running now after this video.
Why does buffer need that 5 + thingy? And why its "GET"?
It's GET since that is the http request type that the browser sends when it retrieves a web page, the buffer + 5 skips over the first 5 characters of the request buffer (which in the case of a GET request, starts with "GET /"), so we can go ahead and extract the path that is requested
How did you get the man page for sockaddr_in? I have the man pages for socket and bind, but not the struct
Looks like you may be missing the man pages for C Lib (man 3) does "man scanf" work for example?
Make sure you have the manpages-dev package (name of package on Debian/Ubuntu)
@@nirlichtmanVery strange, `man scanf` works, but `man sockaddr_in` does not. Just to make sure, I reintalled manpages-dev with the same result. I also get the same result on my mac. Did you perhaps install additional manpages?
@@archerheffern4301 to get info about sockaddr_in run "man sockaddr", it will include information about sockaddr_in structure :)
@@nirlichtmanI figured out the issue! My manpages-dev was version 5.10 (current is 6.03).
Fixed by...
adding the debian mirror to my /etc/apt/sources.list
allowing insecure installing from the mirror (edit sources.list again)
running `apt-get update`
running `apt-get install manpages-dev`
@@nirlichtman new version of manpages-dev contains sockaddr_in and sockaddr and all the other fun structures
It's pretty cool but would u actually use this type of thing?
Like if you just want to send html around wouldn't go or rust be a better choice?
For production I would use nginx.
The code in the video is just for fun and learning purposes and definitely not suitable for prod :)
How were you able to do strchr(file, ' ') = 0? Shouldn't that throw a compiler error?
I am dereferencing the pointer returned from strchr with * and assigning the location in memory to 0, notice that because the code is for fun and not for production use I am skipping many checks and in this case I am not handling the case strchr fails
Excellent POC, thanks!!
wait are you Nir as in the maker of the Nirsoft utilities?
No that is a different Nir :)
i think you summarized 30 hours of a uni course in networking
Man, your content is good, but your keyboard is so loud I cannot focus on the video.
Thanks for the feedback, I will try to think how to better position the mic
Agree, the keyboard is unbearable in this but the video is soooooo good otherwise.
@@nirlichtman I think your best option is to record a voiceover afterwards. Do the video first, then narrate it. The bonus there is your voice will also sound better, and you already have a very pleasant speaking voice. Some people love this ASMR style of sound, but a lot of us find it quite stressful/distracting. Thanks for considering it!
Only if there was modern full name alias for old short named function/vars in C. It would've made it so much easier for dum dum like me
So what you are telling me, is that I should read the documentation?
Love your videos!
great job Nir!
So this channel is great!!!
Nice!! by the way, where are you from?(write me the SHA256 of the name of the country in lowercase)
😂 Israel
@@nirlichtman 💪💪💪 you have a great accent😂
Thanks! originally from the states :)