As a compiler engineer I have a few problems with this video 😬 I do understand that it tries to be as simple and comprehensible as possible, but there are a few things I'd change, for example I'd remove the part where "instructions correspond to transistors" and maybe I'd rework on the AST part. Also, you could continue the video for an extra minute or two talking about how the kernel handles interrupts in programs, how it saves the state and loads it again at a later point, etc.
Writing a programming language was one of the most fun projects I did(almost as fun as writing games in a handheld). Lots of rewrites and lots of grammar changes.
1:50 "We can write the binary by hand, but no one in their right mind would do it" My uncle had to compile the assembly code he wrote, for his graduation, by hand, because the memory of the hardware he was working on was insufficient for the assembly compiler & the code (he was designing a printer software, and it was the 1970's / 1980's)
Hey there, The example in @2:50 is technically not detected at lexer stage, it is at the parsing stage. The lexer, usually*, only tokenizes the source file (separates into tokens, shown around @3:13). So there should be an invalid token to cause an error there (I think '1x' is an example for the C language). The parsing stage "joins" the tokens to build a AST (Abstract Syntax Tree, mentioned around @3:15). The Grammar of the Language gives rules on how to build the AST. The grammar/parser says that "int x x;" doesn't make sense (that's my interpretation, hopefully accurate, on the meaning of "Syntax Error"). "Syntax Error" is closely related to parsing and AST. It was a nice overview on the steps from source code to running the program. *usually because sometimes people join the meaning of lexer and parser into "Lexing" or "Parsing"
You are absolutely correct. At least on the compiler my team wrote that is exactly how it works, and to be fair, I don't think anyone would implement grammar into the lexer. In fact, if we consider the lexer to be a Finite Automaton (pretty sound assumption), the lexer cannot even comprehend the grammar.
We wanna see you write some code exactly like NASA does . Following their blue prints and rules . That would be interesting. I loved that NASA code video
Binary code represents assembler instructions. The CPU interprets assembler instructions and it's internally made of combinational circuits which are made out of transistors.
You left out the step between compiler and executing- the linker step (not to be confused with the dynamic linker you did mention). The output of the compiler is in object format, with external references unresolved. The linker resolves those references and produces the execution (elf) file.
@@OGTennyson bad, I made a language so that creating my language is easier But then my Study was going down so I'm still trying to increase my study and get the top 3 position in grades, until I do that I can't program, I get only 30 mins a day to rest only. Although it's half finished it's convertion to CPP it supports high-level types where it automatically detects the type on compile time, I was trying to convert the language into lamda calculus or something but it's too complicated I Wana simplify it to make less intensive or optimize the code using math, after that the compiler is done, although the custom language I use is Converted in a js code (the actual compiler) node & Web js I got the following cool features apart from other languages (it's compiled language) Sec keyword to switch between Santax and language Sec Cpp{ The is cpp code inside } There's assembly (any) , cpp, and a custom Santax I call cjs supported I do Wana support the more lower level languages if posible So it's full controll It supports condition, loops, functions, Classes, high level types (you can specify types or it will automatically fine on compile time or run time), Goto statement I have idea for eval but it's complex because than I need the entire compiler and executer packaged in the translated code
2:00 I know you were going for simplicity, but I don't think it's been true for a long time that x86 assembly code corresponds to transistors. My understanding is that there is a RISC interpreter translating the x86 code, part of the CPU firmware blob.
I'm an undergrad, and I'm starting with a space company. One of the guys says the first thing he teaches people who are inexperienced is how a compiler works. He also says that one of the first things he asks experienced people is how a compiler works. Just my 2c. I'm on the learning end of this all
So the HVAC would shut off again when temperature is 70? I hope it can cool fast enough ;) I've never written a compiler, I wrote an interpreter and an assembler but that was enough of that. The latter was also more for myself because in college we had to do Z80 machine language. So we typed opcodes etc on a small uProfessor board, I have a video on the luxurious MFP-1. So I wrote an assembler to generate the opcodes and operands, just to speed up the assignment. So I could stream it from my tiny Atari Portfolio, the one John Conner used to hack the ATM, also wrote that in assembly and streamed it out of the serial interface using a MAX232 to convert RS232 to TTL level. That made it really easy for me to do that Spartan assignment. Even something as seemingly trivial as a washing machine that needs to control a motor, a pump, temperature (69F online ;) ) is a hassle in machine code. Yet trivial in assembly.
Wow, when i saw that the video was just 8mins i thought it couldnt get into that much detail without skipping stuff, but u smh made it! + its nice that things are explained on linux as example, i use debian btw
And all of this is very similar in windows as well, of course the file is different, and there are extra steps between asking the OS for process creation, and the syscall. Very nice video! Thanks!
There's some misleading information: at 7:40 it's stated that "the kernel brings in a program called the dynamic linker that runs alongside your program when it starts". Later, at 7:56 it's stated that "... the dynamic linker that comes with your program ...". This is contradictory information and it's not clear whether the kernel or the program provide the "dynamic linker" part. Also, at 7:27 it's not clear what exactly the "loader for your program" means or does, as it is only mentioned and later discarded. It would be great if you can clarify these points but I like your channel anyway. Cheers!
I can't answer your question because youtube thinks my comment contains a URL and deletes it. This is really an abysmal platform for educational discussion.
While the compiler is free to choose between bss and data, your uninitalized_memory global will ordinarily be placed in bss and initalized to 0. Explicitly initializing to 0 could have the unintended consequence of shifting it to .data, but most compilers are smart enough to position it in .bss anyways. GCC at least has flags to control this, -fzero-initialized-in-bss/-fno-zero-initialized-in-bss, with the former being the default.
4:16 uninitialized_memory that is initialized with zeroes, it takes space in your exe or elf file instead of just storing the size of memory that needs to be allocated at runtime. If you want uninitialized variable just don't write any value to it with = sign.
I am doing a TA job for a parsing class. but in theoretical linguistics. And some of the stuff we teach is helpful for the stuff I am working on. But using tree sitter and it's matching language - is still difficult
Really cool, but you kind of left us half way there… there us a cool video that complements this from David Smith that might be work looking into, it’s a hand made executable (elf) hello world for Linux X856.
How to extract the Abstract Syntax Tree and visualize it in Constellation or other data Viz app. With LLVM? How would the same program written in different languages appear in the Abstract Syntax Tree?
These type videos attract the backseat driving "I would have done it this way" type guys like moths to a flame. Scrolling the comments just to see how many nitpickers who've never made their own compiler video tell this guy how to do his.
Compilers do kind of blow my mind. Is there a source (like a book, not a program) that would illustrate compiler design? I’m trained as an electrical engineer and am proficient in assembly. I only mention this as something to provide context, if anyone would answer this.
Where to learn C? My hobby is programming and learning how computers works but I learned everything by my own and I started from high level languages like C#, python, js and idk where to learn C in most efficient way and understand things like handles, memory leaks, making code less vulnerable but without repeating everything like "This is variable. Variable is ... " over and over. And how is C compared to C++. Is It worth to learning C and then next C++ or just take everything at once ( Cherno has awesome tutorial about C++ but I decided to first learn C. Maybe I'm wrong ). I thought raw C is good starter for rust which is language i would like to learn in future. So finnaly, where to learn C for mid averange programmer ( mostly with high level languages ) and is It worth to learn C and C++ separatly or just learn C++ right away? Thanks for every help.
Bro, this Brilliant blocky cody thingy is sooooo infantilising. Their other courses are amazing, though. But, again, if you're not using the keyboard, you're not programming
Also, It's not just the .text section that gets mapped in, into that program segment that you talked about, although you called it the ".text segment". Other executable sections also get mapped in there, like .init, .fini, and .plt - it's really arbitrary, but it is executable code that ends up in that segment (because it has the executable flag set 'X'). So you're confusing the .text *section* with "segment". A segment is a combination of multiple *sections*. So it's the "text section" that it's called, not the text segment.
Don't forget to check out my limited time merch at lowlevel.store/
As a compiler engineer I have a few problems with this video 😬 I do understand that it tries to be as simple and comprehensible as possible, but there are a few things I'd change, for example I'd remove the part where "instructions correspond to transistors" and maybe I'd rework on the AST part. Also, you could continue the video for an extra minute or two talking about how the kernel handles interrupts in programs, how it saves the state and loads it again at a later point, etc.
Link some resources for the same please.
@@0xhhhhff Whats google?
@@sesu6743 your mom
@@0xhhhhff Link some resources for the same please.
@@sesu6743 can't. She left you when you were born
Writing a programming language was one of the most fun projects I did(almost as fun as writing games in a handheld). Lots of rewrites and lots of grammar changes.
You remind me of one of my best friends - super chill, friendly, funny, caring, passionate, outgoing, real, loving.
Cheers dude!
1:50 "We can write the binary by hand, but no one in their right mind would do it"
My uncle had to compile the assembly code he wrote, for his graduation, by hand, because the memory of the hardware he was working on was insufficient for the assembly compiler & the code (he was designing a printer software, and it was the 1970's / 1980's)
how much did he get paid for the the job
ive written binary code for some virtual machines. Its really not that hard
@@jordixboy it's not that it was hard, but really time consuming, and he didn't had many resources to ease up the workload
The best way to become a better software engineer is to try and write lexer, parser and interpreter/compiler. Great video ❤
Hey there,
The example in @2:50 is technically not detected at lexer stage, it is at the parsing stage.
The lexer, usually*, only tokenizes the source file (separates into tokens, shown around @3:13). So there should be an invalid token to cause an error there (I think '1x' is an example for the C language).
The parsing stage "joins" the tokens to build a AST (Abstract Syntax Tree, mentioned around @3:15). The Grammar of the Language gives rules on how to build the AST. The grammar/parser says that "int x x;" doesn't make sense (that's my interpretation, hopefully accurate, on the meaning of "Syntax Error"). "Syntax Error" is closely related to parsing and AST.
It was a nice overview on the steps from source code to running the program.
*usually because sometimes people join the meaning of lexer and parser into "Lexing" or "Parsing"
You are absolutely correct. At least on the compiler my team wrote that is exactly how it works, and to be fair, I don't think anyone would implement grammar into the lexer. In fact, if we consider the lexer to be a Finite Automaton (pretty sound assumption), the lexer cannot even comprehend the grammar.
Are you working on any open-source?
We wanna see you write some code exactly like NASA does . Following their blue prints and rules . That would be interesting. I loved that NASA code video
Next step, writing your own compiler. Would make for a fun video series. I'm writing one for my own language, and trust me, it is fun.
Binary code represents assembler instructions. The CPU interprets assembler instructions and it's internally made of combinational circuits which are made out of transistors.
You left out the step between compiler and executing- the linker step (not to be confused with the dynamic linker you did mention).
The output of the compiler is in object format, with external references unresolved. The linker resolves those references and produces the execution (elf) file.
The clock on the sponsor message is a nice touch
It's fascinating
I was just working on my own programming language
@@CC1.unposted please respond... How's the journey been going dawg?
@@OGTennyson bad,
I made a language so that creating my language is easier
But then my Study was going down so I'm still trying to increase my study and get the top 3 position in grades, until I do that I can't program, I get only 30 mins a day to rest only.
Although it's half finished it's convertion to CPP it supports high-level types where it automatically detects the type on compile time, I was trying to convert the language into lamda calculus or something but it's too complicated I Wana simplify it to make less intensive or optimize the code using math, after that the compiler is done, although the custom language I use is Converted in a js code (the actual compiler) node & Web js
I got the following cool features apart from other languages (it's compiled language)
Sec keyword to switch between Santax and language
Sec Cpp{
The is cpp code inside
}
There's assembly (any) , cpp, and a custom Santax I call cjs supported
I do Wana support the more lower level languages if posible
So it's full controll
It supports condition, loops, functions, Classes, high level types (you can specify types or it will automatically fine on compile time or run time), Goto statement
I have idea for eval but it's complex because than I need the entire compiler and executer packaged in the translated code
as a self taught software engineer, I really love this topics!
Dude please make a course on low level stuff (bits, bytes, byte-code, machine-code, interpreted langs vs compiled langs, etc.)
2:00 I know you were going for simplicity, but I don't think it's been true for a long time that x86 assembly code corresponds to transistors. My understanding is that there is a RISC interpreter translating the x86 code, part of the CPU firmware blob.
I'm an undergrad, and I'm starting with a space company. One of the guys says the first thing he teaches people who are inexperienced is how a compiler works. He also says that one of the first things he asks experienced people is how a compiler works. Just my 2c. I'm on the learning end of this all
Dynamic linking at runtime!
This is the type of content i subscribed for! amazing
Feedback : your videos audio are lower than other RUclips videos, please increase it for the next ones!
Please do a series on this, it's very interesting
So the HVAC would shut off again when temperature is 70? I hope it can cool fast enough ;)
I've never written a compiler, I wrote an interpreter and an assembler but that was enough of that. The latter was also more for myself because in college we had to do Z80 machine language. So we typed opcodes etc on a small uProfessor board, I have a video on the luxurious MFP-1. So I wrote an assembler to generate the opcodes and operands, just to speed up the assignment. So I could stream it from my tiny Atari Portfolio, the one John Conner used to hack the ATM, also wrote that in assembly and streamed it out of the serial interface using a MAX232 to convert RS232 to TTL level. That made it really easy for me to do that Spartan assignment. Even something as seemingly trivial as a washing machine that needs to control a motor, a pump, temperature (69F online ;) ) is a hassle in machine code. Yet trivial in assembly.
I am destroyed... I learned more during this 8 and a half minute vide than during a couple of semesters in my university
thanks for making us better developers, sincerely
definitely not first
correct
But you're the first in my heart bro 😄🫶
Definitely not last
@@Proferk you are an absolute beast bro. neo of this world
Frame of Essence *cOuGH
Wow, when i saw that the video was just 8mins i thought it couldnt get into that much detail without skipping stuff, but u smh made it!
+ its nice that things are explained on linux as example, i use debian btw
Glad you enjoyed!
And all of this is very similar in windows as well, of course the file is different, and there are extra steps between asking the OS for process creation, and the syscall.
Very nice video! Thanks!
As a JavaScript developer i would like to write an operating system in JavaScript and the ui in react. Minimum memory required 64gb
I once contemplated building an object code keyboard
There's some misleading information: at 7:40 it's stated that "the kernel brings in a program called the dynamic linker that runs alongside your program when it starts". Later, at 7:56 it's stated that "... the dynamic linker that comes with your program ...". This is contradictory information and it's not clear whether the kernel or the program provide the "dynamic linker" part. Also, at 7:27 it's not clear what exactly the "loader for your program" means or does, as it is only mentioned and later discarded. It would be great if you can clarify these points but I like your channel anyway. Cheers!
I can't answer your question because youtube thinks my comment contains a URL and deletes it. This is really an abysmal platform for educational discussion.
2:02 got me laughing🤣😆
While the compiler is free to choose between bss and data, your uninitalized_memory global will ordinarily be placed in bss and initalized to 0. Explicitly initializing to 0 could have the unintended consequence of shifting it to .data, but most compilers are smart enough to position it in .bss anyways. GCC at least has flags to control this, -fzero-initialized-in-bss/-fno-zero-initialized-in-bss, with the former being the default.
This is amazing, no one could have explained this easier than you.
Fantastic Video! Now I have to step through this process with my own programs!
What a co incidence.. I was literally reading many articles about this same topic, then I got the notification😅😅
4:16 uninitialized_memory that is initialized with zeroes, it takes space in your exe or elf file instead of just storing the size of memory that needs to be allocated at runtime. If you want uninitialized variable just don't write any value to it with = sign.
Amazing. Keep on.
We will!
2:24 👽 my respecc good sir
This was a fantastic and clear explanation. Thank you!
I am doing a TA job for a parsing class. but in theoretical linguistics. And some of the stuff we teach is helpful for the stuff I am working on. But using tree sitter and it's matching language - is still difficult
4:21 Low Level Learing: Uninitialized memory.
C: Initialized variable as {0, 0, 0, 0 .... 60 more zeros}
sup! thanks for the vid, always get excited when new vid comes, god bless
It's amazing how good this video is!
Really well explained the complete process! Thanks a ton for this video😊
I can't wait to get my elf parsed.
Yep. This is what I'm here for
More like that!
I read the message of binary in @0:21, I got rick rolled, "Never gonna give you up Never gonna let you down". Nice one
FIRST ONE TO NOTICE :D
At 0:35, why are there floppy disks on that programmer's desk?
yes
You don’t want to upset people who write assembly, there’s at least 6 of us.
Awesome.
Man, I just found this channel and it's definitely sent by God, because I'M INTERESTED IN EVERY SINGLE VIDEO.❤
Welcome aboard!
I use the assembly programming language.
Really cool, but you kind of left us half way there… there us a cool video that complements this from David Smith that might be work looking into, it’s a hand made executable (elf) hello world for Linux X856.
Simply awesome 🎉
How to extract the Abstract Syntax Tree and visualize it in Constellation or other data Viz app.
With LLVM?
How would the same program written in different languages appear in the Abstract Syntax Tree?
These type videos attract the backseat driving "I would have done it this way" type guys like moths to a flame. Scrolling the comments just to see how many nitpickers who've never made their own compiler video tell this guy how to do his.
lmao missing
spotted at the beginning
Hello man
its the magic smoke obviously
I love love love your videos I just wonder where can I learn more about software development ad how?
Compilers do kind of blow my mind. Is there a source (like a book, not a program) that would illustrate compiler design? I’m trained as an electrical engineer and am proficient in assembly. I only mention this as something to provide context, if anyone would answer this.
Try "crafting interpreters"
@@Gooberpatrol66 my guess would be that an interpreter would be a tougher job. I’ve been distracted from this , lately.
What font is used in the "coming up" intro part?`
You know you've made it when you got that Brilliant money 😂
Where to learn C? My hobby is programming and learning how computers works but I learned everything by my own and I started from high level languages like C#, python, js and idk where to learn C in most efficient way and understand things like handles, memory leaks, making code less vulnerable but without repeating everything like "This is variable. Variable is ... " over and over. And how is C compared to C++. Is It worth to learning C and then next C++ or just take everything at once ( Cherno has awesome tutorial about C++ but I decided to first learn C. Maybe I'm wrong ). I thought raw C is good starter for rust which is language i would like to learn in future. So finnaly, where to learn C for mid averange programmer ( mostly with high level languages ) and is It worth to learn C and C++ separatly or just learn C++ right away?
Thanks for every help.
Fun fact: The Intel iAPX 432 was a CPU designed to be programmed in high-level language
AT&T CRISP, used in the Apple Pippin
I am swimming but I don know if I am drinking coffee or not :D
RUclips recommended me while watching Javascript tutorial 🤡🤡
it knows just how fed up you get with JS
what's that vim theme?
Why we need loader for EACH segment? That doesn't make any sense
bruh
So riddle me this: what makes compilers not AI? It seems like they basically write assembly code for you based on vague suggestions
well one, who said they arent ai? and two, programming languages are hardly 'vague'
Compilers don't learn and improve, like AI does
Ok. Next topic. Your program in a microcontroller. Go.
Hence proved we live in a matrix
3:07
Who is this guy @1:55 . i have alway wanted to try and program in binary
Just a random guy on internet~
I think that's Linus Torvalds. I'm not sure tho.
@@capthehacker9985 yep its him -- creator of the linux kernel
Lol it was meant a sarcasm
The joke is *he really did* program in machine code. *Because he didn't know there was such a thing as an assembler.*
Notification gang 😈💻💯
We love casting spells😈
its kinda what chat gpt does
*clickbait* title lol
Ok.
audio too low
sound too quiet
Definitely not last
Doesn’t answer the question when everyone keeps personifying computers
Bro, this Brilliant blocky cody thingy is sooooo infantilising. Their other courses are amazing, though. But, again, if you're not using the keyboard, you're not programming
There's a slapping sound in the background that's annoying and distracting
j
Second
i love pineapples on pizza
Also, It's not just the .text section that gets mapped in, into that program segment that you talked about, although you called it the ".text segment". Other executable sections also get mapped in there, like .init, .fini, and .plt - it's really arbitrary, but it is executable code that ends up in that segment (because it has the executable flag set 'X'). So you're confusing the .text *section* with "segment". A segment is a combination of multiple *sections*. So it's the "text section" that it's called, not the text segment.