Another very dry topic we simply have to cover - it would be weird if I just started using some nodes for portable executables out of nowhere, wouldn't it? I really tried to make this one shorter, since I feel like video quality suffers the longer a video goes, but turns out there really isn't a way to discuss all those different bytes in less than 5 minutes :D Hope it's not too boring, it really is just the setup for some cool sh*t later down the line!
I'd argue the dry nature of it is the exact opposite of boring, yes it's technical, but I imagine a lot of your viewers, including myself, are here to learn the mysteries of the extremely low level computing you're implementing. Using nodes to explain this is a really simple way of visualising what these structures are actually doing, rather than reading (and failing to understand) even dryer implementation documents.
@@dec4dent I suppose you're right :) I'm a bit torn tbh, on the one hand I really reaaaally want to get started with computer graphics, on the other I want to make sure all these fundamentals are implemented as robustly as possible :D Can't wait to share what I'm working on for my master's...
I found it really interesting and not at all dry. Yes there are a lot of technical stuff but that's in my opinion the charm of your videos as well. You explain all of this so well that it's easily understandable and very interesting
Maybe. Just seems so god damn wasteful - no wonder concepts like DLLs had to be invented, if this is what minimal executables look like. Executable size is definitely something we'll have waaaay more control over with the compiler... Only the code that is actually used would be included for starters.
I think that modern compiler strips unused/unexported fonctions. For example (not a c compiler but) zig do that (for aig code at least). It's a little yet simple solution to large executables, and i think that libc has become a basic sharef library that every system includes (look at vcruntime, it's litteraly microsoft's libc). So most of OS's might just load libc at startup and when a program gets loaded, it takes libc functions from here (this is a BIG guess, might not be true at all)
@@darkfllameI'm sure there are compilers that link individual functions and get rid of the rest but the majority will still work with libraries. They will absolutely toss away a "import x.lib" line when no function of said lib is ever called. But removing parts of a lib seems like a good way of breaking a program if the dependencies are not well documented
Go's hello world takes 1852 bytes (Amd64/Linux). And I'm pretty sure that contains whole garbage collector. Stripping it from debug info gives 1208 bytes.
You know what's insane about this project? Because all behaviour is emergent from the most minimal form, there can be *_more than one programming language_* in this format. Prodecural vs Functional, Object Oriented vs Data Oriented, all of the dilemnas that we solve and resolve with text-based programming can be solved and resolved once again using these nodes. I can already see it; a day when you can start your program with the "Whole Ass Game Engine" node and begin scripting character controllers with "Update GameObject" nodes and push your gLTF files into "Render Mesh" nodes. This is so infinitely extendable and I am honestly shocked at how well executed this whole project is. I will absolutely be tuning in every week :)
It's fascinating to watch you put this compiler together piece by piece from scratch, I'm very curious to see how things will progress as you keep implementing more complex code and abstractions. Refreshing to see so many new and interesting ideas being explored so thoroughly.
This feels like the start of Wintergatan Wednesdays: slowly watching an insurmountable problem be chipped away at, excitedly rooting for its success. Best of luck!
This is fascinating! I've been a C programmer for years and have done some simple assembly programs, so seeing how it all unwinds is really beautiful. I really really love the node system, it reminds me in a way of a good makefile that only recompiles files that have changed! This is easily my most anticipated video series at the moment, and I'm so excited to see where this will go!
Build software is actually one of my key motivators to make this compiler - I was simply so fed up with them that I decided to build my own :D It's a bit like a makefile, except it works on the level of singular bytes, not entire files, so it saves a lot more time in that regard! Glad you feel that way, It's refreshing to have people around you that are excited as you are :D
I think a video with a more in depth explanation of the built-in nodes or showing how they work would go a long way for me to make more sense at what you're doing, I get the general idea but it feels to abstract to me, I rewatched your first video couple of times and it stills feels to abstract. Maybe I'm just dum...
I may do that soon actually, would be a good topic I could cover to free up some time for debugging of the prototype - since I don't have to do any research for that one, so it wouldn't take too much time. Maybe even split it up into two or three videos... the more I think about it, the more it seams like a good idea. I might do that after the executable stuff is done, thanks!
I have to add support for my current desktop pc and the pcs of my university before anything else, since the first big project will be my masters thesis in computer graphics. Those all happen to be windows.
I've always wondered what goes on inside a .exe file. Great vid! Looking forward to seeing what the node graph looks like for the checksum implementation
Thanks mate! Looking forward to that too, since I have no idea myself at this moment :D not sure if it'll actually be a node graph though, might be a dedicated node since the checksum calculation would have to run every time something changes - doing that algorithm as a node graph would add a ton of preventable overhead.
@@Kronark I wish I wasn't such an amateur programmer so I could help out. This would be a dream project for me, essentially undoing decades worth of code plumbing hell with essentially a unifying code abstraction layer that's tray size agnostic and can support both scalar and tensor operations (in the future). And its portable!? I'm not a very capable programmer but I am a good theorist type, and into defi and open-source, all energy physics topics you can imagine, and also data engineering (databricks, Excel, an overview of most things). If you ever wanted somebody to explain actually useful unified field theory and direct you to the actual scale-invariant math which can unify relativity and quantum physics, just let me know. The theory is called E-infinity theory, and it's hard to jump into and 'get' which is why I'm offering to explain things. 🤔 I have experienced over years that holding more accurate physics in mind while solving math or engineering is always an advantage, and that infact the world it trending towards needing to unpack the complexity of higher dimensional organization and space-time attributes in euclidian and non-euclidian space forms-- to do AI, to do code and defi, to compress data, to bridge data formats, to analyze, synthesize... I'm sure you're group is quite smart and so you'd very quickly surpass my understanding in some areas, since this information really adds up and follows some loose guidelines like duality and spinor-like modulations. I have years of research in crypto utility and business deals made that help predict the trajectory there, and I have plans to become a full stack developer of the decentralized internet using open-source technologies and a few novel applications of networking techniques. I'm seeking friends who develop actually good code for good intent purposes, so that perhaps one day we can assemble a project given there's mutual availability, interest, and assurances. However or whenever you feel comfortable to work or talk that's fine, I recognize genius marches to the beat of its own tempo and art cannot be rushed. When possible, I may even be able to provide funding in the future if my crypto bets are right. I just want to help the world with whatever time and resources I'm given.
Gonna be a bit until then unfortunately, it's still quite a buggy mess behind the scenes :) definitely going to be one of my first software products available in my online store though (for free, of course)
I wonder, will you be creating your own graphics API for this system, or would you recreate existing API's 1:1 in your node language as a testament to its power.
We‘ll be targetting the vulkan api for now, but obviously add some abstraction layers to make it a little less heavy :) So yeah, we‘ll implement the ABI for vulkan essentially :D And thanks! :)
Thanks! The code of the prototype probably won't be published, no. The release version will be built using the prototype though, and the node networks for that may be published. Either way, it will probably be under a restrictive licence instead of open source. Not giving this sh*t away for free to large companies, just for them to put a price-tag on it, which unfortunately happens all to often.
I have a quick question: how will we be able to represent code, i mean, your entire language is just an "interpreted language that write bytes to a file", how will we be able to represents things like control flow statement, function, variables manipulation ? This is a dumb yet good question to ask, because a language is used because it's practical in "some" aspects (*cough* rust) and honestly, as bad as i would like to have that much power on how my program is built, having a headache while using that compiler will be an issue. Maybe i didn't understood correctly the first video but i'm hyped for the next ones :D
My understanding is that if this system is Turing complete, then a user of this program could define all of those things you mention. I expect that when this is done there will be a “standard library” that will have many typical features implemented. This project is interesting and I hope the dev gets it all figured out.
@@sillythewanderer4221 I think he will eventually explain in details the language when it gets released to the public, anyway the project is colossal work so iwould give it at least a years before a release and a stable version...
If you wish to make an apple pie from scratch, you must first invent the universe. - Carl Sagan This guy is doing just that. Basically writing their own Assembler. This is lower than C. And way lower than C++ and Java or any modern language. The concept of a function doesn't exist in machine code or asm. A while loop doesn't exist. Objects and methods and arrays don't exist. All of that needs to be written/implemented first. C does this and many other languages rely on C (that is to say, the code is translated to C and then gcc is used to compile to machine code). Other languages translate to asm directly. And this language which is being built here even skips that step and compiles to machine code directly, or that's the intention it seems. Wild stuff 😅
jup, that's exactly what I'm planning on doing. all the abstraction layers we'll be building throughout the next years will be shipped with the release build - anything else will be downloadable as needed via my website eventually. so any user of the release build won't need to touch any of this complexity if they don't want to, but they could do so any time.
I'd probably give it at least 2 years to be honest. It's going to take another 6 months minimum until I've ironed out every single little detail and made this prototype stable. But then I'm definitely going to take a bit of a break from compiler design and do some fun graphics stuff for a while. We'll eventually implement that release build in a small series on this channel though, using the prototype!
i don't understand much about code or math but i understand nodes. i think they are still criminally under utilized in most implementations. As a graphic designer and ui/ux designer i have a long list of stuff i would love to try to improve visibility, readability and the connections between nodes. Very curious how your project will develop.
We‘ll definitely be taking a look at node graph ui design enhancements further down the road, when we‘ve built the release product and the frontend is exchangeable :D
See the first video in the series for more context. Basically, imagine you're programming using nodes like you'd have in Blender or Godot. The end goal is for the author to use it in his thesis about lighting
@@Kronark OK, so it's a programmable compiler but can be used at upper levels not just the compile level, because you'll be using the whole thing like an os to make games and it's also all In machine code?
It‘s designed for the construction of a significant amount of abstraction layers on top of each other, allowing for high level programming while still providing access to low level implementations. The day to day usage will be similar to standard programming and every now and then some optimisations can be implemented on lower levels to make those high level abstraction as efficient as possible.
Maybe I‘ll upload a gif or screen recording via a community post soon-ish. Don‘t think there‘s a point in making a video of „look at me shoving nodes around“ :)
Another very dry topic we simply have to cover - it would be weird if I just started using some nodes for portable executables out of nowhere, wouldn't it? I really tried to make this one shorter, since I feel like video quality suffers the longer a video goes, but turns out there really isn't a way to discuss all those different bytes in less than 5 minutes :D Hope it's not too boring, it really is just the setup for some cool sh*t later down the line!
I'd argue the dry nature of it is the exact opposite of boring, yes it's technical, but I imagine a lot of your viewers, including myself, are here to learn the mysteries of the extremely low level computing you're implementing.
Using nodes to explain this is a really simple way of visualising what these structures are actually doing, rather than reading (and failing to understand) even dryer implementation documents.
@@dec4dent I suppose you're right :) I'm a bit torn tbh, on the one hand I really reaaaally want to get started with computer graphics, on the other I want to make sure all these fundamentals are implemented as robustly as possible :D Can't wait to share what I'm working on for my master's...
I found it really interesting and not at all dry. Yes there are a lot of technical stuff but that's in my opinion the charm of your videos as well. You explain all of this so well that it's easily understandable and very interesting
Cheers mate!
Tbh, i like longer videos
I think the reason a simple c executable is so large is the inclusion of the entire standard library
Maybe. Just seems so god damn wasteful - no wonder concepts like DLLs had to be invented, if this is what minimal executables look like. Executable size is definitely something we'll have waaaay more control over with the compiler... Only the code that is actually used would be included for starters.
I think that modern compiler strips unused/unexported fonctions. For example (not a c compiler but) zig do that (for aig code at least). It's a little yet simple solution to large executables, and i think that libc has become a basic sharef library that every system includes (look at vcruntime, it's litteraly microsoft's libc). So most of OS's might just load libc at startup and when a program gets loaded, it takes libc functions from here (this is a BIG guess, might not be true at all)
@@darkfllameI'm sure there are compilers that link individual functions and get rid of the rest but the majority will still work with libraries. They will absolutely toss away a "import x.lib" line when no function of said lib is ever called. But removing parts of a lib seems like a good way of breaking a program if the dependencies are not well documented
Go's hello world takes 1852 bytes (Amd64/Linux). And I'm pretty sure that contains whole garbage collector. Stripping it from debug info gives 1208 bytes.
I think the compiler aligns the code to page boundaries
You know what's insane about this project? Because all behaviour is emergent from the most minimal form, there can be *_more than one programming language_* in this format. Prodecural vs Functional, Object Oriented vs Data Oriented, all of the dilemnas that we solve and resolve with text-based programming can be solved and resolved once again using these nodes. I can already see it; a day when you can start your program with the "Whole Ass Game Engine" node and begin scripting character controllers with "Update GameObject" nodes and push your gLTF files into "Render Mesh" nodes. This is so infinitely extendable and I am honestly shocked at how well executed this whole project is. I will absolutely be tuning in every week :)
It is so god damn refreshing to read comments that actually get the potential of this tool
It's fascinating to watch you put this compiler together piece by piece from scratch, I'm very curious to see how things will progress as you keep implementing more complex code and abstractions. Refreshing to see so many new and interesting ideas being explored so thoroughly.
Thanks mate! :)
Hope it all turns out nicely in the end :D
This must be the most ridiculously ambitious RUclips Channel I have ever encountered. I'm curious to see where you end up with this project.
I'm just as curious as you are! Hope it all goes well...
I really appreciate the comprehensiveness of the videos
Glad you think so! :)
This feels like the start of Wintergatan Wednesdays: slowly watching an insurmountable problem be chipped away at, excitedly rooting for its success.
Best of luck!
Thanks man! :D
This is fascinating! I've been a C programmer for years and have done some simple assembly programs, so seeing how it all unwinds is really beautiful. I really really love the node system, it reminds me in a way of a good makefile that only recompiles files that have changed! This is easily my most anticipated video series at the moment, and I'm so excited to see where this will go!
Build software is actually one of my key motivators to make this compiler - I was simply so fed up with them that I decided to build my own :D
It's a bit like a makefile, except it works on the level of singular bytes, not entire files, so it saves a lot more time in that regard!
Glad you feel that way, It's refreshing to have people around you that are excited as you are :D
I think a video with a more in depth explanation of the built-in nodes or showing how they work would go a long way for me to make more sense at what you're doing, I get the general idea but it feels to abstract to me, I rewatched your first video couple of times and it stills feels to abstract. Maybe I'm just dum...
I may do that soon actually, would be a good topic I could cover to free up some time for debugging of the prototype - since I don't have to do any research for that one, so it wouldn't take too much time. Maybe even split it up into two or three videos...
the more I think about it, the more it seams like a good idea. I might do that after the executable stuff is done, thanks!
@@Kronark Hey, thanks for the answer, I will keep watching the series and best of luck in your project.
Thanks mate :)
The inofficial but widespread way of saying that there is no linker is to set the mayor and minor linker version to 255 if I remember correctly
Ah perfect, thanks for the info mate! I‘ll implement that for the next video :)
@@Kronark Nice. But look it up first, my memory is not the best 😄
Of course :D thanks for the hint either way!
The next year is gonna be really fun.
It‘ll be wild :D
amazing editing! love your videos
Thankyou! :)
This really gives insight into the the ultra low level stuff
.
PS: editing and ur voice is top notch
Glad you think so! Thanks :)
This reminds me of Sebastian Lague. Really nice.
It's an honour to be likened to that legend!
Would be based if this was not Windows...
I have to add support for my current desktop pc and the pcs of my university before anything else, since the first big project will be my masters thesis in computer graphics.
Those all happen to be windows.
3:48 foreshadowing
hehe
I've always wondered what goes on inside a .exe file. Great vid! Looking forward to seeing what the node graph looks like for the checksum implementation
Thanks mate!
Looking forward to that too, since I have no idea myself at this moment :D
not sure if it'll actually be a node graph though, might be a dedicated node since the checksum calculation would have to run every time something changes - doing that algorithm as a node graph would add a ton of preventable overhead.
Once these basics are covered (which almost never change) I guess this is going to be pretty exciting seeing the first programs being implemented :O
Can‘t wait myself! It‘s already pretty neat to see the first „hello world“ working :D
This video feels pretty epic. The editing, the sound effects!
Thanks! :)
Honey cancel all my meetings, my weekly novel has started 🗿
oh boy, the pressure is getting real :D
Ah! So you are going to support Risc-V! I'm so happy! 🎉
Gonna be a while until it becomes relevant though, but yes, definitely planning on it :)
@@Kronark I wish I wasn't such an amateur programmer so I could help out. This would be a dream project for me, essentially undoing decades worth of code plumbing hell with essentially a unifying code abstraction layer that's tray size agnostic and can support both scalar and tensor operations (in the future). And its portable!?
I'm not a very capable programmer but I am a good theorist type, and into defi and open-source, all energy physics topics you can imagine, and also data engineering (databricks, Excel, an overview of most things).
If you ever wanted somebody to explain actually useful unified field theory and direct you to the actual scale-invariant math which can unify relativity and quantum physics, just let me know. The theory is called E-infinity theory, and it's hard to jump into and 'get' which is why I'm offering to explain things. 🤔
I have experienced over years that holding more accurate physics in mind while solving math or engineering is always an advantage, and that infact the world it trending towards needing to unpack the complexity of higher dimensional organization and space-time attributes in euclidian and non-euclidian space forms-- to do AI, to do code and defi, to compress data, to bridge data formats, to analyze, synthesize...
I'm sure you're group is quite smart and so you'd very quickly surpass my understanding in some areas, since this information really adds up and follows some loose guidelines like duality and spinor-like modulations.
I have years of research in crypto utility and business deals made that help predict the trajectory there, and I have plans to become a full stack developer of the decentralized internet using open-source technologies and a few novel applications of networking techniques.
I'm seeking friends who develop actually good code for good intent purposes, so that perhaps one day we can assemble a project given there's mutual availability, interest, and assurances. However or whenever you feel comfortable to work or talk that's fine, I recognize genius marches to the beat of its own tempo and art cannot be rushed. When possible, I may even be able to provide funding in the future if my crypto bets are right. I just want to help the world with whatever time and resources I'm given.
I can’t wait to try out the final product
Gonna be a bit until then unfortunately, it's still quite a buggy mess behind the scenes :) definitely going to be one of my first software products available in my online store though (for free, of course)
I wonder, will you be creating your own graphics API for this system, or would you recreate existing API's 1:1 in your node language as a testament to its power.
As Always, Beautiful Video. Will be rewatching the video a few times, so more questions or statements may come down the line 😅
We‘ll be targetting the vulkan api for now, but obviously add some abstraction layers to make it a little less heavy :)
So yeah, we‘ll implement the ABI for vulkan essentially :D
And thanks! :)
All of this is very interesting,Will you publish the code,if so will it be open source.
Thanks!
The code of the prototype probably won't be published, no. The release version will be built using the prototype though, and the node networks for that may be published. Either way, it will probably be under a restrictive licence instead of open source. Not giving this sh*t away for free to large companies, just for them to put a price-tag on it, which unfortunately happens all to often.
Quality channel.
Glad you think so! :)
I have a quick question: how will we be able to represent code, i mean, your entire language is just an "interpreted language that write bytes to a file", how will we be able to represents things like control flow statement, function, variables manipulation ? This is a dumb yet good question to ask, because a language is used because it's practical in "some" aspects (*cough* rust) and honestly, as bad as i would like to have that much power on how my program is built, having a headache while using that compiler will be an issue. Maybe i didn't understood correctly the first video but i'm hyped for the next ones :D
My understanding is that if this system is Turing complete, then a user of this program could define all of those things you mention.
I expect that when this is done there will be a “standard library” that will have many typical features implemented.
This project is interesting and I hope the dev gets it all figured out.
@@sillythewanderer4221 I think he will eventually explain in details the language when it gets released to the public, anyway the project is colossal work so iwould give it at least a years before a release and a stable version...
If you wish to make an apple pie from scratch, you must first invent the universe. - Carl Sagan
This guy is doing just that. Basically writing their own Assembler. This is lower than C. And way lower than C++ and Java or any modern language.
The concept of a function doesn't exist in machine code or asm. A while loop doesn't exist. Objects and methods and arrays don't exist. All of that needs to be written/implemented first. C does this and many other languages rely on C (that is to say, the code is translated to C and then gcc is used to compile to machine code). Other languages translate to asm directly.
And this language which is being built here even skips that step and compiles to machine code directly, or that's the intention it seems. Wild stuff 😅
jup, that's exactly what I'm planning on doing. all the abstraction layers we'll be building throughout the next years will be shipped with the release build - anything else will be downloadable as needed via my website eventually. so any user of the release build won't need to touch any of this complexity if they don't want to, but they could do so any time.
I'd probably give it at least 2 years to be honest. It's going to take another 6 months minimum until I've ironed out every single little detail and made this prototype stable. But then I'm definitely going to take a bit of a break from compiler design and do some fun graphics stuff for a while. We'll eventually implement that release build in a small series on this channel though, using the prototype!
i don't understand much about code or math but i understand nodes. i think they are still criminally under utilized in most implementations.
As a graphic designer and ui/ux designer i have a long list of stuff i would love to try to improve visibility, readability and the connections between nodes.
Very curious how your project will develop.
We‘ll definitely be taking a look at node graph ui design enhancements further down the road, when we‘ve built the release product and the frontend is exchangeable :D
i didn't received the notification D;
yeah I figured something went wrong on youtube's end... this upload had a much slower start than the previous two weeks.
@@Kronark i disabled pratically all notifications on my phone tho (except system ones), they were too annoying...
I wish you were my teacher back in college!
It would‘ve been an honour, i‘m sure.
@@Kronark sensei!
Did you use the structure definitions in winnt.h?
I think the resource I was using to mostly understand this whole thing was using them, I just cleaned it up a bit - so I suppose indirectly, yes.
do you show anywhere what the so-far-defined 'atomic' nodes are in your system?
previous videos cover some of the atomics a bit, but the next videos will cover them in more detail, as by popular request :)
Another great vid 🔥🔥🔥
thank you mate! :)
ELF is the most based format.
true, but ELF doesn't run on windows as far as I know, and I need to target windows for my masters.
bro is cooking, but what exactly is he doing in these series? it feels like they are all out of context
See the first video in the series for more context. Basically, imagine you're programming using nodes like you'd have in Blender or Godot. The end goal is for the author to use it in his thesis about lighting
It'll all make sense soon, I hope! :)
@stefanalecu9532 already said everything else :D
Ok its wednesday the 28th you can upload the next episode now!
oh sh*t! How did I miss an entire week?!
Neat!
cheers!
cool!
You haven't even watched it yet :P
He watched at 200x speed.
so, can we call these PEX Files?
I don't get it... ^^ Why PEX?
@@Kronark i dont know, since executables were called EXE files probably
anyone understand what's going on?
You might need to check out the other videos :)
@@Kronark is it that you're making a programable complier. Or the user tells the complier what to do manually
I think „programmable compiler“ summarises it quite neatly actually :)
@@Kronark OK, so it's a programmable compiler but can be used at upper levels not just the compile level, because you'll be using the whole thing like an os to make games and it's also all In machine code?
It‘s designed for the construction of a significant amount of abstraction layers on top of each other, allowing for high level programming while still providing access to low level implementations. The day to day usage will be similar to standard programming and every now and then some optimisations can be implemented on lower levels to make those high level abstraction as efficient as possible.
Can you actually show your program running?
Maybe I‘ll upload a gif or screen recording via a community post soon-ish. Don‘t think there‘s a point in making a video of „look at me shoving nodes around“ :)