every gamedev at some point thinks "man this would all be better if i just created the entire computer system from scratch" and then realizes that would be a waste of time, would go nowhere, and the dev goes back to using godot you let the intrusive thoughts win, and youre actually brilliant enough to do it. a true chad
Eh... it's super heavily commented, especially in the complicated parts, because it's so easy to lose track in the core algorithm... commented to the point it's unnecessary ^^ I wouldn't say that's high quality tbh :S
oh I am EXCITED for the rest of this. Editing style is beautifully original and clean. Videos are short and simple without all the padding to take advantage of the algorithm preferring longer videos. It's perfect. I'll be here for every future video.
I'm really excited at the thought of seeing a CPU simulator in action, with the node graph interface integrated. This video series is really exciting. Best of luck with the project!
Fascinating project you're laying out here, I hope there would be an opportunity to contribute in future, this could really go some places! Depends on how far you're willing to take it.
There will, although that‘s probably further in the future than some people in the comments hope. We‘ll get there eventually, gonna focus on actually making the thing work properly first and check if it actually works for large projects :)
Not for now, but eventually, yes. I'll not be releasing this prototype, the code quality is abysmal to be honest. I'm planning on getting the prototype feature complete and fully capable of developing any form and size of software - then I'll use the prototype to build a release-version of itself. It's most likely gonna be another couple of years until then to be honest, I need a break from compiler design once the prototype is done. But yeah, once a proper release version is built, I'll be providing the software for free on my website - probably gonna build an online node-sharing platform to go with it.
@@Kronark "Use the prototype to build a release version of itself" - that sound wild χD I wish you all luck/power/will and whatever else you'll need to get this thing done. Neat editing, btw 👍
brooo, this is looking insane, i hope i'd be able to use this, i've always dreamed of making a mixed node/scratch like programming language but never got past graphics since SDL is slow, OpenGL is overkill for such a simple program and for the sake of commiting not live anymore i want to make it in zig. Anyway good luck my man, take your time.
My dream is to do freelance software development - this compiler will probably be the first product on my page (and it'll be free) together with a little node sharing platform. So you'll definitely get to use it some day! But that's still a bit in the future :D I'll have to take that last part to heart man, the growing pressure is stressing me out right now ^^
Weird to see the overlap of people interested in such niche technologies. I'm a big Zig guy and was considering making a graphical language that transpiles to Zig, but I've got too many other projects on my plate right now
I'm trying to understand the benefits of this representation vs coding in assembly. I guess the hierarchical abstraction of code into higher level nested nodes helps, and makes clearer what the inputs and outputs are, whereas in assembly the calling convention is implicit (e.g. args might be on the stack or in registers).
You got it :) This will be extremely similar to any other assembler in the beginning, but we'll work our way up to higher level abstractions eventually + add new compilation targets that no assembler currently targets. We have the control of an assembler, but the ability to do so much more.
absolutely, in fact most modern binary and text form platforms are supposed to be supported eventually. So not just x64 for all OSs, but also ARM, RISC-V etc - and then also html, css and javascript, python, c and c++ source code etc.
Of course! We'll actually need those eventually once we start implementing the release build of this compiler using the prototype - arbitrary precision integers are essential for parsing number strings
How do you represent a stream of bytes? How do you interpret them? As an array of bytes? But how do you process that? How do you KNOW that you are getting a number and not a stream of bytes? How do you handle explicit types? How do you handle global variables? Even loops would require problems. And what about for each element nodes? You speak of compilation, but I don't see how this could be anything other than interpreted. It does sound really interesting though.
Data is passed between nodes via an object. This object stores the data type in addition to the data itself and some other metadata. Each input socket / "port" node has to have at least one "type" node associated with it to be able to receive a connection. Output sockets can only output data of a single type. For a connection to be created, the type of the output socket has to match the type of the input socket. This ensures that no malformed data is ever received, except of course you break it on purpose. That's how I know that I'm receiving a number and not a stream of bytes. And of course, there also is an "inspector" tool which I haven't shown yet, which lets me inspect any given node and see what data is provided at each socket and what type that data has. Regarding global variables, that's an upcoming topic - there's a state system for node instances. But that's a bit to complicated to explain in a RUclips comment. Regarding loops, there is a built-in compiler-loop node planned which will allow repetitive node executions. If you mean loops in terms of actual output instructions, why would there be a problem? Jump instructions are just like all the other instructions - If you're worried about the determination of a jump destination, that will also be solved using the aforementioned state system. What do you mean by "for each element nodes"? It's a compiler by definition because it transforms a high level (visual) language into other languages (anything that's a bunch of bytes in a file). An interpreter (again by definition) directly executes the high level language it is given - which is not the case here. The more complicated topics you mentioned will be tackled eventually - just know I have already thought about them and have found solutions for them.
Hey man! Not this specific tool, but additional software could be written once the necessary x64 instructions have been designed - then you‘d just have to match byte sequences to node outputs and build a valid node file from there, to be displayed with this node graphing frontend
I know you don't want to release a prototype but It'd be cool that as soon you are bored of it to make it open source. I'd love to participate in development!
Thanks for your video, the idea looks interesting. If I get it, using graphs you will compile platform specific instructions. How your system will interact with OS? If it will not interact with system, how you will handle user input/output? Also what about scaling? If you will write something more complex than arithmetic you will have very large graph, so I am curios about modularization or some way of splitting graph into reusable components. You can easily overflow integer after multiplication. How you deal with undefined behavior? What about debugging?
Initially yes, it‘ll be platform specific, but I‘ll eventually implement a state system to switch between different platforms - so the instructions won‘t be platform specific from then on. OS interaction is done through system calls, which are just fancy CPU instructions that give control to the system for specific tasks (e.g. system in and out, filesystem access, etc.) - they‘ll be handled just like any of the upcoming arithmetic instructions Ah yes, that‘s something I didn‘t touch on because it was a bit unnecessary for now in my opinion. The program comes with a module system. Each node is part of a module and modules can be part of other modules - basically a standard folder-file approach like in operatings systems. Modularisation and node-graph reuse is a central concept to the entire thing. You actually don‘t have any other choice since there are hard limits on node instance count within a given node (sub-) space. Regarding integer overflow: True, but for now I just wanted to prevent value overflow on definition. We‘re effectively preventing the equivalent of „uint_8 var = 300“ or something like that. overflow during multiplication is usually handled by increasing the bit count of the result variable as far as i know - i think that‘s what the x86 machine instructions do if i‘m not mistaken. We‘ll deal with other undefined behaviour the same way it‘s dealt with in other approaches :) Debugging will be done initially via print statements, just like it‘s done in traditional contexts. I‘ll add dedicated debugging functionality sometime in the future - not the highest priority right now.
This is amazing. Yet so obvious. Like all genius ideas. Say, will nodes be able to have side effects? Also, I think you should start with the ELF executable format (for linux), it is much simpler than EXE.
Thanks! Side effects as in „output one thing, but also add something somewhere else“? - yes, there‘ll be a state system we‘ll use for global variable declarations any possibly register assignment. Yes, I‘ve worked through the ELF format already. However, EXE isn‘t super complicated either imo. Also, my desktop PC is windows and so is the hardware my master thesis stuff will be tested on - unfortunately i‘m locked in regarding that, it wouldn‘t make sense to spend time setting up linux right now. But we‘ll add support for it next year, after my thesis is done.
Will you ever share the compiler or at least show snippets the code that's building this? I feel like this is a really interesting idea, but doesn't really feel fulfilling to me to not see any of the code that allowed this to exist. Kind of just hand wavey magic.
Eventually, yes. Since this has been commented a lot, I may consider moving forward on that a bit earlier... but right now I need to get out of the woods regarding my masters thesis - there is still a lot to do before I can tackle anything close to light simulation, and that takes higher priority right now.
What benefits do you expect to gain by using a node network over writing assembly code? Generally I found node programming to be fairly unergonomic due to reliance on mouse. Further I feel like any node network takes up significantly more screen space than corresponding code would have. As such I have a harder time reading the program as it requires more scrolling/moving around. Getting a gui with a spinning triangle up and running with this system would be truly impressive. But I am curious how far you can manage especially with such an ambitious timeline. Left a sub and will check in every so often :P
There definitely is a trade-off every user has to determine for themselves, I personally don't mind mouse usage - In the end, most of my programming time is spent conceptualising and debugging, not navigating. A lot of stuff is made easier via hotkeys already and I am planning on adding navigation via keyboard in a future public release build e.g. jump to different nodes on the grid via WASD or arrow keys, jump to a certain socket via another hotkey combo etc. - It then would be pretty close to editing a text file in my opinion. This compiler concept heavily incentivises coding in a reusable manner, since there is a hard node limit per given node space (= basically a grid that defines any given node) and clarity decreases rapidly the larger a network gets (as you mentioned) - so what I've experienced so far is that any given network on screen actually stays pretty small, because you keep wrapping sections into new custom nodes that hide all the visual complexity behind a more compact format. My main problem with current programming practices is the lack of control I have over what actually is written into the final executable. Yes, I could try writing it in assembly, I could technically build custom abstractions using macros - but text editors just aren't made for building large amounts of abstraction layers, I would eventually just have huge files that have nested macros in them, which will just become a huge pain to debug in the end. I think that's probably one of the main reasons why we have programming languages if I'm not mistaken - but then when I'd use basically any programming language, I have to rely on 3rd party implementations regarding optimisation and I lose full control over my output. All this node compiler does is string bytes together, the user controls everything else - there's definitely gonna be a lot of setup work, but I think it's going to be worth it in the end. Additionally, using node networks allows for some heavy compile time optimisations - providing me with a much smoother development experience since I essentially never have to wait for a build to finish. It'd be interesting to see how other people use the tool in 2 years or so, once I start doing public beta-tests or something like that. I definitely think the right community could make this tool superior to what we have now. I actually won't be using triangle geometry in my masters thesis - it'll be an implicit surface rendering approach, so compute shaders rendering spheres and boxes all the way :D Oh, I'm just as curious as you are, thanks for sticking around :D
@@Kronark You seem very enthusiastic about this, and I don't want to dampen your motivation. I just enjoy playing devil's advocate. Regarding your main concern, I believe that any form of abstraction inevitably results in a loss of control over the binary. For instance, if you provide a standard library with common nodes, like a "print line" node that allows me to print text directly to the console, I, as the user, would need to delve into that node to fully understand how the printing is implemented. This includes understanding any implicit type conversions, as explained in your video, and how your node optimization works. Without this deep understanding, I would no longer have control over the bytes in the output-I would be relying on your node implementation much like I rely on the print line function in C and the gcc optimizations. Moreover, controlling the bytes in the executable is just one aspect of output control. We still depend on abstractions from the operating system to load our program into a memory location as it deems appropriate and to access resources. The overall system's output when running still relies on many abstractions that we likely don't fully grasp.
@@kn0rkk All good mate! Regarding your print-function example, it's true that you need to delve into the subject matter a bit to understand lower level abstractions - but the key difference in this node editor is, that you can do that at any time. You don't have to clone some git repository of the source code and you don't have to decipher a specific programming style. It's always right there at your finger-tips (quite literally) in the same format as any other node network. There is no hurdle to get to the lower level implementation besides potential missing know-how to properly grasp it. Additionally, there is no such thing as closed-source for node networks in the compiler. The node network of any functionality has to be available on your device in a compiler readable format - and if the compiler can read it, so can any user. When trying to understand library functions during traditional programming, you may not always have access to the source code, only the library binary. Regarding the OS, absolutely. This compiler only shaves off any black box design in user-space, not in the entirety of a software lifecycle.
What is the meaning of "channel", "slot: 1", "no repetition"? Why some sockets do not have any white dot? And what is the meaning of "default" and "custom" in the integer type node? Thanks
I'll be making some dedicated videos explaining these built-in nodes more thoroughly, but for the time being: the channel socket of a port is where external information is delivered. So for example, if you enter a number into a number socket, it arrives through this channel connection and is then provided in the port's value output socket at the bottom. the slot socket of a port node is used to specify in which order different ports are displayed in the external interface. Any rendered ports are sorted by their socket values. So a port with slot 5 will be visible after a port with slot 3 for example. I'll cover repetitive ports in the aforementioned video properly. Some socket types can be made repetitive, allowing for multiple inputs into the same socket type. It's a bit like "rest" parameters in javascript. The white dots are the connection nodules. Some socket values can not be specified via connection and have to be input directly. This can also be achieved in custom nodes by not specifying a type, but i'll get into that in a future video. Any given input socket / port can have more than one data type associated with it i.e. multiple types of data can be used as input. If more than one type can be received, at least one of them needs to be marked as the default type. This is necessary in case the data is provided directly and not via connection. If a value is input directly, it will always be the default type specified within. The "custom" input in the type node is used to switch between a built-in type selection and a text input to define a custom type. If that socket is set to "custom", an arbitrary string can be input into the field below, which will act as a type name used in the resulting socket. If that socket is switched over to "built-in", a selection of three built-in types is made available (text, number and truth). Hope that cleared some things up :)
Are the nodes already usable or are they just as visual representation right now? If you already made them functional I would be interested in knowing how you did that without using any code
Yes, they're already usable. All nodes you see in my videos are just screenshot cut-outs of the actual program. Not quite sure what you mean with "without using any code". The node graph editor was programmed using standard programming approaches (unfortunately). You can think of a node as a visual interface to a simple function that transforms a bunch of input into one or more outputs. At 1:33 you can see the built-in "format" node, which is basically the core functionality of the integer types. You pass in some data string into the "data" socket and then specify what that data is originally in the socket of the same name. In this case selecting "integer" basically tells the node to expect a string which contains a number without a decimal point. You can then select a target format in the "target" socket, in this case integer again - it's a bit confusingly named, because the "integer" option in the "target" socket refers to the actual binary representation of a number, not the textual form. Based on your "target" format selection, additional options may become available - in this case we can specify if we want our output to be signed (twos-complement) and how many bits it should have. The output then contains the binary representation of whatever number is provided in the "data" input. As I explained in the video, the output of the format node is always big endian - but the ISA we'll target needs numbers in little endian format. So all we need to add is another built-in node called "reverse", which just flips the byte order. Then all you need to do is connect some built-in "port" nodes to the input and output and the program can interpret that node graph as a custom node, which is what you see at 1:57. I don't know if that answered your question though :D There is code running behind the scenes, I just don't have to type any. It's a bit like having a bunch of building blocks and just combining them to get new building blocks.
This prototype is an electron.js app, so whatever they use to render the HTML and CSS files for the UI and vanilla JavaScript for the backend. I attempted making it in C++ with my own little Vulkan graphics engine, but the amount of debugging necessary was just preventing any sort of progress. I'm still getting millisecond compilation times for large node networks (although I obviously haven't been able to test it on proper large projects yet), despite the choice of programming language - kind of speaks for the theory behind everything I suppose.
@@Kronark that's honestly amazing, given how notoriously slow node and JS can be. I wonder if it might be worth using FFI to write the compiler in a language like C++/Rust and then call those functions in the electron app
@@kainoa82858 might speed things up a bit, but ultimately not necessary currently. I personally don't mind waiting if it eventually starts taking a bit longer (e.g. a full second lol), I think it'll be more efficient to get the prototype bug free and maybe a little optimised - and then just build a proper release version of the program using the prototype itself. Then it will be faster than FFI and most likely native C++ / Rust anyway - because then I can apply some low level optimisations for this very specific use case, which those other compilers just don't have.
@@Kronark Nothing to be saved for, as poc and being graphical I think is a great choice, also, it doesn't mean the output code is going to be js, so where is the harm? Btw, I'm a big fan of node graphs but is quite hard to find resources for node graphing libraries and tools because, well, node.
I disagree, but I see where you're coming from - some programming languages have a "byte" primitive type (I think java among others). In my opinion, the most basic form of data is data that doesn't assign any additional meaning to the bytes used to represent it. Floats aren't basic because different bits in the bytes have different meanings for the data stored. Colours aren't basic because each byte represents a different component of the stored data. One might even argue, that SIGNED integers aren't basic, because the first bit technically represents the sign of the number, not an actual value. But UNSIGNED integers don't have any underlying components or bits with special meanings, they are equivalent to the concept of one or more bytes. In essence, "byte" is just a fancy word for an 8-bit integer, same as "word" for 16-bit integers and "double word" for 32-bit integers etc... I personally think, bytes shouldn't be classified as a data type, since they're simply the underlying format any data is stored in on a computer. For me, a *data type* in the context of programming is something a human can understand intuitively - like numbers, colours, text and so on.
@@bedro_0Only if you choose to interpret it as such. Those "numbers" can also be pointers or instructions depending on what you tell the hardware to do with them. Integers are still an actual datatype that requires specific handling
Booleans appear simpler than integers on the surface, but they can actually be a little more complex under the hood. This is mostly due to the fact, that there aren't really any single bit values, only flag registers - which themselves are also technically just a bit-packed integer. I consider the integer the most basic form of data, since it's the natural state of any sequence of bytes. Everything else are just integers bestowed with some meaning e.g. 1 = true, 0 = false in the case of booleans.
@@Kronark And yeah: Whenever I need to look into bytearrays or such I find that booleans often use way more space than just a bit. Thanks for explaining! 🙇♀
![Lil Uzi Vert - Chill Bae [Official Music Video]](http://i.ytimg.com/vi/D7F42F_JM3U/mqdefault.jpg)
Real ones were here before this guy's journey was adapted into a critically acclaimed HBO mini series
I‘m still not paying for another streaming service…
They’ll be paying you if everything goes well
I am one real
That's so impressive! I've never thought visual programming could be used at such a low level. Definitely will follow your journey!
Welcome aboard! :)
every gamedev at some point thinks "man this would all be better if i just created the entire computer system from scratch" and then realizes that would be a waste of time, would go nowhere, and the dev goes back to using godot
you let the intrusive thoughts win, and youre actually brilliant enough to do it. a true chad
Thanks! :)
You had me in the first half, not gonna lie 😂
Some of these devs actually had to create their own systems. We call them consoles, you know.
This video is so high quality, the code quality is also probably amazing
Eh... it's super heavily commented, especially in the complicated parts, because it's so easy to lose track in the core algorithm... commented to the point it's unnecessary ^^ I wouldn't say that's high quality tbh :S
oh I am EXCITED for the rest of this. Editing style is beautifully original and clean. Videos are short and simple without all the padding to take advantage of the algorithm preferring longer videos. It's perfect. I'll be here for every future video.
Welcome aboard! :D
I'm going to be rewatching this series so much lol
You better, I need 700 more watch hours to be able to earn something from the ads youtube places in front of my videos either way :P
I'm really excited at the thought of seeing a CPU simulator in action, with the node graph interface integrated. This video series is really exciting. Best of luck with the project!
Thank you! I'll need it :D
I’m here for the ride! This looks very interesting
Thanks! Welcome aboard :D
Fascinating project you're laying out here, I hope there would be an opportunity to contribute in future, this could really go some places! Depends on how far you're willing to take it.
There will, although that‘s probably further in the future than some people in the comments hope.
We‘ll get there eventually, gonna focus on actually making the thing work properly first and check if it actually works for large projects :)
This is just mind blowing.
Glad you think so! :)
Do you plan on making this open for anyone to use?
Not for now, but eventually, yes.
I'll not be releasing this prototype, the code quality is abysmal to be honest. I'm planning on getting the prototype feature complete and fully capable of developing any form and size of software - then I'll use the prototype to build a release-version of itself. It's most likely gonna be another couple of years until then to be honest, I need a break from compiler design once the prototype is done.
But yeah, once a proper release version is built, I'll be providing the software for free on my website - probably gonna build an online node-sharing platform to go with it.
@@Kronark "Use the prototype to build a release version of itself" - that sound wild χD
I wish you all luck/power/will and whatever else you'll need to get this thing done. Neat editing, btw 👍
Bootstrapping at its finest, eh? :D
Thanks!
@@Kronark That's how it's done! 👍👍👍
o7 I can already tell, this is gonna be a fun journey.
I hope you're right!
awesome idea! Can't wait until you somehow magically turn thousands of lines of annoying vulkan boilerplate into a single node
That one‘s gonna be super satisfying :D
brooo, this is looking insane, i hope i'd be able to use this, i've always dreamed of making a mixed node/scratch like programming language but never got past graphics since SDL is slow, OpenGL is overkill for such a simple program and for the sake of commiting not live anymore i want to make it in zig. Anyway good luck my man, take your time.
My dream is to do freelance software development - this compiler will probably be the first product on my page (and it'll be free) together with a little node sharing platform. So you'll definitely get to use it some day! But that's still a bit in the future :D
I'll have to take that last part to heart man, the growing pressure is stressing me out right now ^^
@@Kronark btw, if you need i'd be glad to help :)
Weird to see the overlap of people interested in such niche technologies. I'm a big Zig guy and was considering making a graphical language that transpiles to Zig, but I've got too many other projects on my plate right now
I'm trying to understand the benefits of this representation vs coding in assembly. I guess the hierarchical abstraction of code into higher level nested nodes helps, and makes clearer what the inputs and outputs are, whereas in assembly the calling convention is implicit (e.g. args might be on the stack or in registers).
You got it :) This will be extremely similar to any other assembler in the beginning, but we'll work our way up to higher level abstractions eventually + add new compilation targets that no assembler currently targets. We have the control of an assembler, but the ability to do so much more.
Question: is compiling executables for other OSes on the roadmap or will only windows executables be supported?
absolutely, in fact most modern binary and text form platforms are supposed to be supported eventually. So not just x64 for all OSs, but also ARM, RISC-V etc - and then also html, css and javascript, python, c and c++ source code etc.
Will you also implement arbitrary precision integers?
Of course! We'll actually need those eventually once we start implementing the release build of this compiler using the prototype - arbitrary precision integers are essential for parsing number strings
hope this is released to public before gta VI
Before HL3
How do you represent a stream of bytes? How do you interpret them? As an array of bytes? But how do you process that? How do you KNOW that you are getting a number and not a stream of bytes? How do you handle explicit types? How do you handle global variables? Even loops would require problems. And what about for each element nodes? You speak of compilation, but I don't see how this could be anything other than interpreted. It does sound really interesting though.
Data is passed between nodes via an object. This object stores the data type in addition to the data itself and some other metadata. Each input socket / "port" node has to have at least one "type" node associated with it to be able to receive a connection. Output sockets can only output data of a single type. For a connection to be created, the type of the output socket has to match the type of the input socket. This ensures that no malformed data is ever received, except of course you break it on purpose. That's how I know that I'm receiving a number and not a stream of bytes. And of course, there also is an "inspector" tool which I haven't shown yet, which lets me inspect any given node and see what data is provided at each socket and what type that data has.
Regarding global variables, that's an upcoming topic - there's a state system for node instances. But that's a bit to complicated to explain in a RUclips comment.
Regarding loops, there is a built-in compiler-loop node planned which will allow repetitive node executions. If you mean loops in terms of actual output instructions, why would there be a problem? Jump instructions are just like all the other instructions - If you're worried about the determination of a jump destination, that will also be solved using the aforementioned state system.
What do you mean by "for each element nodes"?
It's a compiler by definition because it transforms a high level (visual) language into other languages (anything that's a bunch of bytes in a file). An interpreter (again by definition) directly executes the high level language it is given - which is not the case here.
The more complicated topics you mentioned will be tackled eventually - just know I have already thought about them and have found solutions for them.
Hey, Kronark! Could this tool be used to reverse-engineer applications and present them in a node format?
Hey man! Not this specific tool, but additional software could be written once the necessary x64 instructions have been designed - then you‘d just have to match byte sequences to node outputs and build a valid node file from there, to be displayed with this node graphing frontend
This is dope!
Thank you!
I know you don't want to release a prototype but It'd be cool that as soon you are bored of it to make it open source. I'd love to participate in development!
Will definitely do that IF i ever get bored :D
@@Kronark I understand that you want to do it yourself now haha
fyi, The playlist is not available
excited to see the next one
Yeahhh, I‘ll fix that once the next video is locked in - that‘s the higher priority right now ^^
Thanks! :)
Cool idea! The flashing animation is a bit distracting, however.
Thanks! Which one do you mean? :)
@@Kronark I mean the flickering whenever something fades in
Ahhhh ok. Yeah that‘s gonna stay unfortunately, fades would be too boring imo :)
Thanks for your video, the idea looks interesting.
If I get it, using graphs you will compile platform specific instructions. How your system will interact with OS? If it will not interact with system, how you will handle user input/output?
Also what about scaling? If you will write something more complex than arithmetic you will have very large graph, so I am curios about modularization or some way of splitting graph into reusable components.
You can easily overflow integer after multiplication. How you deal with undefined behavior? What about debugging?
Initially yes, it‘ll be platform specific, but I‘ll eventually implement a state system to switch between different platforms - so the instructions won‘t be platform specific from then on.
OS interaction is done through system calls, which are just fancy CPU instructions that give control to the system for specific tasks (e.g. system in and out, filesystem access, etc.) - they‘ll be handled just like any of the upcoming arithmetic instructions
Ah yes, that‘s something I didn‘t touch on because it was a bit unnecessary for now in my opinion. The program comes with a module system. Each node is part of a module and modules can be part of other modules - basically a standard folder-file approach like in operatings systems. Modularisation and node-graph reuse is a central concept to the entire thing. You actually don‘t have any other choice since there are hard limits on node instance count within a given node (sub-) space.
Regarding integer overflow: True, but for now I just wanted to prevent value overflow on definition. We‘re effectively preventing the equivalent of „uint_8 var = 300“ or something like that. overflow during multiplication is usually handled by increasing the bit count of the result variable as far as i know - i think that‘s what the x86 machine instructions do if i‘m not mistaken. We‘ll deal with other undefined behaviour the same way it‘s dealt with in other approaches :)
Debugging will be done initially via print statements, just like it‘s done in traditional contexts. I‘ll add dedicated debugging functionality sometime in the future - not the highest priority right now.
This is amazing. Yet so obvious. Like all genius ideas. Say, will nodes be able to have side effects?
Also, I think you should start with the ELF executable format (for linux), it is much simpler than EXE.
Thanks!
Side effects as in „output one thing, but also add something somewhere else“? - yes, there‘ll be a state system we‘ll use for global variable declarations any possibly register assignment.
Yes, I‘ve worked through the ELF format already. However, EXE isn‘t super complicated either imo. Also, my desktop PC is windows and so is the hardware my master thesis stuff will be tested on - unfortunately i‘m locked in regarding that, it wouldn‘t make sense to spend time setting up linux right now.
But we‘ll add support for it next year, after my thesis is done.
Will you ever share the compiler or at least show snippets the code that's building this? I feel like this is a really interesting idea, but doesn't really feel fulfilling to me to not see any of the code that allowed this to exist. Kind of just hand wavey magic.
Eventually, yes. Since this has been commented a lot, I may consider moving forward on that a bit earlier... but right now I need to get out of the woods regarding my masters thesis - there is still a lot to do before I can tackle anything close to light simulation, and that takes higher priority right now.
What benefits do you expect to gain by using a node network over writing assembly code? Generally I found node programming to be fairly unergonomic due to reliance on mouse. Further I feel like any node network takes up significantly more screen space than corresponding code would have. As such I have a harder time reading the program as it requires more scrolling/moving around.
Getting a gui with a spinning triangle up and running with this system would be truly impressive. But I am curious how far you can manage especially with such an ambitious timeline. Left a sub and will check in every so often :P
There definitely is a trade-off every user has to determine for themselves, I personally don't mind mouse usage - In the end, most of my programming time is spent conceptualising and debugging, not navigating. A lot of stuff is made easier via hotkeys already and I am planning on adding navigation via keyboard in a future public release build e.g. jump to different nodes on the grid via WASD or arrow keys, jump to a certain socket via another hotkey combo etc. - It then would be pretty close to editing a text file in my opinion.
This compiler concept heavily incentivises coding in a reusable manner, since there is a hard node limit per given node space (= basically a grid that defines any given node) and clarity decreases rapidly the larger a network gets (as you mentioned) - so what I've experienced so far is that any given network on screen actually stays pretty small, because you keep wrapping sections into new custom nodes that hide all the visual complexity behind a more compact format.
My main problem with current programming practices is the lack of control I have over what actually is written into the final executable. Yes, I could try writing it in assembly, I could technically build custom abstractions using macros - but text editors just aren't made for building large amounts of abstraction layers, I would eventually just have huge files that have nested macros in them, which will just become a huge pain to debug in the end. I think that's probably one of the main reasons why we have programming languages if I'm not mistaken - but then when I'd use basically any programming language, I have to rely on 3rd party implementations regarding optimisation and I lose full control over my output. All this node compiler does is string bytes together, the user controls everything else - there's definitely gonna be a lot of setup work, but I think it's going to be worth it in the end.
Additionally, using node networks allows for some heavy compile time optimisations - providing me with a much smoother development experience since I essentially never have to wait for a build to finish. It'd be interesting to see how other people use the tool in 2 years or so, once I start doing public beta-tests or something like that. I definitely think the right community could make this tool superior to what we have now.
I actually won't be using triangle geometry in my masters thesis - it'll be an implicit surface rendering approach, so compute shaders rendering spheres and boxes all the way :D
Oh, I'm just as curious as you are, thanks for sticking around :D
@@Kronark You seem very enthusiastic about this, and I don't want to dampen your motivation. I just enjoy playing devil's advocate.
Regarding your main concern, I believe that any form of abstraction inevitably results in a loss of control over the binary. For instance, if you provide a standard library with common nodes, like a "print line" node that allows me to print text directly to the console, I, as the user, would need to delve into that node to fully understand how the printing is implemented. This includes understanding any implicit type conversions, as explained in your video, and how your node optimization works. Without this deep understanding, I would no longer have control over the bytes in the output-I would be relying on your node implementation much like I rely on the print line function in C and the gcc optimizations.
Moreover, controlling the bytes in the executable is just one aspect of output control. We still depend on abstractions from the operating system to load our program into a memory location as it deems appropriate and to access resources. The overall system's output when running still relies on many abstractions that we likely don't fully grasp.
@@kn0rkk All good mate!
Regarding your print-function example, it's true that you need to delve into the subject matter a bit to understand lower level abstractions - but the key difference in this node editor is, that you can do that at any time. You don't have to clone some git repository of the source code and you don't have to decipher a specific programming style. It's always right there at your finger-tips (quite literally) in the same format as any other node network. There is no hurdle to get to the lower level implementation besides potential missing know-how to properly grasp it. Additionally, there is no such thing as closed-source for node networks in the compiler. The node network of any functionality has to be available on your device in a compiler readable format - and if the compiler can read it, so can any user. When trying to understand library functions during traditional programming, you may not always have access to the source code, only the library binary.
Regarding the OS, absolutely. This compiler only shaves off any black box design in user-space, not in the entirety of a software lifecycle.
What language are you using to write your graphical code?
this prototype is just an electron.js app, so the web stack - the release product will be made using the prototype and the Vulkan API.
What is the meaning of "channel", "slot: 1", "no repetition"? Why some sockets do not have any white dot? And what is the meaning of "default" and "custom" in the integer type node? Thanks
I'll be making some dedicated videos explaining these built-in nodes more thoroughly, but for the time being:
the channel socket of a port is where external information is delivered. So for example, if you enter a number into a number socket, it arrives through this channel connection and is then provided in the port's value output socket at the bottom.
the slot socket of a port node is used to specify in which order different ports are displayed in the external interface. Any rendered ports are sorted by their socket values. So a port with slot 5 will be visible after a port with slot 3 for example.
I'll cover repetitive ports in the aforementioned video properly. Some socket types can be made repetitive, allowing for multiple inputs into the same socket type. It's a bit like "rest" parameters in javascript.
The white dots are the connection nodules. Some socket values can not be specified via connection and have to be input directly. This can also be achieved in custom nodes by not specifying a type, but i'll get into that in a future video.
Any given input socket / port can have more than one data type associated with it i.e. multiple types of data can be used as input. If more than one type can be received, at least one of them needs to be marked as the default type. This is necessary in case the data is provided directly and not via connection. If a value is input directly, it will always be the default type specified within.
The "custom" input in the type node is used to switch between a built-in type selection and a text input to define a custom type. If that socket is set to "custom", an arbitrary string can be input into the field below, which will act as a type name used in the resulting socket. If that socket is switched over to "built-in", a selection of three built-in types is made available (text, number and truth).
Hope that cleared some things up :)
Are the nodes already usable or are they just as visual representation right now? If you already made them functional I would be interested in knowing how you did that without using any code
Yes, they're already usable. All nodes you see in my videos are just screenshot cut-outs of the actual program.
Not quite sure what you mean with "without using any code". The node graph editor was programmed using standard programming approaches (unfortunately). You can think of a node as a visual interface to a simple function that transforms a bunch of input into one or more outputs.
At 1:33 you can see the built-in "format" node, which is basically the core functionality of the integer types. You pass in some data string into the "data" socket and then specify what that data is originally in the socket of the same name. In this case selecting "integer" basically tells the node to expect a string which contains a number without a decimal point. You can then select a target format in the "target" socket, in this case integer again - it's a bit confusingly named, because the "integer" option in the "target" socket refers to the actual binary representation of a number, not the textual form. Based on your "target" format selection, additional options may become available - in this case we can specify if we want our output to be signed (twos-complement) and how many bits it should have. The output then contains the binary representation of whatever number is provided in the "data" input.
As I explained in the video, the output of the format node is always big endian - but the ISA we'll target needs numbers in little endian format. So all we need to add is another built-in node called "reverse", which just flips the byte order.
Then all you need to do is connect some built-in "port" nodes to the input and output and the program can interpret that node graph as a custom node, which is what you see at 1:57.
I don't know if that answered your question though :D There is code running behind the scenes, I just don't have to type any. It's a bit like having a bunch of building blocks and just combining them to get new building blocks.
What's a port node ? Is this some basic concept I'm missing and should read up on ?
The latest upload should explain that :)
Lmao I wanted to do sth like this too
Hey There bud , Do you happened to have a Subreddit or something of that caliber ?
Hey man, not yet, no. I started this channel 2 days ago… :D
Why?
What are you using for the UI ? And which language are you doing all of this in ?
This prototype is an electron.js app, so whatever they use to render the HTML and CSS files for the UI and vanilla JavaScript for the backend. I attempted making it in C++ with my own little Vulkan graphics engine, but the amount of debugging necessary was just preventing any sort of progress. I'm still getting millisecond compilation times for large node networks (although I obviously haven't been able to test it on proper large projects yet), despite the choice of programming language - kind of speaks for the theory behind everything I suppose.
@@Kronark that's honestly amazing, given how notoriously slow node and JS can be. I wonder if it might be worth using FFI to write the compiler in a language like C++/Rust and then call those functions in the electron app
@@kainoa82858 might speed things up a bit, but ultimately not necessary currently. I personally don't mind waiting if it eventually starts taking a bit longer (e.g. a full second lol), I think it'll be more efficient to get the prototype bug free and maybe a little optimised - and then just build a proper release version of the program using the prototype itself. Then it will be faster than FFI and most likely native C++ / Rust anyway - because then I can apply some low level optimisations for this very specific use case, which those other compilers just don't have.
you earned a stalker and a subscriber
F*ck yeaaahh
Where can I contact you?
There‘s an email in my channel info :)
Just a note, you never actually linked the introduction video in the description
Thanks! Should be fixed now :)
@@Kronark Not a problem, definitely didn't hurt the enjoyment of the vids :)
interessante!!
gracias!
Subscribed. This is incredibly cool. What language are you implementing this in? Can I guess?
Is it Rust?😊
Thanks!
And no, this prototype actually is an electron.js app, so javascript (god save my soul).
@@Kronark Nothing to be saved for, as poc and being graphical I think is a great choice, also, it doesn't mean the output code is going to be js, so where is the harm?
Btw, I'm a big fan of node graphs but is quite hard to find resources for node graphing libraries and tools because, well, node.
hahaha yeah I've run into that problem as well, you always need to add that "graphs" when googleing about nodes :D
wouldn't bytes be the most basic form of data?
Every number from 00000000 to 11111111 is inherently an integer by itself, so your comment makes no sense.
@@bedro_0 what?
I disagree, but I see where you're coming from - some programming languages have a "byte" primitive type (I think java among others).
In my opinion, the most basic form of data is data that doesn't assign any additional meaning to the bytes used to represent it. Floats aren't basic because different bits in the bytes have different meanings for the data stored. Colours aren't basic because each byte represents a different component of the stored data. One might even argue, that SIGNED integers aren't basic, because the first bit technically represents the sign of the number, not an actual value. But UNSIGNED integers don't have any underlying components or bits with special meanings, they are equivalent to the concept of one or more bytes. In essence, "byte" is just a fancy word for an 8-bit integer, same as "word" for 16-bit integers and "double word" for 32-bit integers etc... I personally think, bytes shouldn't be classified as a data type, since they're simply the underlying format any data is stored in on a computer. For me, a *data type* in the context of programming is something a human can understand intuitively - like numbers, colours, text and so on.
@@bedro_0Only if you choose to interpret it as such. Those "numbers" can also be pointers or instructions depending on what you tell the hardware to do with them. Integers are still an actual datatype that requires specific handling
can you show us how this can be achieved in coding level
What exactly do you mean?
Hey, just a little nitpick. Please number your episodes. Interesting premise though. ❤
Thanks :) And I just did :D
@@Kronark Thanks!
0:13 eeehh Booleans?? 🤔🤔
Booleans appear simpler than integers on the surface, but they can actually be a little more complex under the hood. This is mostly due to the fact, that there aren't really any single bit values, only flag registers - which themselves are also technically just a bit-packed integer. I consider the integer the most basic form of data, since it's the natural state of any sequence of bytes. Everything else are just integers bestowed with some meaning e.g. 1 = true, 0 = false in the case of booleans.
@@Kronark I see. When I think of implementing a calculator with just logic ops it would probably also just do with "integer" data only.
@@Kronark And yeah: Whenever I need to look into bytearrays or such I find that booleans often use way more space than just a bit. Thanks for explaining! 🙇♀
Windows ? Thanks but no thanks
🤷🏻♂️