do you know how "return" works under the hood? (are you SURE?)
HTML-код
- Опубликовано: 31 июл 2024
- Programming is amazing. Computers allow us to do things that otherwise would be impossible. But sometimes, the code that we write feels like MAGIC. How does all of this stuff work?
Let's talk about how return works.
🏫 COURSES 🏫 Learn to code in C at lowlevel.academy
📰 NEWSLETTER 📰 Sign up for our newsletter at mailchi.mp/lowlevel/the-low-down
🛒 GREAT BOOKS FOR THE LOWEST LEVEL🛒
Blue Fox: Arm Assembly Internals and Reverse Engineering: amzn.to/4394t87
Practical Reverse Engineering: x86, x64, ARM, Windows Kernel, Reversing Tools, and Obfuscation : amzn.to/3C1z4sk
Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software : amzn.to/3C1daFy
The Ghidra Book: The Definitive Guide: amzn.to/3WC2Vkg
🔥🔥🔥 SOCIALS 🔥🔥🔥
Low Level Merch!: lowlevel.store/
Follow me on Twitter: / lowleveltweets
Follow me on Twitch: / lowlevellearning
Join me on Discord!: / discord 
Наука
first
i wasn't first
F
congratulations michel 👑
Nice
🍕
Note that this does not apply to all architectures but is specific to x86 (and maybe some others).
For example, on Arm, calls ("branch and link" instruction bl/blr) put the return address into a register (x30).
Thus functions must push the return value on the stack themselves before calling another function (functions that do not call other functions do not need to push it as the register containing the return value is never overwritten).
The return on Arm is just a normal branch instruction, but encoded in a different way such that the CPU is aware that it is a return, not just a normal jump to a register (this enables microarchitectural optimizations like a return stack buffer to speed up execution).
I really liked the video though :)
It would be nice if you can talk about out-of-order CPUs at some point since understanding (at least having a rough idea) how modern CPUs work prevents some optimization pitfalls one may fall into when just thinking about a "traditional" Van Neumann Computer.
And it is the gateway drug to the world of microarchitectural optimizations, side channels, and transient execution attacks :D
I mean.. there is the 6502 and z80, how about some 6800 too?
In MIPS... there's a jump and link(jal), which also saves the current pc+4 to a register 31(or $ra)... and it's responsibility of the subprogram to take care of it(pushing to the stack if it needs to call another function and popping it back).
By convention, registers 4-7(or $v0 to $v3, inclusive) are used for arguments, registers 8-15(or $t0-$t7) are used for temporaries and may be modified by inner functions calls, while registers 16-23($s0-$s7) are to be preserved at the return(if you really need to use them, you should push their values... and pop them back when you are done)
Finally registers 2 and 3(or $v0 and $v1) are the convention for the return.
I took a course in IBM 360 (370?) assembly in 1982 and my second favorite instruction was BAL - branch and link. My favorite was ZAP - zero and add packed (packed decimal), because, well, ZAP.
@@matheusjahnke8643 That convention is the o32 ABI, there's also the n32/64 ABI which defines a slightly different usage for the registers:
Arguments are now from $4-$11 ($a0-$a7)
Temporaries are from $12-$15 ($t4-$t7)
The callee saved registers are the same, $16-$23 ($s0-$s7), plus the others ($gp, $sp, $fp)
And the return registers are also the same $2 and $3 ($v0 and $v1)
The Arm calling convention expects the callee to preserve registers r4 to r12. Complier people know this so they would do this extra work (pushing and poping the registers in the callee if it requires extra registers). R0 is usually used as return value. Actually the first few registers as passing parameters
Turns out I ACTUALLY did know how they work.
Gotta admit that Assembly was very challenging for me, I think the most challenging task in my life, because it feels so alien. However I'm very slowly starting to make sense of it. This is a testiment to perseverance. No matter how difficult you think something is, continue. Take your time, try multiple times, look for different sources, whatever, but don't quit, ever. You'll get there, and after some time you'll look behind yourself and will realise how much you've learned. Thank you for this awesome video and helping come a little bit closer to really understand assembly.
The Journey is Hard Than it seems.
Hardness builds a man .
Assembly is very similar to basic in terms of program flow
Assembly is very easy if you start thinking about it in terms of physical circuits and the mosfet transistors and all their electric fields and depletion regions. Basically remember that it's all physics in the end, and assembly occupies some intermediate step.
It’s a lot easier to comprehend when you learn with computer architecture and organization. It’s why a lot of colleges combine the two classes together nowadays it’s usually called “Computer organization and Assembly” or something along those lines.
@@fackyoutube8452 very true, I remember going through the whole organization of basic 8086 arch and the registers and their uses etc and then it made much more sense what each instruction is doing and where its storing stuff and how its storing and updating etc
Do a video about stack unwinding on ARM next. I had a FreeRTOS fw that was running on ARM, and would dump the contents of the stack into the UART. We used this to reproduce the call stack. It was an interesting task and I learned a lot. Could honestly be a direct follow-up to this video.
A few things to add.
Floating point values on x86 are returned via the first scalar of xmm0 register.
In general the way data is transferred between caller and callee is defined by what's called "Calling Conventions". There are a bunch, and they are determined by the the hardware architecture and the operating system, among other things. Fascinating stuff really.
Calling conventions is something that you have to have in mind when you are calling a function from a different programming language. It is quite fascinating to delve into.
Another term worth looking into is ABI or Application Binary Interface
If the language is higher level, there's likely a calling convention layered on top of the architecture's convention, especially if it's a multiplatform higher level language.
Sick video, wish you made it before my compilation exam lol
Thanks sharing all this quality content !
love your content man! As an aspiring malware analyst, your content is always enlightening and fun to watch! Long live to low level languages!
The calling convention (stdcall, fastcall, cdecl) also controls how arguments are passed to functions and who's responsible for cleaning up the stack after returning
Great one. It is easy to forget this when not thinking about it directly and remembering this helps greatly in reverse engineering efforts.
teehee ty
@@LowLevelLearning bro I love you 💋💋💋
Loved the video :) Watched it to make sure I remembered what I learned from Assembly and Comp Arch courses
Thank you for good explanation, as a self-taught dev it really helps deeper knowledge ! Keep it up please 👑
Although I had a general idea of how it worked, it was nice to learn a few extra details about assembly, thanks! Great video!
Thanks for the video. I learned this back in the uni but totally forgot about it lol. Always good to refresh memory on low level stuff.
I actually knew this! Proud of myself.
same, from ben eater's videos iirc
Out of love and interest for a specific game I dove deep into it's assembly code using debuggers, even though I had no previous knowledge about assembly. I learned so many things just by observing which was a wonderful and fun task, and it's nice to see that these things are actually universal at least for x86.
What was the game, out of curiosity?
I've found most asm languages easy to learn once you've learned any one of them. While they have different instructions etc, they broadly work in a similar fashion.
@@jemmerllast8492 Don't know if RUclips is glitching right now but my reply seems to be gone. I am talking about Maniaplanet, the platform for the Trackmania 2 games.
I just got done with a computer architecture class where we learned about the risc-v architecture. 2 minutes into the video I realized I already knew how return works lol. Nice video
Great video, with very simple but very clear explanation! Even though I knew most of this, it clarified certain terms for me (stack frame, as the term for the fraction of the stack related to 1 function/subroutine scope).
Maybe I am going to learn ASM some day, at least the x86_64 version. At the moment I know a wide range of languages, varying between more functional to more imperative and from OOP to procedural, but I have never really done ASM. Although I have quite seriously played around with the IL assembly language from DotNet, which is certainly more abstract than real ASM, but a lot lower level than DotNet's C# language.
Basic understanding of assembly is important when using any language that compiles directly to assembly. A lot of programmers, even good ones, don't have a clue as to how some of this stuff works. My basic recommendation is to learn to read some assembly by taking a peek at the generated output of a simple program. Look up each instruction one at a time and work out what is happening. It is amazing to me how few programmers don't know some of this stuff.
The same is true for any language that compiles into some intermediate form, like JVM bytecode, BTW.
I learned a lot of assembly by doing exactly this but then going in, cleaning it up, adding comments, and gutting half of it because gcc generates some really inefficient assembly. Seriously, I've seen gcc do this:
mov %eax %ebx
mov %ebx %eax
multiple times. Still haven't figured out what possible purpose that could have.
@@nightfox6738 This is why I never trusted compilers. I've seen them do amazing things but also really dumb things. But that's a new low lol
Got my sub with that witty function
It’s cool to see this video and actually already knowing how return work
This was more interesting than I had expected. Thank you
Great video, I never *knew* how return statements worked, but what I always assumed was pretty much this.
How about a video on stack unwinding and C++-like exception handling?
Very informative and clear, thanks for this video!
I literally had to write about this in a final-exam yesterday about what CALL and RET statement does in 8086 assembly
Thank you for this. I tried to elaborate on the semantics of `return` (and the case of Rust's implicit return for expressions) to my classmates but failed to wrap the concept in words.
I only understand most of this coz of the assembly language course we did in 3rd year. Thanks for breaking it down.
cant wait to watch for the next five times while I try to figure out what basics I'm missing out to get this.
I got interested in assembly as a kid while learning to program. Never wrote anything in asm, but I learned how some C calls are broken into instructions, and the whole concept of the stack and registers. So my intuition was basically the same thing that this video laid out. Just wasn't sure if the return value was going on the stack somehow or a register, but the register made more sense.
The reality is that often in Assembly you're free to implement these conventions however you like. Some architectures like x86 have a "call" instruction that makes you jump to a particular address and also puts the return address on the stack, but really you could just jump and store the return address in a register instead.
The reason to have conventions, however, is that if your program is calling a C function it is often expected to follow the "C convention" (in the early days a "Pascal convention" was common on Apple Computers). And some architectures like MIPS just declare what the convention for programmers are, but it isn't really enforced.
Some instructions may assume that certain values are stored in certain places though. But Assembly is absolute freedom.. make as big of a mess as you like!
Awesome video! Thank you for sharing!
I watched because you asked if I was sure. 😊 I was but that's not a problem; this was nice and concise. Also nice to hear, at the end, something pentesters and CTF folks should appreciate.
Read the title, but have not yet watched the video...
It pops a value off the stack and writes that value to the instruction pointer. In the case the return statement is paired with a value, IIRC, the value is stored in memory and a specific register/pointer/flag is set to indicate there was a value returned from the subroutine. In general, at least; different architectures might handle things a little differently.
After watching the video...
I was more or less correct. What I referred as "the instruction pointer" is sometimes a counter in different architectures, but I covered that at the end with the "in general..." bit. My explanation of the returned value was vague/broad because it's been a while since I've dealt with low level code. Good to freshen up every so often.
An interesting video. I'll ret to it in the future. Many pointers I can count on here.
I absolutely love low level stuff. It's sad that we only learn the basics of ARM and x86 at uni. Great video. (and yes, I was sure I knew how it worked :P)
There's a very valid reason why we develop high level abstraction on top of low level architecture like x86 and ARM - to solve more complex problems that will take forever to accomplish using assembly
I totally agree. I used to be a teaching assistant in my old uni and we had a class where we covered assembly. It was so much fun. It was so annoying that the Uni did more advanced architecture classes for final year undergrads only after I left 😢
as someone who has an on and off interest in implementing esolangs, creating control stacks in software is a really common bit of boilerplate. Even a language as simple (and devoid of conventional subroutines) as Brainf**k may need to know several potential jump points simultaneously in order to function properly. It's really interesting seeing these things I typically think of as snippets of C code being performed completely "on the metal" as it were.
Great stuff!
Can you do a video on linkers and binary file formats?
also interesting (and useless) fact: when returning a struct from a C function (that's not gonna fit into 32/64-bit eax/rax register) a compiler allocates the struct on the caller stack and passes the pointer to it as an extra hidden argument to the called function (callee), which is also what one would typically do in C when wanting a function to "return" multiple values. On asm level both do the same thing.
Return value optimization. This is why you shouldn't be afraid to return structs.
This is more complicated than I assumed it would be. I'm glad I clicked on this so that I could understand it better.
Ah damn, the title made me unsure but I actually knew all dat,
but it was very nice to watch anyways 👍
Returning function calls was the first concept I really understood without explanation! Lol!
Funny enough, for the longest time when beginning learning about OOP, "return" was the most difficult thing for me to wrap my head around as previously in learning "echo"ing was the "alternative" used with "returns".
Love your videos!
I figured it would save the return adress at call, and moving the stack related pointers actually explain lifetimes of stack variables quite well.
Yup! The way the stack frame is constructed is the reason local variable scope exists. Thanks for watching!
I was able to follow this reasonably well because of how carefully and methodically you speak, but I laughed when you said "don't worry, it's not complicated"
Would love to see some kind of reverse engineering series in which you go step by step, reversing a real-life application and explain every difficulty that you come across.
We did something similar to that at Uni. We were given C code and had to turn it into MIPS (the assembly language Playstation 1 was written in). For small programs it's okay, but for a real-life application written in a high-level language like JS or Python, it would take weeks, months or even years I imagine... Our C was really simple, for example, figuring out if a number is a leap year. Assembly is fun, you can just "go to" whichever line you like lol. I found it really enjoyable but I think C runs almost as fast, so cost-to-benefit ratio-wise, it isn't worth coding in assembly. Check out Rollercoaster Tycoon from 1999, as it's coded entirely in assembly which is pretty cool :)
@@FreeWithinMe I've learned assembly but I barely understand C, and often with C, I have to figure out how to get the toolchain working with makefiles etc... it's a huge pain. I'm not sure which route to take since most people learn C before asm and I ended up doing the opposite
The best way to learn this stuff is to implement your own/sort of programming language. I never completed it, but you had to think about go to instructions to implements ifs and loop statements and return statements and pushing parameters to functions.
Turing Complete is also a good game to learn that kind of stuff
lmao
I remember ret had a parameter as to how many bytes to pop from the stack. I pretty much used it in conjunction with ADD SP, n for garbage values. They were faster than popping regs off the stack. Experienced lots of crashes messing around with those regs. Lol
Years later, made an interpreted language that called functions without a stack(challenging myself or just masochism). Had to invent a sort of name mangling scheme so that I could fetch functions from the symbol table itself along with the return address ina vectorized AST.
A really clear refresher, this is what I love about this channel
I actually guessed this is how it's done. Such a simple but brilliant way to handle it
It's unfortunately also the reason why buffer overrun attacks exist. If an array stored on the stack is indexed out of bounds, a hacker can overwrite the return address with the address of any function he wants and "return" to it, even if the program hasn't been there before.
There are different "calling conventions" for calling functions. __stdcall and __cdecl are the most popular in C/C++. Keep in mind that C somewhat assumes that you already know how to write assembly language code as it is a mostly convenient layer over assembly.
I better understand it considering that I studied two courses of computer science (based on the well known book from Patterson) and I had to learn the basics (architecture and assembly) of MIPS-architecture. I don't know the details of X86 assembly though.
Which book you are referring to?
It's a bit different, in that there are fewer registers and more specialized commands. Also there are much fewer limitations on addressing modes (you can use register offsets for indexing unlike MIPS)
i was wondering how the "returrn" work thanks for the video love to see more video like this
IIRC The GEC 4000 series processors didn't have a stack - the return address was stored in an address in the subroutine header. This made for an interesting C compiler - it used the RY register as a stack pointer (fun days interfacing C with Babbage routines).
Just finished my Assembly class, so I did in fact know about how "return" works.
push pc onto the stack
frame pointer stores the location of pc
... code ...
return values go in r0-r3 registers
use the frame pointer to bring pc back
Basically, return and function calling in general works as if the called function had an implicit return address parameter. It's filled by the caller and used by the callee.
Named return value optimization (NRVO) does the same for the result: the caller makes space for the result and the callee has an implicit return parameter that is bound to the address of the space for the result.
You made me question my knowledge, but is is exactly what I learned at Uni. So yes. I'm sure
One thing I'd like to pointt out is that if you see something like "RET 8" in assembly, it is NOT the same as "return 8;" in C. In assembly, a number after RET tells the CPU how much to adjust the stack before/after returning (I forget which)
It adjusts the stack after popping of the return address into IP in 80x86 systems. Also, whether the value passed back is in the register or on the stack can be language and compiler dependent even on 80x86 systems.
As Alex said. When the calling code needs to get more parameters to a function than the CPU can hold in its registers, one strategy is to put them onto the stack before calling (another one is to put them somewhere in memory and have a pointer to that address in a register). So the stack would hold the parameters, then the return address, then the function's local variables. When returning, those extra parameters need to be removed from the stack, too. The calling code can do that manually (it put them there, so it knows how much to remove) or the function can use a RET that can do that and let the CPU do that for free.
That depends on the calling conventions. Some architectures have a “frame pointer” in addition to the stack pointer; IIRC the frame pointer would save the value of the stack pointer just after a jump to subroutine. Then no matter how much data the function pushed onto the stack all the CPU has to do on a return is copy the frame pointer back to the stack pointer to get back to where the return address is on the stack.
Your strlen will return 1 for an empty string.
Thank you for this video. It would be great if you could also demystify Lamba. It seems there is another level of complexity added in there
I couldn't help but notice that the strlen() implementation shown in the video has an off-by-one error. It always returns the string length plus one (i.e. it includes the null character).
In R, we don't have to explicitly call *return.* The function will return what ever value it lastly assigned
before watching this video i didnt actually now what "return" did, but after watching it, i now know that i dont know what it does
That was a great explanation
My name is not jeff
MYNAMEEHJEGF
And this is not the code report
my name, will never be Jeff either
Pfp says PP, but Names PK??
my name's not shane kid
On a 6502, I wanted to copy a disk volume to a RAM disk, the disk was then ejected leaving the RAM disk as the only boot device. I wanted to reboot the machine, this was achieved by a small machine language procedure which threw away the return value causing the reboot.
More of these please.
I think we need a video or series of the i2c and USB protocol with microcontrollers
I really like the hand shake between Arnold and a dude whose been pushing too many pencils as the image for an agreement!
Oooh. I know this one. Its the function epilogue. Functions get compiled into some bits and the function calling conventions have a prologue that stuffs the current program counter pushed to the stack. Return just pops that value back when the function returns. This is why recursive function calls can overrun the stack
Call stack has a limited size on most platforms. On Linux the default is 8 MB. So if a program uses more than that then a stack overflow error will occur.
8 MB is actually very generous, using more than that is quite difficult.
I remember the 6502 architecture had a hardware stack of 255 bytes. I dimly recall you had to push/pop two numbers.
Exceeding this stack caused a Stack Overflow fatal error.
Not sure how newer architectures do it, I imagine there is a stack in RAM which could be any size or location.
Don’t forget that the operating system also uses the stack.
@@darylcheshire1618on Monday modern systems every thread has its own call stack. They're randomly placed in memory to avoid hackers from accessing them.
@@darylcheshire1618 Modern systems typically use the upper portion of heap memory, if your stack gets two big it can start to clobber the heap but this doesn't usually happen except with a botched recursion or the like
I've built a cpu in logisim and implemented "JSR" (Jump to Sub Routine) and "RET" and a hardware stack and wrote a recursive factorial program for it so I do indeed know how return really works but this was a really cool video.
I believe that for m68k the instruction is not "call" but rather "jsr" or "Jump to SubRoutine". I used to make my own games for Sega Genesis when I was 12 years old 😅
Can confirm, about a year ago I was trying to make a genesis game. But I couldn't get the sound to work
u must be a genius
Best videos I've seen to answer the question, "How does my computer work?"
I always thought it meant return this value as in “give back this integer” because you can usually leave off the return if the function is void…but I’m also mostly a c# dev. Now that I think of it I had the same confusion with C# “yield” which I thought meant “yield this value” like a yield of crops or something lol, as opposed to “break” which “yields” nothing by itself. I think my interpretations aren’t too far off from the actual behavior though because like where in the word yield is there an implication that you can enter back into the loop. I guess “return” is a little more straightforward haha. That’s self taught CS for ya
I am self taught in the CS that I know and I actually came up with almost the exact same solution to the problem when I was interested enough to try a CS game that had me build a computer from nand gates. It was annoying for me having to repeatedly write the same logic over and over again where I needed it so, I had come up with putting Call and Return in as instructions, all Call did was take the instruction pointer plus instruction width and store that on the stack, and all return did was pop a value off the stack and jump to it. I still manually had to create a stack frame by pushing any state I needed to the stack before the call and popping all the state I saved off the stack after the return but it worked well enough that I didn't change it for quite a while.
You were correct about what return means in C-like languages. It takes a few more steps to do that in machine code however, as the video explains. If you had a function like this:
void doNothing{
}
it would generally be optimized away by the compiler, but let's say your compiler didn't do that it would look like this in ASM:
doNothing:
ret
returns are implicit from void functions. Basically a QOL feature for many languages that you don't have to explicitly write it but it's still there in the compiled machine code / assembly.
@@nightfox6738 I should have just explained it like this, it's much better than what I said
More of these, please
Great Video. Thanks
This reminds me of the compiler design course.
Without watching the video, I’m guessing that the memory location of the function call is pushed onto the stack and when return is called the location is popped from the stack and the program jumps back to where it left off. I’m pretty sure that’s how it works in assembly, so in a compiled language it should be the same.
Update: HOLY SHIT I WAS RIGHT LET’S GOOOOOOOO
nice, quality video, I am glad I chose to watch it
Abstract machines almost always work in this way. In real world processor architectures there are myriad of variations, mostly regarding the use of registers (because they are faster than memory based stack, although there are architectures where the pot of the stack is aliased to a register bank).
However, to make this useful to viewers sufficiently "naive" not to know this already, an explanation of the concept of stack (or LIFO) as a data structure is in order.
Yay new upload
Yes, I definitely understand how return works. I've learned nearly everything I know about assembly from Ben Eater. He is quite an excellent teacher.
I learned mostly from Chibiakumas
I learned about computers from Ben Eater and redstone
I love his breadboard computer series. It taught me so much about computer architecture.
I love "pointers" in Pascal.
Truly, this is one of the things that happens in a computer
awesome video! nice work
ty :)
I was thinking that when the function is first called, it can just have a return address, and the return in the called function returns to that.
Though I would have thought the compiler would pop the function off the stack and write the return value to the new top of the stack.
Exactly what I expected
Added to a playlist "Forever"!
Love these kinds of videos: Here my prediction before the video actually started: It depends on the actual CPU-architecture, if we are dealing with a regiter-less CPU, the result probably gets pushed on the stack somehow, either by the caller reserving memory on the callstack, or implicitly using the address after the return address. If the processor has no memory but registers, some registers will be chose. If we are talking exclusively X86/amd64 - it is mostly registers nowadays, but since now higher-level languages like JavaScript and/or Python could come into the mix, the waters get murky and it could be some random field in an interpreter as well... That would be my answer if this was asked in a job interview :D
Darn it... I thought this was about the result not only the return-address.. OK :D
1:36 I was thinking, bl and blr (powerpc asm)
So this is asm level. Good to remember. Thanks.
I've always assumed that it simply stores a pointer and returns to it once finished. I had no clue how it worked closer to the machine, but it was never something I thought much of. At least now I know a bit better.
how about knowing a _byte_ better
Seeing the stack grow upward really makes me feel dizzy (though I never have problems calling the lower end that "top" of the stecak :) )
Excellent.
I will leave it to the compiler thanks
Is there a subject or topic to study that goes more in depth into this lesson? There is learning assembly but how about learning how the cpu works and runs a program. All the stuff covered in this video. What would be a good resource?
I think Chibiakumas is a good resource (look it up on youtube.) The CPU in general terms operates a lot like a BASIC interpreter from the 80s. Now, there are some more complex things going on in modern CPUs, but in general your assembly code is executed from the start to the finish, from the top of your source code document going down, one line at a time. Chibiakumas' videos are more in the context of retro game consoles than computers, which do have a few differences to be aware of. For example, on a Game Boy or Sega Genesis, your "main()" isn't going to return since there's nothing to return to. Trying to return from main on one of those systems is more than likely going to crash, as you're just taking whatever junk is on top of the stack and blindly GOTOing it.
I like LLL cuz he makes videos on stuff you dont think about regularly but happen to be very interesting
I spent a decent amount of this video shouting STACK!!! at my monitor
Nice, cany ou please do a video on how a operating system "exposes" system calls, and how the execution of a normal code hands control over to the os, and how the os hands control back to the normal program? thank you.
It's mostly the same as actual functions. But instead of a call and ret(well it can be used for this as well, but I'm not sure if there is a modern OS that implements x86 call gates) it would raise a software interrupt or use a specific instruction(syscall/sysret or sysenter/sysexit for x86) to call the operating system. Every syscall on the user side are wrappers to this function: int syscall(int syscall_number, ...); that is actually called when syscalls are performed.
I remember when I was fist learning to code as a kid in the flash days being really confused about “return”