How much memory are you using: C: a little Java: some Javascript: yes Haskell: it's trivial to measure, first you prove Furgelton's 2nd theorem of algebraic manifolds, then simply construct an operator system over the field of... Prolog: true.
@@simonmaracine4721 exactly, i compiled an android OS from source once. My 8 GB machine with 8 GB of zram, just couldn't keep up. Had to decrease Java heap size.
For some reason, learning about what happens at such a low level is very stimulating. Would be cool if there was an OS or computer with some interface to see in real time what's in your CPU registers, and what are the physical memory addresses of processes. Basically the under-the-hood details you wanna see.
You can use i r option in gdb. Run program with gdb and write i r in order to see the contents of the registers. I have used it before both for debugging and for understanding what's going on under the hood. You can use disass command in gdb in order to see the assembly code and wher you are in the assembly code. You can use nexti and stepi in order to go instruction by instruction. I find it very interesting that with C I can compile the program and then use gdb or objdump -d in order to inspect the assembly and the machine code instructions and follow what's going on. I learned the basics of assembly, how function calls, stack frames, stack pointer, base pointer, instruction pointer etc. work because of it. It is especially engaging because in C you can practice your knowledge by writing inline assembly (you can do it like this __asm__(assembly code goes here, put it inside parenthesis and put /n character after each instruction);) and you can even write machine code. You can write machine code instructions in a char array which you put on the stack and then cast it into a function (you can write typedef (*Function)(); Function f = (Function)charArrayWhichContainsCPUinstructions; f(); Just keep in mind that if you want to do this you have to compile your program by writing gcc -z -execstack... I don't have that option on Windows I don't know how to download it if it is even possible, but it works on my Linux. Another approach (which I have just tried out yesterday) is to use mmap with PROT_EXEC flag and put your machine code there and then do the exact same process that I have described earlier in the comment.
Another nice thing is that there are man pages which you can read which explain to you how different functions in C work, what headers you need to include etc.
Computers from years ago (1960s) would have the register values displayed in a series of lights (representing bits) on the front panel. I'm not sure when the idea died out -- probably went the computers got fast enough that the blinking lights couldn't keep up.
Could you make a video on how to/is code in memory executed? It's always been quite an obscure subject for me and I really appreciate the work you put into your videos. I think it would make for a very interesting video!
You could also write wrapper defines for memory alloc functions, which also provides the benefit of telling you where the memory is being allocated and how much
Great work. Can you do a presentation on how the a multicore server allocates multithreaded processes to its CPUS and discuss how the efficiency of these CPUS are effected according as the number threads increase. Also with that how the modern scheduler handles timeslices in a multicore system?
It is so good that on Linux I have man pages for C, it is basically a cheat sheet. I don't have to spend the time memorising what header to include, how is which constant called etc, what arguments and flags which function has etc. I studied C for one semester in university back then I had no idea that man pages existed (and neither did my colleagues as far as I know), so I don't know if it was considered cheating to read a man page on a test. I wonder if it is considered cheating to look at a man page on a test.
I guess the behaviour (reporting variable sizes in each run) is related to the implementation of libc library on your device, it is interesting to see other implementation`s behaviour like uclibc and musl in this context
Amazing video Jacob .. Just a small question, can this be used only for the process or a particular function inside a process as well? Let's say there is a big process in production code which involves multiple functions out of which I want to find out which functions are time consuming and want to optimize the same, can this tool be used for that?
I was wondering how to find out , what amount of memory OS gives to my program and how many pages are given without heap allocation? So is the vm max size the limit of the memory which is given to it?
today I was like yeahh I know now something about programming and some inside about it ;)... then I stumble upon this video and after watching, I read some comments that confirm my thoughts but.... ...then I realized hmm I know definitely nothing about computer science at all, haha WTH !
Is this paging out what's known as "swapping"? If yes, when we say that a machine is noticeably slowed down due to swapping, do we really mean that swapping is happening way more often than usual?
Yes, the operating system is using the swap file (or a partition) on the disk to move out rarely used pages to make room in the physical memory - which means you are running out of it. It should be the last resort to keep your applications running, as the disk access is much slower than RAM. If the swapping occurs constantly, those "rarely used" pages are still being accessed a lot, forcing the OS to swap them in and out all the time.
Are you thinking of offering an advanced OS course for online users? I am talking about really hardcore things that theoretical courses do not go into :) If you do, I would sign up. It has been too long since university, and the OS course there was not much to write home about.
@@JacobSorber Process scheduling, thread scheduling/synchronisation, things an OS does to be more reliable, I/O handling and interrupts, etc. :) Of course the textbooks have a lot of information, but this is mostly theoretical, copied from research papers, and mostly outdated.
could the variations be because of address space randomization from the os causing it to not align with pages, which causes pages not to be used fully?
No actually you may be right, I've tried with a sample program and addresses in the heap varies. It doesn't prove it's ASLR that causes it, as it may be libC initialization allocating a varying amount of memory each time (which is unlikely): $ for ((i =0; i < 5; i++)); do ./a.out; done maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffcb27884b4 heap=0x1d782a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff060c5e74 heap=0x8392a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff0335ae04 heap=0x12f72a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd87970e44 heap=0x22fb2a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd220bb174 heap=0x22d02a0 Code: static int in_data_segment = 42; int main() { int in_stack = 42; int *in_heap = malloc(sizeof(int)); printf("maxrss=%ld main=%p data segment=%p stack:%p heap=%p ", get_mem_usage(), main, &in_data_segment, &in_stack, in_heap); }
Now that I think about it that would require a bunch of emtpy space inside the page leading to a bunch of internal fragmentation seems like too big a waste...
How much memory are you using:
C: a little
Java: some
Javascript: yes
Haskell: it's trivial to measure, first you prove Furgelton's 2nd theorem of algebraic manifolds, then simply construct an operator system over the field of...
Prolog: true.
Love it.
Except that Java usually uses a lot more memory than "some"...
@@simonmaracine4721 exactly, i compiled an android OS from source once. My 8 GB machine with 8 GB of zram, just couldn't keep up. Had to decrease Java heap size.
Memory is to Java what cookies are to the Cookie Monster.
Me, not much. MS? heck a lot..
For some reason, learning about what happens at such a low level is very stimulating. Would be cool if there was an OS or computer with some interface to see in real time what's in your CPU registers, and what are the physical memory addresses of processes. Basically the under-the-hood details you wanna see.
You can use i r option in gdb. Run program with gdb and write i r in order to see the contents of the registers. I have used it before both for debugging and for understanding what's going on under the hood. You can use disass command in gdb in order to see the assembly code and wher you are in the assembly code. You can use nexti and stepi in order to go instruction by instruction.
I find it very interesting that with C I can compile the program and then use gdb or objdump -d in order to inspect the assembly and the machine code instructions and follow what's going on. I learned the basics of assembly, how function calls, stack frames, stack pointer, base pointer, instruction pointer etc. work because of it. It is especially engaging because in C you can practice your knowledge by writing inline assembly (you can do it like this __asm__(assembly code goes here, put it inside parenthesis and put /n character after each instruction);)
and you can even write machine code. You can write machine code instructions in a char array which you put on the stack and then cast it into a function (you can write typedef (*Function)();
Function f = (Function)charArrayWhichContainsCPUinstructions;
f();
Just keep in mind that if you want to do this you have to compile your program by writing gcc -z -execstack...
I don't have that option on Windows I don't know how to download it if it is even possible, but it works on my Linux.
Another approach (which I have just tried out yesterday) is to use mmap with PROT_EXEC flag and put your machine code there and then do the exact same process that I have described earlier in the comment.
Another nice thing is that there are man pages which you can read which explain to you how different functions in C work, what headers you need to include etc.
Computers from years ago (1960s) would have the register values displayed in a series of lights (representing bits) on the front panel. I'm not sure when the idea died out -- probably went the computers got fast enough that the blinking lights couldn't keep up.
Consider trying cheat engine
English is not my mother tounge , but when this man explains ,it flows like butter in my mind
Really, love the way you dig deep under the hood, coming back again and again
Could you make a video on how to/is code in memory executed? It's always been quite an obscure subject for me and I really appreciate the work you put into your videos. I think it would make for a very interesting video!
You could also write wrapper defines for memory alloc functions, which also provides the benefit of telling you where the memory is being allocated and how much
You're a legend.
This is actually very very useful
Great work. Can you do a presentation on how the a multicore server allocates multithreaded processes to its CPUS and discuss how the efficiency of these CPUS are effected according as the number threads increase. Also with that how the modern scheduler handles timeslices in a multicore system?
Thanks, Pat. That would be a serious video. I'll put it on the list and see what I can do.
Great. Looking forward to it
great vid, thanks a lot!
Thank you so much! This was brilliant... subscribed! 😊
It is so good that on Linux I have man pages for C, it is basically a cheat sheet. I don't have to spend the time memorising what header to include, how is which constant called etc, what arguments and flags which function has etc. I studied C for one semester in university back then I had no idea that man pages existed (and neither did my colleagues as far as I know), so I don't know if it was considered cheating to read a man page on a test. I wonder if it is considered cheating to look at a man page on a test.
thank you!
In windows instead of ps ux what is the alternate command?
How about a discussion of determining memory used/available on an embedded system?
How many pages does a program get by default to use, if any before needing a new page or does it depend on the size of the program being loaded
any chance you could do a virtual memory video at some point? i've always struggled with the concept
I guess the behaviour (reporting variable sizes in each run) is related to the implementation of libc library on your device, it is interesting to see other implementation`s behaviour like uclibc and musl in this context
Amazing video Jacob
.. Just a small question, can this be used only for the process or a particular function inside a process as well? Let's say there is a big process in production code which involves multiple functions out of which I want to find out which functions are time consuming and want to optimize the same, can this tool be used for that?
I was wondering how to find out , what amount of memory OS gives to my program and how many pages are given without heap allocation? So is the vm max size the limit of the memory which is given to it?
So this is how load balancers are made?
Love this video
I hope you also have a video which compare circular buffer using power-of-2 and the traditional circular buffer.
Sounds interesting. Are you looking for a speed comparison? Or a space comparison?
@@JacobSorber Yes both speed and space comparison. I want to know more about it and how the mask work in it.
Hi Jacob, I'm running into a problem of tracing the memory allocated on a thread (or a task on a thread pool). Is there anyway I could trace that?
It gives you a page to draw on!
today I was like yeahh I know now something about programming and some inside about it ;)...
then I stumble upon this video and after watching, I read some comments that confirm my thoughts but....
...then I realized hmm I know definitely nothing about computer science at all, haha WTH !
Is this paging out what's known as "swapping"? If yes, when we say that a machine is noticeably slowed down due to swapping, do we really mean that swapping is happening way more often than usual?
Yes, the operating system is using the swap file (or a partition) on the disk to move out rarely used pages to make room in the physical memory - which means you are running out of it. It should be the last resort to keep your applications running, as the disk access is much slower than RAM. If the swapping occurs constantly, those "rarely used" pages are still being accessed a lot, forcing the OS to swap them in and out all the time.
Are you thinking of offering an advanced OS course for online users? I am talking about really hardcore things that theoretical courses do not go into :) If you do, I would sign up. It has been too long since university, and the OS course there was not much to write home about.
Interesting. Thanks. What topics would you be looking for in that course?
@@JacobSorber Process scheduling, thread scheduling/synchronisation, things an OS does to be more reliable, I/O handling and interrupts, etc. :) Of course the textbooks have a lot of information, but this is mostly theoretical, copied from research papers, and mostly outdated.
Hi jacob can you do a vedio on object oriented programming
Just an OOP overview, or do you have a specific topic you're interested in?
What's the name of the font you're using
Which one? In the text editor? Video titles?
@@JacobSorber text editor
I would assume it is the default font family for Visual Studio Code which is "Consolas, 'Courier New', monospace"
Can you take vedio on kernel module?
could the variations be because of address space randomization from the os causing it to not align with pages, which causes pages not to be used fully?
user address space randomization aligns on pages. It's the base virtual address of the page that changes, not the offset inside the page.
No actually you may be right, I've tried with a sample program and addresses in the heap varies. It doesn't prove it's ASLR that causes it, as it may be libC initialization allocating a varying amount of memory each time (which is unlikely):
$ for ((i =0; i < 5; i++)); do ./a.out; done
maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffcb27884b4 heap=0x1d782a0
maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff060c5e74 heap=0x8392a0
maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff0335ae04 heap=0x12f72a0
maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd87970e44 heap=0x22fb2a0
maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd220bb174 heap=0x22d02a0
Code:
static int in_data_segment = 42;
int main() {
int in_stack = 42;
int *in_heap = malloc(sizeof(int));
printf("maxrss=%ld main=%p data segment=%p stack:%p heap=%p
", get_mem_usage(), main, &in_data_segment, &in_stack, in_heap);
}
@@unperrier5998 I'm not sure but I thought the offset into the page might be randomized too
Now that I think about it that would require a bunch of emtpy space inside the page leading to a bunch of internal fragmentation seems like too big a waste...
what is your text editor?
VS Code
Which open source IDE do you recommend for C/C++ development including embedded system development?
I don't usually use an IDE. Here's a video of me explaining why (a while ago). ruclips.net/video/Pqf6H1WSbeY/видео.html
Nice Video. Can You make video in fuse file system and data structure. #StayHome
Thanks. I'll see what I can do.
Linux FTW!!
Indeed.
Ps vz proc.psuedo file
Getrusage
you're hot, just saying. Also thx for your help
You're welcome.
Last
Not last
@@shahnawazazam last as of the time of posting this
Thora khaaya peeya kro0.