I had to dive pretty deep into this stuff when I wrote a VOL plugin for the HDF5 file format last semester at uni. Still really interesting to see your explanation video on it. it's comprehensive as always :D
LiveOverflow It's the Virtual Object Layer which catches the API calls from HDF and forwards them to the actual objects in memory. I wrote my plugin to optimize execution on a cluster so it used shared memory. Quite a cool project to work on :)
If you are OK with the string being unmodifiable, you don't have to allocate new memory for the char* in the struct. You could have just assigned a string literal to the char* and that string literal would be placed in the executable file, after the section which contains the CPU instructions. However, this memory is read only, the program is not allowed to modify itself (at least not this part of itself). Which means that there is no need to allocate additional memory for the char* in the struct if you want that pointer to point to a read only string. If you want the char* to point to a string which is modifiable (a string which is placed on the heap in this instance), then you need to allocate additional memory in which you will place the chars to which char* will point to.
Damn! You really know your shit dude; guess from now on, I'm taking more seriously your videos than a lot of my school lectures for my educational career
If you allocated the memory of the size of the entire struct then why did you allocate memory again for char* name? 8 bytes for char pointer (or 4 bytes on a 32bit machine) have already been allocated.
Pointer is a variable like other variables but what makes it different is that it points to a memory location and that memory locations contetnt can be read by dereferencing the pointer. So if it was a char arr[some size]
@@shobhitdogra7924 On my PC I can write f2->name = "Hello World"; and it compiles and runs correctly. I can also write char* text = "Hello World"; and it compiles and runs correctly. I am using gcc compiler. I see what you mean though, now I am a little bit confused as to why it works, because if I try to do the same thing with ints or any other type, it doesn't work, I get a segmentation fault. For instance, if I write int* arr = {1, 2, 3}; printf("%d ", arr[1]); This will not work, and it should not work, arr is a pointer, it is not an array. I can use that pointer to manipulate an array or read from an array but the pointer itself is not an array. It is just an 8 byte number which is a memory address of some other variable, or struct, or of another pointer. If I write the previously mentioned code, compiler assumes that I wanted ar to point to the memory address of 1, because 1 was the first number I wrote. However, I can write char* text = "Hello World"; and it will compile and run. I don't understand why I am not required to write char text[12] = "Hello World"; char* ptr = text; If I wanted a pointer to Hello World. In other words, what you are saying makes sense to me but that's not how my compiler works.
I don't know if you ever figured this out on your own, but in C a string between two quotes gets placed in memory somewhere, depending on platform and compilation options will tell you where, and you're merely assigning a pointer to this space that was allocated for you. So char *s = "Hello, World!"; will already have the space allocated for the string. In most cases you can't modify that space, and that is indeed standard behavior, but you can do char s[] = "Hello, World!"; and make it modifiable, but at a set length.
@@anon_y_mousse By now I know this. When posting previous comments I probably thought you have to write char string[20]= "this is a string"; And then you could write char *ptrToString = string; Or something to that effect. I was confused why writing a string can work in any other way than this way. char string[20] part takes 20 bytes on the stack in which you can put characters and the null character in the end, then char* ptrToString can point to the place in the stack where this array of char begins. So, writing char* s = "some text" ; did not make sense to me. In my mind, it would be logical that you first have to write char string[10] = "some text" ; and only then char* ptrToString = string; This is how int, double, float etc. work. You can't write int* arrayPtr = {0, 1, 2, 3, 4}; You have to write int array[5] = {0, 1, 2, 3, 4}; int* arrayPtr = array; I know by know that you are allowed to do this with char* And the example from the video makes sense now. When you are just writing char* string = "This is a string"; in the main function, compiler can put the chars and null in the end on the stack, no problem. If you make a strut with char* on the stack, compiler can again put those string bytes in the current stack frame (which means these bytes are lost after you exit the function). Still, no problem. However, when you are putting a struct on the heap then the compiler doesn't know where to put this array of bytes which represents a string. That's why in this case when you are allocating a struct on the heap you have to provide a place in memory where to put the bytes of the string. You have to either malloc that memory or just set the char pointer of the struct to be pointing to some place in memory where you have already stored the bytes of the string.
@@anon_y_mousse But I still don't know the difference in modifiability when it comes to char* str = "some text; And char[] str ="some text"; I haven't done much C in my life so I never noticed that the first one isn't modifiable. If it really isn't modifiable (I have no reason to assume you are lying) then why is that?
I don't get it. I do a malloc(16) but if I look on my heap I see a 0x21 (so 33). Shoudn't it be 0x20 ? ... Ok nevermind, I didn't watch until the end huhu
0x21 has the last bit set (as opposed to 0x20). That is the `prev in use` bit, which means that the block before it is in use (is allocated). Which means if you ignore that metadata bit, the size is still 0x20. I have kinda ignored that for this video, to keep it simple. Just keep watching the Heap videos, it will be explained :)
@@LiveOverflow only vid on here which actually explains the internals on malloc. Great job! A noob question. Why is the block size double of the allocated bytes?
In this program yes but in larger programs the mallocs would allocate a lot more of space (for example malloc(1000)) so the 8 bytes (blank space and size) aren't that big of a deal.
from my understanding your use of "the heap" here is not correct. malloc() calls do not give you memory from the heap, there is no heap (in C). as you explained, malloc() asks the operating system for a chunk of arbitrary memory, which you then get a pointer to, but that memory does not come from "the heap", it comes from wherever the operating system decides it comes from. "the heap" is from C++ where the compiler does set up an actual heap for you, effectively doing a single malloc()-like request for one single large chunk of memory at the start of the program. it then manages this chunk of memory for you, and when you request a `new` object, the memory allocation is taken from the heap (the previously allocated chunk of memory). and there's a whole big family of algorithms and problems that come from maintaining this heap. but this is C++-specific and does not exist in C. (unless you manually implemented one or something, using malloc()).
Did you pronounce the abbreviation for character the same as "char" (to burn)? Abbreviations are normally pronounced the same as the unabbreviated syllables aside from a vowel shift from change in stress.
I dont like intros. Distracting time wasting and yours is waaaayyyyyy too long mate. I understand you make money from hype but our attention is our money. Your customers are kids spending their money on junk. Those who spend mindfully would like videos that explains as much as possible in as little time as possible without overwhelming the learner so they can get to the next thing to learn instead of getting hypee candy fatigue.
I lost it at the "This is real fucking magic" part xD
me too thanks
Thank you. I am happy that this is not web app related :p
+ashikombat some more vids of this series are coming :)
awesome
I had to dive pretty deep into this stuff when I wrote a VOL plugin for the HDF5 file format last semester at uni. Still really interesting to see your explanation video on it. it's comprehensive as always :D
+SonKomischerTyp cool! What is VOL?
LiveOverflow
It's the Virtual Object Layer which catches the API calls from HDF and forwards them to the actual objects in memory. I wrote my plugin to optimize execution on a cluster so it used shared memory. Quite a cool project to work on :)
+SonKomischerTyp ah awesome. Sounds like a cool project. So you also had to deal with concurrent access and make it safe?
"This is all real fucking magic!" - LiveOverflow 0:54
If you are OK with the string being unmodifiable, you don't have to allocate new memory for the char* in the struct. You could have just assigned a string literal to the char* and that string literal would be placed in the executable file, after the section which contains the CPU instructions. However, this memory is read only, the program is not allowed to modify itself (at least not this part of itself). Which means that there is no need to allocate additional memory for the char* in the struct if you want that pointer to point to a read only string.
If you want the char* to point to a string which is modifiable (a string which is placed on the heap in this instance), then you need to allocate additional memory in which you will place the chars to which char* will point to.
Damn! You really know your shit dude; guess from now on, I'm taking more seriously your videos than a lot of my school lectures for my educational career
CGP Grey easter egg? Love it :D
Best educational channel on youtube, thanks a lot dude
My favorite part, "this is all real fuck magic!"" Hilarious!
Thank you ! Have a nice day :)
I f-ing love these vids. Can't wait for more!
Γειά σου ρε stratan master των pwners !
Thanks a lot. Much appreciated stuff :)
I didn't expect you to be using ad block. Great tutorial BTW.
I like your old videos as well as your newer ones! Also, is it right that I think from your accent that you're german?
This channel is amazing!
He said "This shit is truly magic..."
Much needed video, thanks!
2:24 :):):) best explanation ever ;)
I use malloc a lot. I had a basic idea of how it worked so this both clarified somethings and makes a lot of sense
great channel
Love the videos, thanks for taking the time to teach me all of this !
I didn't learn what malloc does. Too simplified.
sehr gut !
Can you pls put link to playlist in your video. Great job btw
+Бојан Драшко ah yeah makes sense. I will do it in a couple of days
You should make a follow-up for this and explain what kalloc() does on iOS, good video anyway
Yes please
Nice &CGPGrey
+Dima Kalchenko ;)
3:33 So the dword that stores the size of the chunk is counted in the size ?
yes
The website of expoit exercises is not working 😢
I cant figure out why the stick figure waves his arm!
If you allocated the memory of the size of the entire struct then why did you allocate memory again for char* name?
8 bytes for char pointer (or 4 bytes on a 32bit machine) have already been allocated.
Pointer is a variable like other variables but what makes it different is that it points to a memory location and that memory locations contetnt can be read by dereferencing the pointer.
So if it was a char arr[some size]
@@shobhitdogra7924 On my PC I can write f2->name = "Hello World";
and it compiles and runs correctly.
I can also write char* text = "Hello World"; and it compiles and runs correctly.
I am using gcc compiler.
I see what you mean though, now I am a little bit confused as to why it works, because if I try to do the same thing with ints or any other type, it doesn't work, I get a segmentation fault. For instance, if I write int* arr = {1, 2, 3};
printf("%d
", arr[1]);
This will not work, and it should not work, arr is a pointer, it is not an array. I can use that pointer to manipulate an array or read from an array but the pointer itself is not an array. It is just an 8 byte number which is a memory address of some other variable, or struct, or of another pointer.
If I write the previously mentioned code, compiler assumes that I wanted ar to point to the memory address of 1, because 1 was the first number I wrote. However, I can write char* text = "Hello World";
and it will compile and run.
I don't understand why I am not required to write
char text[12] = "Hello World";
char* ptr = text;
If I wanted a pointer to Hello World.
In other words, what you are saying makes sense to me but that's not how my compiler works.
I don't know if you ever figured this out on your own, but in C a string between two quotes gets placed in memory somewhere, depending on platform and compilation options will tell you where, and you're merely assigning a pointer to this space that was allocated for you. So char *s = "Hello, World!"; will already have the space allocated for the string. In most cases you can't modify that space, and that is indeed standard behavior, but you can do char s[] = "Hello, World!"; and make it modifiable, but at a set length.
@@anon_y_mousse By now I know this. When posting previous comments I probably thought you have to write
char string[20]= "this is a string";
And then you could write
char *ptrToString = string;
Or something to that effect. I was confused why writing a string can work in any other way than this way. char string[20] part takes 20 bytes on the stack in which you can put characters and the null character in the end, then char* ptrToString can point to the place in the stack where this array of char begins.
So, writing char* s = "some text" ;
did not make sense to me. In my mind, it would be logical that you first have to write char string[10] = "some text" ;
and only then
char* ptrToString = string;
This is how int, double, float etc. work. You can't write
int* arrayPtr = {0, 1, 2, 3, 4};
You have to write
int array[5] = {0, 1, 2, 3, 4};
int* arrayPtr = array;
I know by know that you are allowed to do this with char*
And the example from the video makes sense now.
When you are just writing
char* string = "This is a string";
in the main function, compiler can put the chars and null in the end on the stack, no problem.
If you make a strut with char* on the stack, compiler can again put those string bytes in the current stack frame (which means these bytes are lost after you exit the function).
Still, no problem.
However, when you are putting a struct on the heap then the compiler doesn't know where to put this array of bytes which represents a string. That's why in this case when you are allocating a struct on the heap you have to provide a place in memory where to put the bytes of the string. You have to either malloc that memory or just set the char pointer of the struct to be pointing to some place in memory where you have already stored the bytes of the string.
@@anon_y_mousse But I still don't know the difference in modifiability when it comes to char* str = "some text;
And
char[] str ="some text";
I haven't done much C in my life so I never noticed that the first one isn't modifiable. If it really isn't modifiable (I have no reason to assume you are lying) then why is that?
I don't get it. I do a malloc(16) but if I look on my heap I see a 0x21 (so 33). Shoudn't it be 0x20 ? ... Ok nevermind, I didn't watch until the end huhu
0x21 has the last bit set (as opposed to 0x20). That is the `prev in use` bit, which means that the block before it is in use (is allocated). Which means if you ignore that metadata bit, the size is still 0x20. I have kinda ignored that for this video, to keep it simple. Just keep watching the Heap videos, it will be explained :)
@@LiveOverflow only vid on here which actually explains the internals on malloc. Great job! A noob question. Why is the block size double of the allocated bytes?
Shouldn't it be allocated-size+8 bytes (2 words)?
Isn't dlmalloc wasting incredibly much space?
I mean: On your illustration it took up half of the space just for logistics
In this program yes but in larger programs the mallocs would allocate a lot more of space (for example malloc(1000)) so the 8 bytes (blank space and size) aren't that big of a deal.
is this house of lore in phrack issue 67?
from my understanding your use of "the heap" here is not correct. malloc() calls do not give you memory from the heap, there is no heap (in C). as you explained, malloc() asks the operating system for a chunk of arbitrary memory, which you then get a pointer to, but that memory does not come from "the heap", it comes from wherever the operating system decides it comes from.
"the heap" is from C++ where the compiler does set up an actual heap for you, effectively doing a single malloc()-like request for one single large chunk of memory at the start of the program. it then manages this chunk of memory for you, and when you request a `new` object, the memory allocation is taken from the heap (the previously allocated chunk of memory). and there's a whole big family of algorithms and problems that come from maintaining this heap. but this is C++-specific and does not exist in C. (unless you manually implemented one or something, using malloc()).
God, it would have been so nice if your channel had existed 5-6 years ago when things like this one looked like magic to me
Which IDE are you using?
The dark looking one is Sublime
Did you pronounce the abbreviation for character the same as "char" (to burn)? Abbreviations are normally pronounced the same as the unabbreviated syllables aside from a vowel shift from change in stress.
Finally! This has always bothered me when people say char, like to burn, instead of care. It's an abbreviation of character.
cant see text editor text proprly ,they are all dark mode
Why is 0x10 used for 8 bytes? 0x10 is 16?
Because the whole row is 16 bytes (including the blank space at the start and the size itself).
I thougt malloc returnedt void though
good video. but going way too fast. hard to follow.
I dont like intros. Distracting time wasting and yours is waaaayyyyyy too long mate. I understand you make money from hype but our attention is our money. Your customers are kids spending their money on junk. Those who spend mindfully would like videos that explains as much as possible in as little time as possible without overwhelming the learner so they can get to the next thing to learn instead of
getting hypee candy fatigue.
instead of "anyhow" say "anyway"
ع المطبخ