How to Set up Shared Memory in Your Linux and MacOS Programs. (shmget, shmat, shmdt, shmctl, ftok)
HTML-код
- Опубликовано: 29 июн 2020
- Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.thinkific.com
Website ➤ www.jacobsorber.com
---
How to Set up Shared Memory on Linux and MacOS. (shmget, shmat, shmdt, shmctl, ftok) // This tutorial shows you how to set up shared memory between two or more process in any Unix-style OS, using ftok, shmget, shmat, shmdt, ahd shmctl. Implementation example included.
Related Videos:
Shared Memory with mmap: • Simple Shared Memory i...
How programs get memory: • How processes get more...
More memory: • Keeping strcpy and mem...
Make: • Learn make in 60 seconds.
Cloning processes with fork: • Creating new processes...
Bit Fields: • Bit Fields in C. What ...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorber.com
people.cs.clemson.edu/~jsorber/
persist.cs.clemson.edu/
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
+ rep the channel with nerdy merch --- [teespring.com/stores/jacob-so...]
Source code is also available to Patreon supporters. --- [jsorber-youtube-source.heroku...]
Thanks for the awesome content, very well produced.
I'm literally binge watching the entire channel haha
I find your videos incredibly well explained. The drawings and explanations in the begining really help before getting to see actual code.
This video saved my soul in a current project.
Thank you very much, keep it up!!
greetings from Austria
That's an awesome explanation! You made is so much easier to understand this :) Thanks a lot Jacob
I'm loving the new layout and graphic highlighting! This was much easier to follow.
Glad you like it! Always trying make things a bit easier.
Great video Jacob.
To continue in the same subject, I'd like to see some example where share memory is useful.
Also, are multi process queue implemented with shared memory ?
Thank you for the explaination !!
Hi jacob .
Great video, would really like to learn about epoll in next video
Great knowledge sharing. 😄
Great video, would really like to understand semaphores in next.
Thanks. I'm not 100% when the semaphore video will come out yet, but it's coming.
Prior to the C89 standard, identifiers had a max length of 6. That's probably a big part of the function names.
Good point. I sometimes think I have repressed those memories. :)
That was excellent!
Hi Jacob. Thanks for the content. Would this same concept work to share memory between docker containers?
Hi! Love your videos. I would love a video of how to setup a testing environment for C. Is there a testing library for C and how do I set it up? I've heard that some people use C++'s Boost library to test their C code, but I'm not sure if that's the way to go...
Hi Jacob, thanks for the Awsome tutorial. I have a question about the file that we associate with shared memory. If this file(name) is one of the process (executable file) itself, will that may cause any issues??
I just wanna say..thank you..thank you so much..this is so helpful
You're welcome. Glad I could help.
Is shared memory actually faster than cached disk I/O? Cached I/O, especially if it's memory inside the L1 or L2 cache, seems to be as fast as it gets, right?
wow I just completed my assignment a week ago involving concurrency and shared memory... and you just now post this haha I could've used mmap but I preferred using shmget and shmat for my assignment.
Nice. I hope it worked out ok, even though I was slow.
@@JacobSorber No worries it worked out perfectly fine! I actually plan to help my peers by referencing your videos! Keep up the great work!
Hi Jacob,
For ftok system call, the man page written "Today, proj_id is an int, but still only 8 bits are used. Typical
usage has an ASCII character proj_id, that is why the behavior is said to be undefined when proj_id is zero", but in the example you put the 0 there, I tried several alternatives, it seems fine with 0 or nonzero. Do you know why? Thank you
Outstanding!
Thanks!
Hi Jacob,
Good job in making these topics more simplier.
I am a computer science instructor just like you and I highly recommend your videos to my students.
Keep it up!
Thanks, Artemio. I appreciate the support. Let me know if there are topics that you think your students would specifically benefit from.
@@JacobSorbermake a full c language course videos from beginning to advance level
Very useful. Thanks
You are welcome.
What is the logic behind separating the creation of a shared memory segment and getting a pointer to that memory segment?
8:16 that shmdt cracked me up.
What’s the difference of using those methods versus using fopen(), and writing and reading from a regular file?
Just an idea of something I'd love to see: make a video about sharing memory but between Python and C.
That would be fun. It would probably be basically the same mechanism, with some sort of python bindings on it. I'll see what I can do.
@@JacobSorber Thanks for the great videos. Does one already exist that covers the request of @John McKeown?
How do you use shared memory on bare-metal applications (e.g. on an ARM Cortex-A15 core)?
A video on FFT/signal_processing or bluetooth would be cool.
Thanks. I'll see what I can do.
Pronunciation of “shm” functions is what’s made this video
thanks a lot.
You're welcome! Glad you enjoyed it.
You are the best teacher! Do you actually prefer using VS Code instead of vim, or is it just for those lessons?
Vim and I have a complicated relationship. Still trying to figure out exactly how to define the relationship. Let's just say I like both for different reasons.
@@JacobSorber lol
It just doesnt feel like in the spirit of "Shared memory". Like all this in effect is doing is allowing shared access of a file that MUST exist on a disk. Although I rewatched the video and the part where you describe that I must not confuse them with mmaping content of a file but from what I get is shmget() is basically doing the same thing as mmap() but guaranteeing that contents of file are not changed.
Shared memory in essence or what I would normally expect is one process being the host of the memory and other just have access to it like it normally would. For example the earlier tutorials about shared memory for parent-child fit the definition of what is expected. But I think in this shmget method we are introducing uncessary mediators like the file that must access on the disk, and then the particiapting process would intearct with it like they would with a file (but without changing its contents). Seems like a very old and inconvineint shared memory api, because in theory, existence of a mediator (like a file) shouldnt be involved at all during shared memory. It should just work the same as they were in the parent-child scenario
I would love to hear your thoughts on it and if i said something stupid.
however this doesn't work from a sandboxed app on mac (shmget gives "operation not permitted").
It may sound dumb but if we are not using a common file as a means of sharing memory between processes, why do we need to specify a correct pathname of a file that must exist on disk?
This is definitely very interesting, but how does one prevent memory leaks in case the program crashes? Does the linux kernel know how to free memory if none of the programs that interacted with it are running or does that block of memory stay inaccessible until reboot?
You're right. This can get a bit tricky. The memory needs to be explicitly removed with shmctl, so if the program dies, you will have a leak unless you have another process that can clean things up. Depending on the application, it is often possible for one of the involved processes to detect that a process crashed, and make sure that appropriate cleanup happens.
In destroy memory block function you wrote get_shared_block(filename,0);
Shouldn't it be get_shared_block(filename,BLOCK_SIZE);
Am I wrong?
In the future, you may want to use a De-Esser on your audio file.
awesome, !!! could you please post a video about how to use semaphores in thread synchronization,,!
That's the plan. Not sure when it will come out. But, it's coming.
way better than my Chinese tutor
So this is a implementation via the System V standard/mechanisms, right? POSIX way should be thru mmap and shm_open?
Yep. I have a few videos on mmap. Probably need another video on shm_open, though.
great as always.. but instead of saying unix based you might consider the term POSIX :) ..may linux folks might argue otherwise :P
in my operative system course they call this 'message queues' and for me, it's so difficult to grasp. They ask to solve problems with threads synchronization first with semaphores and then do the same but with messsage queues.... using this like blocking access tools! AAAAH
This video can still be helpful for Windows users who use WSL.
Thanks for pointing that out. I haven't spent much time with WSL, so I'm often hesitant to make claims about what it does and doesn't support.
Will this work in android ndk ?
Can you make videos for remaking the functions of the standard C libraries. Like making setjmp,longjmp,malloc,free,etc.
Yeah, probably. I'll add that to the list. thanks.
Maybe I'm an old UN*X hand, but the promise of unified VM/filesystem is that two programs mmaping the _same_ file effectively shared memory.
In a way, I would call this "how to play with SYSV shared memory"
hmmmm, @7:33 are you comparing result to casted char * of (-1) ?
shouldnt it be: if (*result == IPC_RESULT_ERROR) ? ..
Ok , but how to implement that shared memory functions (shmget()) without using existing header files, I mean implement that header files by your own sir
What is the idea behind using a FILENAME and ftok to generate a key for the shmget? Why not hardcode a number that both reader and writer agree on?
I coded along while watching the video and I just wrote
#define SEMAPHOR_PRODUCER "seemaphor producer"
And did the same thing for semaphor consumer as well.
I'm guessing they were supposed to be pronounced "shared get", "shared ctrl" etc
Can a bool function return NULL in c? 9:08 (line 52)
no
i came from Java , No unsigned byte , No accessing Shared Memory,
Going to C++.
Thank you.
Can you plz make a video series in tool chain of gcc or one video on elf file format?
What aspects of the toolchain are you most interested in? There are a lot of pieces in there.
tools that help us to find memory faults like Coredump
@@JacobSorber Can you also make a video on Program execution sequence , what actually happens when we enter ./a.out or (run any executable file)internally in O.S
What you mean by associating an address with filename?
I mean that the block of shared memory is associated with a filename. So, if a process wants to find the block of memory, it looks it up using the filename. It's simply the mechanism for finding the block. There is no relationship between the shared memory and the contents of the file.
I can see OBJS=shared_memory.o but you don't have any shared_memory.o .. so this should give error no such file or directory
Like if I decode it : I am writing filename different .small
GCC -g -wall -c write.c shared_memory.o -o write.o
GCC:error: shared_memory.o: no such file or directory
Consider write.c as writeshmem.c
Hello Jacob - great talk !. Could you make some content about Makefiles ?
Thanks. I already have some. What aspects of makefiles would you like to see more about?
Hello Jacob, due to Makefiles I think following topics to consider:
- %.c : %.o notations and alternatives if any
- some defaults rules to write makefile as short as possible with some variables describing comiler, compile/linkage flags
- how to mix C and C++ build in makefile
- considering CMake to generate makefiles
- imvoking one makefile from another if possible
I use /tmp/file as a shared mem
It's always an option.
very difficult format for compiling as I dont know how to break down your method to compile alternatively - will check out your compile video **make: *** No rule to make target 'writer.elf', needed by 'all'. Stop.**
shmdt, Shared Memory DeTach? duh?
How to detect sin wave in a wav audio file
Meaning, how to detect a particular frequency, additive noise (sine wave added on top of other signal), or how to detect whether the wave file is a single tone (sine wave)?
@@JacobSorber i am trying detect certain pattern . The sound pattern is a sin wave whose length is 0.01 seconds . The sound wave starts with a maximum amplitude but decreases gradually giving it a click sound . When i view the wav file in audacity I can clearly see the two spikes but when i read that file in to a numpy array things get messy . Tried finding the rms valve of 0.01 seconds of audio at a time . I thought I will get a abnormally large rms valve and i get that but i get multiple large rms value in a wav file with just 1 click
@@JacobSorber the sin wave is played by speaker and my mic records that so the sin wave is with some noice lke cpu fan,ceiling fan noise or tv noise
@@ciph3r836 One option would be to compute an FFT of your audio signal, and look for a spike at the expected frequency.
@@JacobSorber sir i just started doing this just minutes earlier. Fft seems to be promising
What tfok!!??