C++ in Python the Easy Way!
HTML-код
- Опубликовано: 2 авг 2024
- You voted, so here it is. I tried to condense everything you might need to know to get going with writing Python interfaces to C++ code. The breakdown is
0:00 Intro
0:39 Absolute minimum example
2:28 Exploration of generated module
3:25 Classes
4:49 STL containers, Python types
6:20 Numpy support
7:24 Properties
8:00 The GIL
#python #cplusplus
All the code for this can be found here: gist.github.com/safijari/f7ae...
The official CMake example: github.com/pybind/cmake_example
Definitely check out my sponsor at www.hbtapp.com
Join the subreddit at reddit.com/r/JackOfSome
![Gunna - one of wun [Official Video]](http://i.ytimg.com/vi/w_JtBYqMgk8/mqdefault.jpg)
1:56 Me noticing the missing semicolon before he compiles while understanding very little about everything else:
"Sometimes my genius is almost frightening"
I'm genuinely impressed since the video is in turbo mode 😅
I usually watch videos in x2 speed, however here I had to put it in 0.5x speed 🤣 I'm discovering both C++ and pybind11 so although it is hard to follow everything yet your explanation is pretty cool as it sums up most of the functionalities available in a short time, I now understand better where I need to start ! thank you
Found your channel a few days ago, and absolutely loving your intuitive example-based explanations. It's also great that when you start covering a topic you stick to the essentials - it's short, to the point, and incredibly helpful when getting started!
Subscribed! I love that you zoom in as necessary, which makes it perfect to watch on my smartphone... that you provide the source code much as possible... that you speak very clearly... that you actually communicate great with your many fans... and let us know what other alternatives there are for those of us not familiar with other modules.
❤️
Stumbling on this video while researching this topic, I was initially disappointed to see, that it was only 10 minutes long but holy crap this is jam packed with info. Thanks a bunch!
man your way of teaching is amazing but interesting and entertaining as well, instant sub
This was really gangsta: thanks for the concise cuts. Never had to hear you take a breath: very well cut 👏🏻🙏🏼
real gangsta right here g
I usually steer clear of youtube videos for technical things. But dang!! this has been super useful.
wow, this is amazing, love your work!
Great video, learned a lot!
Straight to the point and super clear!
is there any way to make intellisense to analyse the python module imported?
how can I use pybind11 as python-package? If I install it via conda/pip, what is then the path in find_package command in CMake?
thanks a lot, but I met an error: ImportError: DLL load failed while importing module_name. I don't know how to fix it.
Amazing video!
Great video. How do you pass by reference from C++ to an embedded python interpreter?
this is the usefull video i found, thanks!
Thx, all I needed!
Keep making more of this, how to pass c++/py types to each other without copying.
I got a question. How do I make regular multidimensional array (ex. static int array[100][100][10] ) to return as a numpy array? What if I just have a pointer to this array (which I created using typedef)? I like to avoid any copying, if possible. Hope I get a reply soon. Thanks in advance!
I have to write a python program to run this video at half speed. Either that or pause it ever 5 seconds. Lots of information packed into a few minutes.
Very useful, although the pace was quite hard to keep up with, for me.
Thank you, but unfortunately I get No module named build.module_name error
Amazing ❤
does someone knows how to switch from compiling from python to compiling with cpp please ?
Hi, thank you for the nice and recent tutorials! I was having trouble running an existing git that binds c++ module. When I try same thing as 2:33, it gives me ImportError: dynamic module does not define module export function (PyInit_shot
I had this issue. The name of the module in main.cpp MUST be the same as the target name in CMakeLists.txt. When you have "PYBIND11_MODULE(name1, m)" in main.cpp and "pybind11_add_module(name2 main.cpp)" in CMakeLists.txt, name1 must be the same as name2. name2 will be the name of the so file generated, which python will look for the module name. So you can do "import name2" and python will identify name2 as the name of a python module. But name1 will be the module that is actually defined inside the so file. But if you try "import name1", python won't be able to find the name1 module.
Literally subscribed to your channel within like 6 seconds of watching your first video 😂❤
Whats your font family of IDE?
I just want to comment that i loved this video! It's the first time a video has helped me more than any written tutorials/docs I could find, I struggled for a day before I came across this!
As a side note - I would love if you had posted the whole file-structure and a note on the dependencies, I struggled a bit before figuring out that pybind11.h needs to include Python.h and related files. I feel like that is a point practically all tutorials on pybind11 are missing, hence why I haven't gotten any of them to work for me :P
hii, were you able to solve this? i'm facing the same problems but don't know how to deal with them :((
@@mariam.magued24 I got it working shortly after posting this, and first of all I want to say: Pybind11 is a GODSEND. Once you figure it out it just works amazingly. It is definitely worth the couple of days you might have to spend before you get everything to work smoothly.
To address the issue: You have to figure out how to download/install the python developer files for your system. For Linux it's something like "apt-get install python-dev". For macOS (darwin) it's something like "brew install python-dev". Then you have to make sure that pybind11 finds the correct python-files on your PATH. Uninstalling old versions of python that you have lying around may help.
@vegardjerven I am currently struggling too, but not only a little bit but a lot. How did you solve your problems and how did you find that out, that Python.h has to be included in pybind11.h? My main problem is actually that I get this error: Python libraries not found. Can you maybe help me with that and tell me what you did to get everything going? I know you probably have better things to do than help a random kid on the internet, but I would really appreciate any help. Maybe you can recommend a good tutorial on YT or google which helped you solve your problems. Thanks in Advance!
@@alexkonopatski429 Hey! I'm happy to help as much as I can :) the first issue is that I haven't found any tutorial on that part of getting pybind to work, it seems like the kind of thing everybody just "expects you to know"... The second thing I want to say is: Don't give up! Pybind11 is worth every second you put into getting it working the first time. After that it's smooth sailing.
With that said: You first have to download and install "python with developer files". The easiest thing to do, might be to uninstall any and all python installations on your system (be careful if you have linux). Then google "install python with headers" or "install python developer" or something like that, and install python with the headers (Sorry, I can't help you more without knowing what system you're on etc.). Once you have those, you can point pybind11 to your python executable with "set(PYTHON_EXECUTABLE )" in your CMakeLists.txt file.
You can also try adding the python include directory to your projects include path using target_include_directories (cmake.org/cmake/help/latest/command/target_include_directories.html#command:target_include_directories) after the call to pybind11_add_module.
If you get that right, pybind11 should be able to find the headers it needs.
Best of luck!
Excellent!
YOU'RE excellent!
very helpful
What is the name of your colorscheme?
I have a silly question. If cmake uses the clang compiler instead, would this be a problem?
I have seen some people doing this online. However, if Python is building modules it would use the gcc or msvc compiler by default, i.e. the compiler used to make the python distribution.
Hello! I am wondering, what theme were you using in this video? Thanks :D
I don't know how to write a Makefile for this :((
Hi, nice contents, really enjoy watching. What is the font in this video?
Useful tutorial, though I when I follow the absolute minimal example, I get the error "ImportError: dynamic module does not define module export function (PyInit_pybind_test)"
what previous knowledge do i need to know to follow this? Any tips? I need to learn about pybind11 but i coulndt really follow your video :/
Fundamentals of C++, python, cmake, git, and basic Linux.
What is the name of the C++ file you create where you write all the commands "#include
nvm got it to work. Thanks!
Hello man! U have great videos. Love it.
Can u please tell me how did you changed the tab fonts in your browser ( u maybe using chrome, i think 😅 )
I'm using the Brave browser, no customizations
Would you please let us know what is this text editor and how you configured it.
It looks excellent.
Emacs. The configuration is forked from github.com/safijari/stupid-spacemacs but I keep this one private because of work related configs.
@@JackofSome this is cool. Thank you 🤓
Thanks for a very quick easy to consume starting point for pybind11. Have you any experience with cppyy? If so, how does it compare to pybind11?
I just found out about cppyy a few days ago. Haven't looked too much into it.
This is the video that I wish had existed when I started using pybind11.
That's basically what I wanted to make, the reference I wish I had.
Great video once again. I like your speed of presentation - I usually watch tutorial or showcase videos with 1.5x speed (sometimes even 2.0x, ugh) because I detest long, drawn out sentences and vacuous rhetoric. I actually had to check whether my playback speed was still up, hahaha. Nice job!
"not wasting the viewer's time" is my mission statement
@@JackofSome Well, you earned yourself a sub.
what is wrong? ImportError: No module named module_name. i want to know the answer. thank you.
What font is this?
Mononoki
Hey, great video, it is a nice intro to pybind11, i have a 2 questions, a pybind11_module is a seperate c++ file right, say i am embedding along with extending python, how do I let the c++ module know of the resources in the main executable?
Second question, vsc just bugs out while i am trying to call the functions in the module, do you have any idea how to fix the intellesence ?
The C++ resources get garbage collected like any other Python object unless you make special changes to their lifetime management.
No idea how to fix the intellisense, sorry :(
Unrelate question: I'm looking for a way to compile python scripts into executables. It's not py2exe and it's not pyinstaller. I came across the site a few weeks ago, but i cannot remember the name of it, and i can't find it in my history. A google search doesn't help. I think i may have heard about it on this channel, which is why i'm asking here.
Unlikely to be here. I've only ever deployed python programs as packages (source or wheel) and with scripts as entey points. Sorry
Auto-py-to-exe
@@dariusravn9982 Looks cool, but it wasn't it. Your hint did lead me to the correct one: Nuitka. Thanks! Now i don't know if that's any good, but i wanted to try it. And then i lost the site :D
Anyone else have trouble with PB11 not generating the Python module? Win11 Visual Studio 2022
Nice content, thank you! I suggest slowing would be helpful as the ends of your sentences and visuals were often cut off too quickly.
Hey nice video.
Pybind is really simpler than other methods i used before.
i shared with some friends thst might find it useful.
if I may, I would give just a little small feedback on the video itself.
i really like the fast paced style but would it be possible to let a second of code in the screen so it makes easier to pause and read? sometimes after the last char of the code is completed screens change and makes watching in mobile is little harder
great video and nice explanation!!!!
Thanks for the suggestion. I was a little worried that it may be too fast. I'll be more cognizant of that in future videos.
@@JackofSome actually I love it's fast.
many people are just to slow and kept in the bare basics to kick out like 10 videos of just too basic stuff
just need a little second there just to click the pause button :)
The code is linked in the description too
@@JackofSome I really liked the speed in this video, it was perfect. I do agree with OP that simply leaving the code on screen for an extra second would be very helpful though.
Awesome example. It would be great if u could do a course for pybind11 from a to z ;- )
This video covers most of what you'd use in normal use case. I'll make another they looks at stone special cases.
Help when I run make it says: make: *** No targets specified and no makefile found. Stop.
What do I do?
You have to generate the makefile first using cmake. Check if the cmake output has errors
@@JackofSome it did have errors, which I fixed. But then the make couldn't be completed because python was giving quite a few errors, the solution to which I haven't found yet. Although I'm on Windows and I'm having a lot of trouble with this. Nearly every major module for integrating C++ and Python is primarily for Linux and I unfortunately can't install Linux as of yet. Anyway, thank you for your time! I'll refer to your video again when I'm actually on Linux, which will hopefully be in a month or two!
You can also look into using WSL.
Man please, could you tell me the name of the font you're using in that editor.
Mononoki
@@JackofSome Thank you so much and great video, very helpful actually :D
I thought the video was sped up but no it is normal. Is that you work like numba in fast mode
Great content, but too much, too fast.
I really liked this video. You immediately answered the two questions I had after messing around for a couple of hours trying to wrap a quite complicated algorithm today.
I have one suggestion for improvement. While you talked a bit fast for me to follow I could slow down the video. However, some of the code after you finish writing and some prints when running are difficult to see. I would like to have at least a few frames with the final result. For example I don't think the final line with gil_scoped_acquire was completely there...
Liked it a lot though, I will check out more of your content!
Glad it was helpful. Yeah I overcorrected in this video. Definitely would have been better as a 15 minute video
@@JackofSome your tips worked perfectly! Now we have a nice Python wrapper for our most used image processing tool! Thanks!
Hi just FYI the link to your code is broken
I was delighted with your vim the entire video! Could you share yours config?
I ... I am so so sorry.
That's not vim that's emacs 😅
@@JackofSome No problem. It can be from emacs too! XD
It's not fully public, but a version of it is here
github.com/safijari/stupid-spacemacs
It's basically my own take on a spacemacs/doom emacs style kit.
What is your OS ?
Manjaro
@@JackofSome Manjaro is ♥️
@@JackofSome thank's
It gives me ' File "../test.py", line 4, in
from build.module_name import *
ModuleNotFoundError: No module named 'build.module_name'
Ok i fixed it by deleting the module_name.cpython... that cmake was making, and replacing with mine after running '$ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) ../main.cpp -o main$(python3-config --extension-suffix)' from pybind11.readthedocs.io/en/stable/basics.html
mee to. how fo you fix it?
d
@@user-kt9nl7sc2l run "g++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) ../main.cpp -o main$(python3-config --extension-suffix)
And replace that file that this command produces with what cmake produced(module_name.cpython...)
I don't know what cmake is. I researched a little and have two Qs:
1) should I pip install cmake? Is there a better way?
2) Is our cmake file .cmake? Is that a file I create?
Pip or use the package manager of your OS.
Cmake files are traditionally named CMakeLists.txt
On Linux it's a piece of cake, on Windows it's impossible!
Having to redefine what functions in a class to expose seems very unnecessary when the original c++ code will already have defined what is public and private.
I find myself treating what's public/private differently than what's kosher for a python interface, since more considerations go into that (e.g. object lifetime concerns), so I personally like that it's this way. I think I did see a macro once that automated some parts of it.
Often what you want to be public and private in code may be different to what you want to expose. For example using helper classes/structs and such in your code.
Guess i gotta learn C++ now, C Py and JS aren't quite enough haha
It's a good investment of time IMO
This must be the only instruction video I don't need to speed up :-)
2:33 "dire" command? I've always said "der" command. I feel so old now.
Nice. I always assumed dire since it's from directory.
@@JackofSome that makes sense. I think the people who say "dire" pronounce directory as "dire-rec-tory" but the people who say "der" pronounce directory as "der-rec-tory".
@@youhackforme I don't think that's got anything to do with it and people pronounce it as they see it.
c is better than c++, i have choosed next lang, thanx for showing coding
dafuq
This is a genuine question. Considering standard Python is so much slower than C / C++ and newer compiled languages like D and Nim are just about as easy as Python with the added benefit of being compiled and pretty much as fast as C / C++, why is Python so popular and used so much?
Legacy. If I were to move my code base to another language it would take a team twice the size of my own (my team is really small) at least a year to make the transition.
@@JackofSome That makes sense. I've heard similar reasons for using Java outside of app development as well. Thank you for answering.
@@JackofSome also availability of libraries. No one wants to write things over and over again and python simply has many many libraries while having "good enough" performance. If a developer cares about speed, c++ or rust may even be better!
@@JackofSome True, but I personally don't feel like "legacy code" is the top reason why python is pervasive. I think people really enjoy writing in python in a way that, imo, no other language has captured. That user experience is what caused the scientific computing community to move their work over to python during the 2000-2010's, which is what opened up python to being the most important language for ML, and why there's so many utilities surrounding that field for CV/NLP/etc. (Also, I think the "python is slow" mantra is overblown, but that's another discussion, lol.) btw, I love your video! This was very interesting and informative, and I know that I'd love some follow up videos if you're interested!
im not seeing why this is easier than just including the python.h in your C++ script and wrapping your function in python types.
CFFI is the easiest way in my opinion.
Im currently learning C to make my own programming language, I'm calling it "Vide", can you tell the basics of a programming language, if you know what the basics are?
This need a written tutorial - let us have a pdf.please
The docs are actually really good
Did he say Python 🐍 11? 😅😂
Like Cython?
Never been able to get into Cython. I know some folks that swear by it though
@@JackofSome
That's OK.
I understand.
I respect Cython, because I do NOT have to learn entire C or C++ to use in Python.
That's perfectly fair. Like I said in the video if my aim is to just speed up a bunch of code I usually turn to Numba. Pybind11 is great for those cases where I have to use C++, for any number of reasons.
@@JackofSome It seems that pybind11 is a better choice than cython for those people who love c/c++.
@@JackofSome
So true and I am very interested in using C or C++ extension for making rts, etc video games in Python.
You've been doing too much python, forgetting the virtues of the ; :)
If ; is so great why isn't there ;2 ?
Why pronounce dir as "dire" and not "dear" 😭
Oh dear, what a dire situation.
did it ever cross your mind that you're going too fast?
Your video is interesting and clear, but I downvoted it because it's way too fast.
It's even kind of hard to pause the video at the right moment to read the code.
I know I can slow the playback speed, but that's just a workaround
That's fair. I overcorrected it seems.
Code is linked in description if that helps. Thanks for commenting and I'll be more cognizant in the future.
@@JackofSome No, no! I like the speed you talk. It's not tok slow, not too fast, it's perfect! Please don't change anything.
nah, I've seen easier ways haha
Please do share
Maybe you could slow down your presentation for us mere mortals
You dont even go over the installation and setup.
Installation is cloning the repo, setup is making the cmakelists file (integration to setup.py and deployment specifics will be another video). Both are covered here.
@@JackofSome one more question. With the example you have in the video. What would be the cmake file. For example, what would CMakeLists.txt look like for you? Thank you.
This is not a tutorial in any form. It would easier to find a piece of code similar to that and read it than listening to this guy pre-record code and debugging process while flying through it. Please do not share :)
40000 people disagree with you.
I guess your counting is as good as the quality of your video. You don’t even have 40000 views. You only have 1.1k likes.
Don’t argue with me. I am telling you, the quality of your tutorial sucks. You wanna know how you should teach a programming piece? Watch one video from Corey Schafer's Python videos. You wanna watch how to produce a nice instructional (non-coding) video, watch ChrisFix. Then come back and redo this please.
Thanks