I used Rust pyo3 to expose some validation tool logic our team wrote in Rust (well, me) to our Python scripts. A useful tip is you can specify a stub pyi file at the top level (alongside your toml files) to provide better type hint and editor support when you import them. Maturin picks these up when it builds the package as long as it's called [module name].pyi
Side note: In a way, it's actually Poetry which is non-compliant here. The standard format for pyproject.toml nowadays is defined by PEP 621, which seems to be used by pyo3, while Poetry still clings to their own legacy format.
That's right! It always bothered me with poetry. According to Maturin project page they comply with PEP 621. But TBF Arjan actually has incorrectly specified dependencies, they should be an array of strings. Maybe Maturin doesn't care and still reads it correctly but that's why Python VS Code extension is showing an error
Great video! A follow-up discussing/showing real life problems that can solved with combing Python and Rust together. Boosting performance, increasing safety etc. Personally I'm currently working a lot with async, websockets and JSON messages in python, any significant and potential performance rewards would be great to see.
ruclips.net/video/D88UAjyFpAY/видео.html it doesn't seem to need to be that much involved as you can experience the benefits already for seemingly simple uses cases like dict insertion.
Really great video and easy to follow guide! When i used pyo3 i decided to make a proc macro the help make wrappers for struct impls so you could basically just add it for python support with little to no hassle. Can definitely say that there are ALOT of limitations on pyo3 that aren't directly visible, maybe a video on these things would be awesome and help some newer people to pyo3!
Hey this is awesome. I actually picked up Rust recently and it’s been going great. Great timing to see this a month later when I’ve gotten a little more experienced. Great video!
Is there some direct benefit of defining your python classes as Rust structs? I can understand that doing number crunching would be useful but not sure about the defining of objects (only). Would be interesting to hear your thoughts on what gives the most “bang for the buck” when combining the languages like this. Thanks for interesting videos.
Looks promising. But how about the interaction between Python and Rust regarding memory management? 🤔 Maybe show an example where you parse a numpy array and perform some transformation on it.
Ive been slowly incorporating rust through polars which uses pyo3. Im pretty convinced at this point replacing pandas for polars is just better. The api handles the same, its faster and infers column types (this is huge). The fact that these apis just copy sql operations and facilitate working on dataframes make the switch to polars all the more obvious.
Hey Arjan, I'm curious if you see yourself writing rust as a day-to-day? what areas would you apply it to? love the rise of rust tutorials on your channel :)
@@LieberLois I just used Rust for an embedded project using an E-Ink display on an ESP32, but I've also used Bevy to make games, Leptos for websites, and Clap to create command-line tools.
I'm just using the standard `difflib.SequenceMatcher` and for sentences and snippets of code it was just fast enough. But atm I'm trying to throw some 1000s lines of JSON against it and want to display it in some Qt text fields. It's already MUCH faster in Python 3.11 and 12 but might still be a couple seconds! I can imagine that this would be a breeze to do with a little Rust plugin!
Nice video thanks, I used maturin but didn't know about rustimport. Python and Rust are my favorites language, I also think that they fit well together.
@ArjanCodes does the original module written in Rust can still be used as a rust library as well or now after importing py03 it can only be used by Python? Also does it give the same performance benefit of rust when we ran it as a module from Python?
Hmm, I been wanting to make a bulk texture converter and possibly downscaler settings too and this might be useful to expose rusts image and image_dds crates to python to do the conversion in rust itself and all the front end code in PyQt. Maybe create the entire backend code in rust and just front end gui be python.
Love it .. going back to c after learning python from you.. today i learned how to cheat c compiler. I was coding for luhn check for three days. Today i made a dumb change in the code by suppying an argument which i assigned to something but not used. The compiler wasnt happy and error parameter not in use. So what i did was i just make a variable int error=0; and then error += s; The error was gone programe compiked 🎉🎉🎉🎉🎉. Tomorroelw i will work on how to get rid of the error variable 😂😂😂😂3
Wow, great tutorial, I always wanted to rewrite the core routines in rust for my project "pyTermTk" (a TUI Lib) What about cross compilation? Does Maturing take care of it? One of the things that prevented me to use rust or c/c++ is that I don't know if I would be able to publish a package compatible with different architectures.
I can't speak for C/C++ but in Rust you publish your package as a "crate" which can then be freely used by other devs. It will be compiled for their architecture when they download and use it (usually with Cargo).
This adds a significant amount of complication to a python project. What I did not see is any measurement at all of how much speed up inserting less modules could achieve! How do I know if I should bother with all this, if I have no idea how much speed up it might get me?
It looks great! But as an amateur python programmer I don't think I am going to use it. I don't know anything about rust so I have to learn a new language from scratch. When I have done that, I have to understand when it is usefull to to use rust. So I stick by python, if you don't mind....🙂😇
Yes, but with certain caveats. Moving objects back and fourth into python can be "slow"-ish. The more/bigger chunks of work you can shift over the boundary into rust, the bigger the benefits :)
We want the two decade experiance of " How would I do it if i start to learn software engineering today " video, Am sure the internet will expolde if you make such video because of the experiance you have.
I agree that in general this would be the better approach, problem is mojo is missing a lot of features and is atm just a proof of concept, but not a full language.
Fancy words like memory safety, superman performance blah blah are delusional. The lucky enough people who work at netflix level software - you know, the 0.0 something percent - need to worry about memory safety and are normally not watching introduction videos on RUclips. The rest of us - mental model shift is a necessary consideration for our day to day jobs.
@@squishy-tomatoIf you write something in rust vs python, you need less resources, if you are in a cloud, could save you some money and the less power we use, the less carbon we put into the air. ;-)
I am just not sold on rust... Its likely that AI will just be able to read your C code and find menory vulnerabilities, use after free, buffer overflows... When that happens, i think rust will just fade a lot...
If you haven't already, give Rust a try! It's a lot easier (and more fun) writing code that just doesn't have those memory vulnerabilities in the first place.
![Jack Harlow - Hello Miss Johnson [Official Music Video]](http://i.ytimg.com/vi/Q86_nlRoIGw/mqdefault.jpg)
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
I used Rust pyo3 to expose some validation tool logic our team wrote in Rust (well, me) to our Python scripts. A useful tip is you can specify a stub pyi file at the top level (alongside your toml files) to provide better type hint and editor support when you import them. Maturin picks these up when it builds the package as long as it's called [module name].pyi
I was going to ask about stub files, glad to know this is supported!
Nice! Gonna push my team to write some Rust around our python projects I guess
Babe, wake up. Arjan uploaded a new tutorial
Fair enough
This is mind boggling I'm enthralled. The publishing seems so incredibly easy wow
Side note: In a way, it's actually Poetry which is non-compliant here. The standard format for pyproject.toml nowadays is defined by PEP 621, which seems to be used by pyo3, while Poetry still clings to their own legacy format.
poproject.yaml is the new generic format, so Arjan could've simply disabled Poetry plugin and look for a generic one for pyproject files.
That's right! It always bothered me with poetry. According to Maturin project page they comply with PEP 621. But TBF Arjan actually has incorrectly specified dependencies, they should be an array of strings. Maybe Maturin doesn't care and still reads it correctly but that's why Python VS Code extension is showing an error
Great video! A follow-up discussing/showing real life problems that can solved with combing Python and Rust together. Boosting performance, increasing safety etc. Personally I'm currently working a lot with async, websockets and JSON messages in python, any significant and potential performance rewards would be great to see.
ruclips.net/video/D88UAjyFpAY/видео.html it doesn't seem to need to be that much involved as you can experience the benefits already for seemingly simple uses cases like dict insertion.
Really great video and easy to follow guide! When i used pyo3 i decided to make a proc macro the help make wrappers for struct impls so you could basically just add it for python support with little to no hassle. Can definitely say that there are ALOT of limitations on pyo3 that aren't directly visible, maybe a video on these things would be awesome and help some newer people to pyo3!
Hey this is awesome. I actually picked up Rust recently and it’s been going great. Great timing to see this a month later when I’ve gotten a little more experienced. Great video!
Glad you enjoyed the video!
This is awesome! I’ve been waiting for a video like this. Thanks for the great overview 🎉
Im glad you liked the content!
Is there some direct benefit of defining your python classes as Rust structs? I can understand that doing number crunching would be useful but not sure about the defining of objects (only). Would be interesting to hear your thoughts on what gives the most “bang for the buck” when combining the languages like this.
Thanks for interesting videos.
If rust is more efficient, I might put some bigger/slow projects into rust and see if I can trim the time down. Thanks for the info!
You’re welcome!
This is GREAT! That's why I love Python. It's so easy to integrate with other stuff to overcome limitations of Python. Thank you for this video
Glad you enjoyed t!
this is very nicely explained! Question how do we add python typehints to rust functions?
Thanks so much! Very clear! Will definitely give it a try!!
Glad the video helped you!
Looks promising. But how about the interaction between Python and Rust regarding memory management? 🤔
Maybe show an example where you parse a numpy array and perform some transformation on it.
Ive been slowly incorporating rust through polars which uses pyo3. Im pretty convinced at this point replacing pandas for polars is just better. The api handles the same, its faster and infers column types (this is huge). The fact that these apis just copy sql operations and facilitate working on dataframes make the switch to polars all the more obvious.
Hey Arjan, I'm curious if you see yourself writing rust as a day-to-day? what areas would you apply it to? love the rise of rust tutorials on your channel :)
I feel the same way. With Go or similar languages it's easy to find use cases, but Rust ... not so much
@@LieberLois I just used Rust for an embedded project using an E-Ink display on an ESP32, but I've also used Bevy to make games, Leptos for websites, and Clap to create command-line tools.
Can you make one for Go and Python too?
I'm just using the standard `difflib.SequenceMatcher` and for sentences and snippets of code it was just fast enough. But atm I'm trying to throw some 1000s lines of JSON against it and want to display it in some Qt text fields. It's already MUCH faster in Python 3.11 and 12 but might still be a couple seconds! I can imagine that this would be a breeze to do with a little Rust plugin!
How do cross-platform builds work when you publish the project to pypi?
Nice video thanks, I used maturin but didn't know about rustimport. Python and Rust are my favorites language, I also think that they fit well together.
Have you looked into Julia Programming Language? What do you think about the multiple dispatch?
this rustimport tools looks perfect for simple scientific calculations that need a speedup in the middle of a simple script or notebook.
@ArjanCodes does the original module written in Rust can still be used as a rust library as well or now after importing py03 it can only be used by Python?
Also does it give the same performance benefit of rust when we ran it as a module from Python?
Amazing video as usual! :)
What do you use to have auto-completion on your terminal? :D
Wondering if you have looked at Robyn not just a fast async python web framework but with a Rust core.
A way forward for Python/Rust integration?
Yaaay! :) Rustimport is pretty interesting for cross platform support... publishing compiled binaries for many platforms is a bit hectic
Hmm, I been wanting to make a bulk texture converter and possibly downscaler settings too and this might be useful to expose rusts image and image_dds crates to python to do the conversion in rust itself and all the front end code in PyQt. Maybe create the entire backend code in rust and just front end gui be python.
Love it .. going back to c after learning python from you.. today i learned how to cheat c compiler.
I was coding for luhn check for three days. Today i made a dumb change in the code by suppying an argument which i assigned to something but not used. The compiler wasnt happy and error parameter not in use. So what i did was i just make a variable int error=0; and then error += s;
The error was gone programe compiked 🎉🎉🎉🎉🎉. Tomorroelw i will work on how to get rid of the error variable 😂😂😂😂3
Wow, great tutorial, I always wanted to rewrite the core routines in rust for my project "pyTermTk" (a TUI Lib)
What about cross compilation?
Does Maturing take care of it?
One of the things that prevented me to use rust or c/c++ is that I don't know if I would be able to publish a package compatible with different architectures.
I can't speak for C/C++ but in Rust you publish your package as a "crate" which can then be freely used by other devs. It will be compiled for their architecture when they download and use it (usually with Cargo).
what about Cython? i think is less complex and better in performance terms
what is the advantage over c++ modules
Please do more rust tutorials 🙏
Is it possible to run python code from rust as well?
Are you going to create a rust programming course?
Thank you....
Julia combines both in one language
This adds a significant amount of complication to a python project. What I did not see is any measurement at all of how much speed up inserting less modules could achieve! How do I know if I should bother with all this, if I have no idea how much speed up it might get me?
Time to have chat gpt write me rust modules and scripts to import into my projects, perfect
I will use this for sure. Do you need to think about accidentally spawning new processes?
It looks great! But as an amateur python programmer I don't think I am going to use it. I don't know anything about rust so I have to learn a new language from scratch. When I have done that, I have to understand when it is usefull to to use rust. So I stick by python, if you don't mind....🙂😇
I'd be curious to see how Cython compares to this
Whatever they do, name the result "Rython". Silly as that name may sound, it sure as hell beats programming in "Pust".
I would call it: Ruthon
a litlle more credit for Rust 🤣😂😁
does this give python the performance of rust?
Yes, but with certain caveats. Moving objects back and fourth into python can be "slow"-ish. The more/bigger chunks of work you can shift over the boundary into rust, the bigger the benefits :)
no
this would create a dependency on an extra toolkit i.e. rust
I haven’t tried rust yet. But I might give it a go. But,I haven’t tried go yet but I will wait and see. I haven’t tried c yet.
Make armorers great again
If C++ can be pronounced as Sepples
C++-ians can also be called Sepplesians
Nah, I'll wait for Mojo.
I’d rather just call Rust libraries (like polars) from Python 😂😂
the thing i hate about rust is that you can only get blockchain job with it
We want the two decade experiance of " How would I do it if i start to learn software engineering today " video, Am sure the internet will expolde if you make such video because of the experiance you have.
You should start when you’re 10
Read "maturin init" in a Bri'ish accent for extra lols.
Nice but not convinced. It you really want Python + Performance - you are better off giving Mojo language a go..
I agree that in general this would be the better approach, problem is mojo is missing a lot of features and is atm just a proof of concept, but not a full language.
Also you should stop thinking in just performance, rust offers memory safety.
Fancy words like memory safety, superman performance blah blah are delusional. The lucky enough people who work at netflix level software - you know, the 0.0 something percent - need to worry about memory safety and are normally not watching introduction videos on RUclips. The rest of us - mental model shift is a necessary consideration for our day to day jobs.
@@squishy-tomatoIf you write something in rust vs python, you need less resources, if you are in a cloud, could save you some money and the less power we use, the less carbon we put into the air. ;-)
I was thinking how you can reverse the rusty python pun. Like pythony rust or pythoned rust but it won`t make any sense
mojo is an existing superset of python which is already 50% faster than rust
Why would we use rust when we can do all of this in Python?
More like the worst of both languages
I am just not sold on rust... Its likely that AI will just be able to read your C code and find menory vulnerabilities, use after free, buffer overflows... When that happens, i think rust will just fade a lot...
If you haven't already, give Rust a try! It's a lot easier (and more fun) writing code that just doesn't have those memory vulnerabilities in the first place.
Arjan simply does not explain things clearly or coherently. People just like his voice.
It smells bad to me if it uses a non compliant pyproject.toml
Why not use another name?
Babe, wake up. Arjan uploaded a new tutorial