If you're looking for more juicy details about 3.13, including the GIL-less Python builds, check out my first look video! ruclips.net/video/e4HOCuJfbGY/видео.html Thanks for stopping by!
The JIT is what excited me the most, once it's fully implement, other language won't smirk at Python devs saying that Python is so slow compared to other scripting language like node
@@Carberra Oh nevermind, it only impacts circular references for the purposes of type annotations (I meant circular, not cyclical btw). Seems circular imports are here to stay (which would make sense)
I wasn't sure if you meant that or whether you'd come across something amazing I'd missed lmao. Yeah the lazy imports PEP (which wouldn't taken out circular imports for good) was rejected some time in 3.12's development, so yeah, they're here to stay unfortunately 😔
The annotation improvements are very welcome, been avoiding upgrading the annotation support in odin because of the strings. The annotation parser is already fairly complex.
2:31 this seems like perhaps the most important new thing on 3.13, I didnt fully understand the description but if it is what I think it is, this fixes the biggest downside to Python when compared to other "more mature" languages. I put that in quotes because the import system in python is extremely immature and the biggest issue it caused was not being able to use types anywhere we would like because of the need for imports. Hacks had to be used, like a import guard for only importing a type at "type checking" time. Which is a PITA do maintain. Hopefully this fixes that
I don't believe this change will resolve that, if you mean what I think you mean. You'll still to import types, though there'll be less of a need to import types at runtime. Still won't nullify all the benefits of that though.
Distributions of python 3.13 will come with an optional additional binary (python3.13t) where the GIL is disabled. It's a checkbox in the windows installer, but it's going to be up to package maintainers to figure out how to distribute on other platforms
Indeed! I didn't include it in the video as I'd covered it before and there hasn't been any major updates since, but it looks to be coming along very nicely. Hopefully it's all ready to go for the final release!
Because it’s basically just like an inlined continuation passing style chained function call JIT. If your interpreter is well written and your language has fundamentally slow semantics (like python) you won’t see much gain from something like this. The reason JITs are fast is due to optimization and speculative optimization especially for dynamic languages. This style of JIT does neither. The generated code still treats every variable as what it is: a big dynamic blob of memory that you can’t assume much about. So basically we’re only seeing the time to find the next byte code instruction and some function call overhead within the interpreter optimized out. I’m very surprised to learn that this style of JIT is “new” though. Maybe it wasn’t formally described until 2021 but I’ve literally written (half of) a JIT for a toy gameboy emulator that works the exact same way because it’s so obvious and easy.
@@AsgerJon I’m not personally familiar with it but from a quick glance at their website, numba is optimizing and using LLVM. It’s unclear if it’s doing any speculative optimization but even without speculative optimization, knowing the order of instructions would allow optimizations like reordering instructions, possibly vectorizing, reallocating registers to avoid register spills, removing duplicated work between bytecode instructions, and fusing instructions together in some cases. LLVM is much, much, much higher overhead in terms of code shipped and time spent compiling your code than the simple technique CPython is using. The main appeal of JITs like the one that CPython is shipping is that they’re simple and obviously correct and you’ll pretty much never spend more time running the JIT than you will gain by having done it. I think CPython is more concerned about strictly following the spec and being simple than being fast. Faster alternative compilers will still have a place for less conservative users that don’t mind more (potential) breakage or heavier python runtimes with more potential bugs for speed. But if it’s as fast as you say then they probably are able to, either speculatively or by changing the semantics of Python, optimize types into simpler ones that can be run quickly. I’m not 100% sure but I assume Int is an object on the heap that works as an arbitrary precision integer without any extra optimizations in CPython. In numba they probably give you access to either a fixed bitwidth number type implicitly or explicitly and can use that alone to get massive speed ups and even further optimizations that will just never be possible otherwise. It can change what would be multiple virtual function calls on an object on the heap doing lots of work into 1 or 2 machine instructions operating on a register.
It basically just collects small parts over time rather than the whole heap at once. I didn't include it in the video as I couldn't find any performance comparisons between the two systems.
I’m a bit worried about this. I’m writing wrappers for AutoCAD where the objects have an open-forread, open-forwrite, and closed state. As of now, GC is mostly deterministic and I rely on it’s current behavior
I wish python would stop using global variables in function definitions unless explicitly told to, like in C++. I didn’t know it did this until it caused a really annoying to fix error for me, and just seems unintuitive.
@@Carberra Yeah, I've done that. But I don't know if it helps type checkers narrow down their analysis as much as these tools (otherwise why would they exist?). Or maybe it's just people coming from TypeScript and wanting to do things a certain way
I'm plugging myself a lot in the comments today apparently, but I made a video on PyScript if you want to know more: ruclips.net/video/VD2s23JgE7I/видео.html
If you're curious, there looks to be a label on GitHub specifically for this work: github.com/python/cpython/pulls?q=is%3Apr+label%3Atopic-free-threading+ I didn't talk about it this time cos there wasn't really much to add over what I've discussed before, but it looks to be coming along rather well!
@themartdog The 3.12 GIL-less build was an unofficial prototype, a proof-of-concept kinda thing. The 3.13 one is (planned to be) a much more feature-complete thing. I'm not quite sure how far along they are at the moment.
If you're looking for more juicy details about 3.13, including the GIL-less Python builds, check out my first look video! ruclips.net/video/e4HOCuJfbGY/видео.html
Thanks for stopping by!
Will version 3.14 run under the name Pi-thon?
Doesn't look as though anyone has made any official references as yet! Surely they will at some point. We'll have to keep our eyes peeled!
It will be Py-π
@@sujalgarewal2685 nah more like π-thon
Sounds like russian pronunciation
Hah. I like it. (The pun I mean)
The JIT is what excited me the most, once it's fully implement, other language won't smirk at Python devs saying that Python is so slow compared to other scripting language like node
As someone who likes to split packages into a lot of files, the cyclical import update is amazing.
Cyclical import update?
@@Carberra Oh nevermind, it only impacts circular references for the purposes of type annotations (I meant circular, not cyclical btw). Seems circular imports are here to stay (which would make sense)
I wasn't sure if you meant that or whether you'd come across something amazing I'd missed lmao. Yeah the lazy imports PEP (which wouldn't taken out circular imports for good) was rejected some time in 3.12's development, so yeah, they're here to stay unfortunately 😔
Ios being added BEFORE Android? Now that is funny to me lmao
Finally support for IOS and Android
I wonder how android apps like Pydroid work 🤔
@@vinylSummer, same question
@@Anonymous-6598 I googled it. There's a python-for-android lib that does some smart cross-compilation magic, so there's at least that
What do they mean by that? There's going to be an interpreter app? Or a library to run python inside other apps?
@@vinylSummer I think it's because Android is just a Linux distribution on which Python can be installed.
The annotation improvements are very welcome, been avoiding upgrading the annotation support in odin because of the strings. The annotation parser is already fairly complex.
Nice one for this! Loved the work on Improved error messages.
Yeah, lot of nice quality of life updates in errors for sure!
XDG base directory support is my favourite one.
Where is this? I don't see a mention of it in the release notes.
Its wild how much work is going into type annotation. its becoming its own weird meta language. which probably isn't good
2:31 this seems like perhaps the most important new thing on 3.13, I didnt fully understand the description but if it is what I think it is, this fixes the biggest downside to Python when compared to other "more mature" languages. I put that in quotes because the import system in python is extremely immature and the biggest issue it caused was not being able to use types anywhere we would like because of the need for imports. Hacks had to be used, like a import guard for only importing a type at "type checking" time. Which is a PITA do maintain. Hopefully this fixes that
I don't believe this change will resolve that, if you mean what I think you mean. You'll still to import types, though there'll be less of a need to import types at runtime. Still won't nullify all the benefits of that though.
Distributions of python 3.13 will come with an optional additional binary (python3.13t) where the GIL is disabled. It's a checkbox in the windows installer, but it's going to be up to package maintainers to figure out how to distribute on other platforms
Indeed! I didn't include it in the video as I'd covered it before and there hasn't been any major updates since, but it looks to be coming along very nicely. Hopefully it's all ready to go for the final release!
The only feature after security patches in upcoming python versions is backwards compatibility, which seems to be missing since... always...
now python is slower even faster
🐍💨
lmao, how can the jit provide such a small improvement? I recall numba being like orders of magnitude faster.
I also would like an explanation on this.
Because it’s basically just like an inlined continuation passing style chained function call JIT. If your interpreter is well written and your language has fundamentally slow semantics (like python) you won’t see much gain from something like this.
The reason JITs are fast is due to optimization and speculative optimization especially for dynamic languages. This style of JIT does neither. The generated code still treats every variable as what it is: a big dynamic blob of memory that you can’t assume much about. So basically we’re only seeing the time to find the next byte code instruction and some function call overhead within the interpreter optimized out.
I’m very surprised to learn that this style of JIT is “new” though. Maybe it wasn’t formally described until 2021 but I’ve literally written (half of) a JIT for a toy gameboy emulator that works the exact same way because it’s so obvious and easy.
@@ilikeshiba But numba-jit provides an improvement at like orders of magnitude, why does this jit give us only 10 % or whatever it was?
@@AsgerJon I’m not personally familiar with it but from a quick glance at their website, numba is optimizing and using LLVM. It’s unclear if it’s doing any speculative optimization but even without speculative optimization, knowing the order of instructions would allow optimizations like reordering instructions, possibly vectorizing, reallocating registers to avoid register spills, removing duplicated work between bytecode instructions, and fusing instructions together in some cases. LLVM is much, much, much higher overhead in terms of code shipped and time spent compiling your code than the simple technique CPython is using.
The main appeal of JITs like the one that CPython is shipping is that they’re simple and obviously correct and you’ll pretty much never spend more time running the JIT than you will gain by having done it.
I think CPython is more concerned about strictly following the spec and being simple than being fast. Faster alternative compilers will still have a place for less conservative users that don’t mind more (potential) breakage or heavier python runtimes with more potential bugs for speed.
But if it’s as fast as you say then they probably are able to, either speculatively or by changing the semantics of Python, optimize types into simpler ones that can be run quickly. I’m not 100% sure but I assume Int is an object on the heap that works as an arbitrary precision integer without any extra optimizations in CPython. In numba they probably give you access to either a fixed bitwidth number type implicitly or explicitly and can use that alone to get massive speed ups and even further optimizations that will just never be possible otherwise. It can change what would be multiple virtual function calls on an object on the heap doing lots of work into 1 or 2 machine instructions operating on a register.
@@AsgerJon???? They explained exactly why in the comment you’re replying to.
Thanks for the useful info 🙏🏼
On android , it already compiles on termux.
I didn't understand a thing! Am I that bad of a programmer ?
He went kind of fast but if you understood literally nothing I’d recommend reading one of the O Reilly Python textbooks.
I want to know more about the new Incremental Garbage Collection
It basically just collects small parts over time rather than the whole heap at once. I didn't include it in the video as I couldn't find any performance comparisons between the two systems.
I’m a bit worried about this. I’m writing wrappers for AutoCAD where the objects have an open-forread, open-forwrite, and closed state. As of now, GC is mostly deterministic and I rely on it’s current behavior
I wish python would stop using global variables in function definitions unless explicitly told to, like in C++. I didn’t know it did this until it caused a really annoying to fix error for me, and just seems unintuitive.
I liked the 'Hell yea'!
The TypeIs/TypeGuard stuff feels like it should be a part of pattern matching
You can actually use built-in functions like dict(), int() etc. to do that! (I think anyways, I've seen it, never tried it personally.)
@@Carberra Yeah, I've done that. But I don't know if it helps type checkers narrow down their analysis as much as these tools (otherwise why would they exist?). Or maybe it's just people coming from TypeScript and wanting to do things a certain way
Solaris still exists?! 😄
Wait, we can't migrate to 3.12 because of dependencies and now you present 3.13?
I wish pyscript would become mainstream.....i don't want to learn another language for front end.
Xd
what's a pyscript?
@@ewerybodyjavascript but py
I'm plugging myself a lot in the comments today apparently, but I made a video on PyScript if you want to know more: ruclips.net/video/VD2s23JgE7I/видео.html
There is transcrypt
the future of python is tython, not multi threaded.
dont know whats removed? lol 3.14 coming
👍
wthell 2k.... may be in 3k it will good(not)
653 or 563?
It is 563, sorry. I've watched this so many times and missed that! Thanks for pointing that out.
@@Carberra No problem. Great video.
Ha Good luck ever running it properly now since the 24.04 LTS fckin broke pip.
Did it? How so?
@@Carberra pip install soandso used to work perfectly. Now its returns env and permission error for everything, on a local admin account 😑😑
@@seansingh4421curious to hear more about this, i just built a server running 24.04 and curious to hear how it went
Nah, I am waiting for πthon
2.7 4 life
😀
Ew brother
Oh are you gonna tell us how all our code will break. Get off my feed!
Get rid of the GIL !
Isn't there a GIL-less build as of 3.12?
If you're curious, there looks to be a label on GitHub specifically for this work: github.com/python/cpython/pulls?q=is%3Apr+label%3Atopic-free-threading+
I didn't talk about it this time cos there wasn't really much to add over what I've discussed before, but it looks to be coming along rather well!
@themartdog The 3.12 GIL-less build was an unofficial prototype, a proof-of-concept kinda thing. The 3.13 one is (planned to be) a much more feature-complete thing. I'm not quite sure how far along they are at the moment.
What has Gil ever done to you?
@@jonragnarsson You should rephrase what has GIL hasn't ever done to you.