Metaclasses in Python
HTML-код
- Опубликовано: 8 июн 2024
- Metaclasses customize the class creation process.
We go over what metaclasses are and give some notable examples, including: abstract base classes, adding function overloads, and dataclasses or code generation.
CONTEST WINNERS (CONTEST ENDED 10/9/21) :
PyCharm: Yannick Kuhn, peter duffy
CLion: BalkanSwine, Laurin Neff
OFFICIAL CONTEST RULES:
1. All entries must comply with the RUclips community guidelines ( ruclips.net/user/tcommunity_gu...) and RUclips Terms of Service (ruclips.net/user/static?gl=US&t.... Entries that violate RUclips guidelines are automatically disqualified.
2. RUclips is not a sponsor of the contest and viewers are required to release RUclips from any liability related to the contest.
3. Privacy notice: no personal data will be collected for this contest.
4. In order to enter, you must (a) be one of my subscribers, AND (b) make a top-level comment to the video including #pycharm or #clion or both. If you only include one you will only have a chance to win that corresponding license.
5. The contest is free, there is no fee required to enter.
6. Winners will be chosen randomly 1 week after the date the video went live from all users who have entered and not been disqualified.
7. Each winner will be notified via a comment reply from me that details what prize was won (e.g. "Congratulations! You have won XYZ. Please email me."). I will ask the winner to contact me by email, and I will reply through email with a random token which must be posted as another reply to the winning comment from the winning account in order to verify account ownership and prevent fraud.
8. Each winner will have 72 hours to respond AND prove account ownership or their prize is automatically forfeited and another winner will be chosen.
9. A winner can only win 1 prize per contest.
10. The prize pool for this contest is: 2 licenses for PyCharm Professional ("Free 1-Year Personal Subscription") and 2 licenses for CLion ("Free 1-Year Personal Subscription"), which are products made by JetBrains. A prize consists of 1 license, which will be delivered in the form of a redeemable code that can be redeemed at www.jetbrains.com/store/redeem/ before May 01, 2022.
11. You may not enter the contest if doing so would be a violation of any relevant federal, state, and local laws, rules, and regulations, including U.S. sanctions.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
Python metaclass docs: docs.python.org/3/reference/d...
Previous video on new: • __new__ vs __init__ in...
Another great metaprogramming video: • Python 3 Metaprogramming
Python descriptor docs: docs.python.org/3/howto/descr...
Python ABC docs: docs.python.org/3/library/abc...
SUPPORT ME ⭐
---------------------------------------------------
Patreon: / mcoding
Paypal: www.paypal.com/donate/?hosted...
Other donations: mcoding.io/donate
Top patrons and donors: Jameson, Laura M, John Martin, Dragos C, Vahnekie, Pieter G, Casey G, Sigmanificient
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding
CHAPTERS
---------------------------------------------------
0:00 Intro
1:28 What is a metaclass
3:58 Simplest possible metaclass
4:59 Load time metaclass
6:08 Abstract base classes
8:03 Function overloads
14:10 Dataclass code generation
15:01 Giveaway instructions 
Наука
![Latto - Sunday Service (feat. Megan Thee Stallion & Flo Milli) [Remix] (Official Video)](http://i.ytimg.com/vi/9gL0F1xIt2I/mqdefault.jpg)
i will literally never have a practical use case for this but it still makes me happy it exists
edit: ok within a year I have used it several times I retract my previous statement
The answer is basically proper encapsulation. You can shift the complexity to the caller (outside the class) or you can keep it in the class where its manageable while keeping the caller (outside the class) simple.
Creativity is everything and this is just another tool to support that.
I actually used Metaclass to modify the __iter__ method of my Enum class to skip some values.
Basically like this:
from enum import Enum, EnumMeta, auto
class MyEnumMeta(EnumMeta):
def ignore(Cls, e):
return False
def __iter__(Cls):
return (e for e in super().__iter__() if not Cls.ignore(e))
class MyState(Enum, metaclass=MyEnumMeta):
Empty = auto()
A = auto()
B = auto()
C = auto()
@classmethod
def ignore(Cls, e):
return e is Cls.Empty
I used it all the time when I was a Java developer when I needed similar blocks of code to handle single item of type X or a list of items of type X. 99% of the time it was two functions with one function handling single item and the "overloaded" 2nd function handling the List of that item
dunno, i use overloads all the time in Nim and C++, its rather handy tool for segmenting code
otherwise you would need to create a single function in python that has all the implementations depending on args* kwargs** signature which is rather clunky
@@user-vt9bp2ei1w crazy
This video was so magical I'm tempted to call it a ___video___
Or a video of a __video__
5 letters are too long. use __vid__
Ah... *dunder video*
magic, like a magic ___methods___ in Python )))
@@zoulei881027 yeah and the __str__ will return “video”
7:55: _"No need to be confused"_
Me: Most confused I've ever been in an mCoding video
#pycharm
Metaclasses looks very powerfull, unfortunately most people using "powerfull" features in a big codebase usually leaves the most powerfull mess to be maintained. Great video! #pycharm
Hello! "Powerful" is spelt with 1 'l' not 2 :)
Well said
@@5h5hz Thanks. At least I was consistent.
@@pericofalcor hahaha such a programmer response, I love it!
He meant powerfull of poo poo
My mind is blown. So many possibilities once you know what Python does under the hood. I would find it super interesting to learn more about how C-bindings like numpy uses them are built and how one could add them too. #pycharm
How do you go about finding what python does under the hood, in a relevant manner?
My mind is not blown as these features have been available in other OOP languages for ages. However I still like Python for its simplicity and would dread going into typing library, generics etc. I prefer simplicity.
This is the most complicated subject of Python I’ve ever seen. Thank you for your work. #pycharm #clion
I taught Python for 2 years at the university level. I thought I knew more deep wizardry than any person should know. After seeing this, it's clear that I know nothing.
#pycharm
Same here, my friends call me when they have a question about Python because they know I love it and know it quite well, but when they call me a "Python master" I humble myself by remembering this channel exists :')
Man the information density in your videos is mindboggling. Every 20 seconds there's something I'd need to sit an hour for to fully absorb the concept and the code. Well done, thanks
"the type of big A is a type but the type of small a is big A, whereby the type of type is type"
That's exactly the feature you will probably not use for a long time, but years from now you will remember and it will save your life!
Best thing about python, it's not like other languages where many base features are hard-coded, but you can customize EVERYTHING
#pycharm
its not always a good feature
@@georgplaz yeah, too much customisation just gets everyone confused, you cant replace every descriptive function name with some __magic__ and have people (and yourself) still understand it.
Since I've -lot's of work- nothing to do right now, I counted how often you've said the word 'class' and 'metaclass' in this video:
*·* class ⟶ *62 times*
*·* metaclass ⟶ *33 times*
Oh my god, I watched the video on 2x speed and at 1:37, my brain just started to hurt.
I really like your content by the way. There are so many useful python features I've hever heard of before. 👍
Do occurrences of metaclass also count towards class?
Don't worry, if you watch plenty of videos at 2x, you'll get used to it
@@mCoding
No, I counted them seperately, I counted things like "class body" though. Still 70 times seems like a very high number.
EDIT: I counted again and updated the numbers, though not much changed.
@@foxcirc was that "edit" a form of youtube commit message? 😁
All programming languages seem to converge towards each other. Combine this with Cython and static typing, and you just made a slower C++, but with more libraries. #pycharm
As the world progresses more we all figure out the best way to do things has been C++ all along.
@@chrisvinciguerra4128 c++ is not a good choice for embedded systems c ftw
@@duality4y May aswell just hammer the opcodes into RAM yourself
@@chrisvinciguerra4128 Yeah that beautiful manual memory management when you're designing a high-level user-facing application was always the right way all along.
Gratz! You (Yannick) get a PyCharm license! Contact me to claim your prize! mcoding.io/contact
I feel like your videos have given me a really deep understanding and appreciation of all the internal magic that goes on behind the scenes on Python. Thanks for the content!
one of my favorite things about python is how modular the language itself is. You can just overload anything you want, and having the syntax be so malleable is so fun!!
This is actually that rare gem content found under the depths of the internet. Been trying to wrap around meta-classes for a week now, with so little luck. The docs/content revolving around such concepts are so scarce, But you made it so clear as day. Thank you :) -don't need pycharm cause VSC is nicer lol-
this is heresies, black wizardry and demonology combined
Every use of this forbidden knowledge needs to be marked with
# Here be dragons!
This is a terrible idea, super slow, and very very dangerous.
I absolutely love it.
You had me in the first half
I've not used python in years, and primarily use TypeScript myself currently.
I had an alright time using Python when I did, but didn't think too much of the language, but your videos including this one really make me appreciate python more!
This whole system shown in the video is very elegant imo, and very powerful.
So you were trying to overload a function with the wrong syntax and didn't work, the you used the correct way to do it and worked.
Mind bending.
Awesome video! It's a bit more advanced than usual, but it's good to hear something like that once in a while!
James, every video I watch you blow my mind with what is possible in Python, I don’t know yet what I’ll do with all this new knowledge but dang is it fascinating! Thanks for the quality content #pycharm
Very clear explanation. After over 10 years of python programming I still learn new tricks from all of your videos. Thanks!
Great to hear!
It's interesting to see overloading implemented in Python, I saw a stackoverflow comment suggesting to use multiple dispatch instead and now I might see why considering the slowdowns. Great video as always :) #pycharm
For once Python is making me appreciate the simplicity of C++ and not the other way around #clion
don't worry, we'll get reflections in c++ soon(tm)
@@kurtmayer2041 i need that
This is so much simpler than anything C++ reflection ever will be. At least this is consistent lol
simplicity... of C++? in what world is it true lol
You must've just started learning C++.
7:33 This is how the enum module is implemented -- you create enum classes by subclassing base classes that have a special metaclass attached.
Fabulous! Just what I needed to understand this shiny yet murky realm of metaclasses.
I've definitely learned more from your channel than any of the other python channels, and courses I've taken combined. Please keep making such great content. #clion
Python gets more intricate and beautiful the more I use it. (and learn from your videos)
Your videos are very useful. Short and to the point and ofter cover subjects you may not encounter when using the language even professionally. Thank you!
Every time I think I am quite knowledgeable in python a new video is posted and I realize how few features of python I actually use
I'm really happy that you are making speed comparisons. This is something I often do with python, to figure out what works best, but I don't always understand why. #clion
David Beazley, as far as I remember, advocates the use of class decorators (where possible) instead of metaclasses. The decorators are easier to handle, easily composable and easier to understand. But sometimes... yes, one has to take the bull by the horns and dive into metaclasses (not for the faint of heart). There's no wizardry at play here but one has to go through a lot of theory and practical examples to really start to see (appreciate?) the structure and intent of the creators of the language. Thank you for the video and spreading the knowledge.
Great video, i had no idea these aspects where so customizable!
This is really fascinating stuff James! I thought I was pretty experienced with Python, but you are helping me take my understanding to another level. Many thanks for sharing your wisdom.
Hi Wizard, thank you very much for your kind words! I'm glad you enjoyed and glad to have your viewership!
Damn, I've got to appreciate how flexible Python is. This is face meltingly awesome.
Thanks for the kind words!
This was actually quite interesting. I didn't know function overloading was even possible in python! Very cool.
#python
This is extremely interesting. I tried to learn about metaclasses about two years ago, but I didn't understand it - until now. Excellent explanation.
Great to hear!
Amazing. I’ll definitely be needing to watch this vid again a few times to understand it, but it’s truly mind blowing what you can do with metaclasses. #pycharm
I think the good part is that there are so many concepts you touch that I didn't even know in Python. I had to stop several times to look the documentation and learned a lot! Thanks! #pycharm #clion
Yet another banger, I didn't even know this existed to be honest! Really interesting watch.
As for the giveaway I 'm a student so I get their stuff for free, but it's actually incredible you got that opportunity from Jetbrains! Congrats on that.
Now, that we've learnt about this from this fantastic video, probably it's time for the designers of the language to learn, from the very same video, that the most basic feature of am OO language should not be paid in blood.
Haha you're welcome to suggest but I think the designers are mostly against checking types at runtime. Also they will tell you python is a multiparadigm language, not just OO!
Being a beginner with Python I'm quite happy I managed to follow what was going on here. Even though the approach discussed here has various negatives the content as a whole was quite informative. Thank you. #pycharm
Loved the part where you showed what's behind the curtain of some standard libraries. You should make a whole series of this. 👏🏻
#pycharm
Thank you for your dedicated work, James Murphy. This video proves it once again. #pycharm
My pleasure!
Great video on how to be taken out back by your team and shot.
Very interesting information there. I would think, though, that most people have gotten around not having overloads built-into standard Python for so long, that normal Python paradigms suffice for all their coding. Overloading is nice to have, but I think it's more of syntactic sugar to other possible syntaxes. Somehow I can't help but think about the Zen of "Explicit is better than Implicit" might apply here as well, especially when usage is coupled with non-keywords args.
Thank you for the wonderful videos, and I must admit this is my first comment on your channel, but I follow your videos constantly. #pycharm
This was pretty mindblowing...
I have no idea what's happening but it works and it's cool!
James Powell had a great talk at some PyData which included metaclasses though your introduction is much more practical. Nice one! #pycharm
I'd love to see a short followup to this concept using the new Python Match syntax, I don't imagine it drastically changing performance, but I could imagine it allowing for a much cleaner code base.
Never thought we could define two functions with same name but receive differents parameters, like I was used in C++. That's very nice! #pycharm
My favourite programming channel
This was amazing! I was just wondering if there was a way to do this in python. I didn’t even know metaclasses existed before now. Thank you for the very detailed explanation! Keep it up! #pycharm
I always wondered how these abc classes worked specifically. Thanks!
#pycharm
Glad it was helpful!
I probably will never use Metaclasses personally, but it's great to know that they're there and what they do!
#pycharm #clion
Excellent video. It took me some time to understand it, but it is pretty genius what you do. I saved this video because of it being so valuable...
I see that your example implementation that allows function overloading, indeed causes a big amount of overhead. Maybe its an idea to raise a PEP for getting a real solution that allows us to overload functions in future version of python. Because honestly, I think it is a very worthwile addition to python.
I feel like this channel both teaches you advance python and the foundations you didn't know you don't know. Thanks! #pycharn
Wow! definitely high potential in overload
At least a clear and also deep explanation of metaclass mechanism and some uses. Not sure I will ever use it since changing much class behaviour makes code difficult to maintain. Thanks man. #pycharm which is my favorite IDE for python.
Thats an interesting deep dive into python!
I learn something new with every video you post, thanks.
#pycharm
#python
Your videos are so impressive, very good explanation and really clear examples - for real you are a cut above the rest in this space and I'm glad your channel is growing so quickly!
#pycharm
Thanks sir, your content is always enlightening. Frankly, it sometimes stupefies me how much there is to learn about a language that many of my friends, and my computer science teacher, dismiss as simplistic... great content as always!
#pycharm
Glad to hear that! It never ends, I learn many new things every day! Sometimes in the process of recording :)
Seriously, if this language had tricks up its sleeves to do a lot of this at compile-time (like Zig), it would've been the most powerful language across the domains.
I mean seriously, so easy to pick up and get started, yet so accessible to dive deep and get lost into the wild wild technicalities.
@@VivekYadav-ds8oz You can look into JIT compiling solutions for Python like numba (library) or PyPI (interpreter) for example, it lets the code be compiled at runtime (which slows down the first function call, but generally makes the next ones way faster, catching up to the realm of fully compiled languages)
@@matthieurochette Yeah numba is awesome!! Relatively simple to use as well
This topic was something I've been wanting to understand better for a while! #pycharm
Wonderful video full of examples and use cases to explain this advanced function!
Looking forward to your next one!
#pycharm
I have always wondered what metaclasses are used for. Thanks a lot! #pycharm
WoW !!! Metaclass are overwhelming. But they are quite amazing. And really like your efforts in teaching these concepts to us. This is the most interesting RUclips channel out of all my subscribers #pycharm
This stuff is super cool. That function overloading still looks like black magic after you explained it.
#pycharm
I have used metaclasses before but I never realized you could do this much with them. Very eye opening #clion
learning these bits of lower-level mechanisms in Python lets me understand how Python magic works, especially for those built-in decorators like property. #pycharm #clion.
absolute madness, the great old ones will be impressed #pycharm
Damn. The type of video to really smack you in the face, when you think you know programming.
Do more, James, your content is fantastic!
#pycharm
and yet again, I feel like my python programming skill has leveled up. #pycharm
Some more C/C++ content would be great.
Man, I might actually try making a cool library that can be used in my fav game, metaclasses are so cool
I gave up trying to understand metaclasses before, but with your video I could get a good grasp of it even though I had to watch some parts twice or thrice (so many concepts). Thanks!
#pycharm
Awesome under-the-hood demo implementation of Python’s missing overload OO functionality. #pycharm
I remember stumbling upon this concept once and wrote some magical shit that even I couldn't parse after the fact, but it worked like a charm so we went with it... If I could track down the code I bet I could understand it better now, thanks for the deep dive! On a side note, abstract base classes are one of my faves, love making use of them #pycharm
Well that's the first video that I watch on your channel that left me puzzled yet excited to watch it again #pycharm
I'm sure you are in good company, metaclasses are probably the most difficult thing to master in Python. Master metaclasses and you can master the whole language!
This was absolutely fascinating. I’ve been using ABC meta for ages but didnt know how it worked under the hood #pycharm
Always love the in depth knowledge of how python works. #pycharm
Been using abc for awhile, and it's super interesting to know how it operates under the hood. #clion
I always enjoy ways of bending the rules like this! #pycharm
I'm currently trying to apply the strategy pattern in python, so such a detailed explanation of metaclasses is very useful. Thank you. #pycharm
Even after watching this video, metaclasses still seem magical. There's no end to what you can do with them. #pycharm
Once again you’ve created a short yet excellent video that I’m going to spend hours replaying and pondering over. Thank you! #pycharm #clion
My secret plan for increasing audience retention :)
#pycharm #clion
You shed some light onto these Metaclasses - I always avoided those up to now. But now I got some weird ideas, though. I really like the way you get into these deep details of the language.
"a class defn is actually just syntactic sugar for calling a type constructor"
There's a tiny caveat here, like being able to pass kwargs to the type definition's bases, which can't be done with `type`. To be able to do that you'd have to use `__build_class__()` instead, so in python3 (i believe starting from py3.1) it's actually sugar for the build class top level function instead.
It's nothing major though, but I think if you're making an absolute statement it should be correct.
Consistently high quality content. Well done James! #pycharm
Wow, my mind is literally blown.
Would be cool to see a video on weakref some time in the future.
#pycharm
Really neat! Just a question: why not use a dict of signature hashes resolve the best_match? I feel like it would be faster than iterating over the list of signatures.
metaclass generated singleton is very good to use thanks very much!!
#pycharm
Thank you for all the amazing videos, you rock! #python #pycharm
been enjoying your videos. i think youtube has a plethora of beginner level content but not enough intermediate to advanced. you're hitting a sweet spot with these videos. #pycharm
Even if I don't make use of metaclasses anytime soon, the deep dive into the how classes work in the background has been very interesting. #pycharm
Yeah, metaclasses are super powerful but also something you should think hard about as to whether they're the right tool for the job. For the purpose of function overloads, I'd probably even say they're not the correct choice, and would just be better just as a property-like decorator where you do @overload for the first definition and then @f.overload for the subsequent definitions. And of course there's the question of whether having overloads in python code is even good
In every large code base that I've worked in, a few people have tried out obscure-but-extremely-cool language tricks. Most, but not all, of those cases resulted in long discussions with the people who have to maintain the code, test the code, see the code and want to know about it just out of curiosity, or worst of all, were tracking down a difficult bug and wondered if this obscure code they don't understand could somehow be the culprit. If you take into account the cost of those discussions, it's very rare for the trick to provide a clear net benefit. It's happened, but it's very rare. In a large code base, the highest priority is to make the code readable by others. Please remember that before you use a feature that most others do not know.
I totally agree, I'm often that person tracking down why a framework needed to change everything to save a few characters. But I do love exploring the limits of what is possible.
I was just wondering how to transform a type hinted (data)class into a PySyde QObject with automatic properties signals and slots; this video made my life easier! #pycharm
Just some feedback... the constant cuts saves viewing time but also impedes comprehension. It takes you time to talk about these ideas and likewise it takes time to comprehend them.
Sums up MetaClasses really well. Impressive. #pycharm
@mCoding would a set be a better structure than a list to keep "seen" parameters? In a list you'd run the risk to overload more than once with the same arguments.
Unless you want to combine overload and duck typing? 🤯
The metaclass overload reminded me of monads from functional programming - essentially 'something' that gets treated differently based on its properties. Very neat!
Oh, and... #pycharm :)