QUESTIONABLE Object Creation Patterns in Python 🤔
HTML-код
- Опубликовано: 22 май 2024
- Sometimes it's good to talk about what not to do when designing software. In this video, I talk about the singleton and object pool patterns, both of which are object creation design patterns. I'll show you how they work in Python and discuss why the singleton pattern in particular is something you shouldn't use. The object pool can be considered a generalization of the singleton pattern, and it has its uses, but it has a few caveats.
💡Here's my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
A few interesting links to articles and books:
- Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides: amzn.to/3jllgyH
- Principles of Package Design: Creating Reusable Software Components by Matthias Noback: amzn.to/2NETK3l
- Clean Code: A Handbook of Agile Software Craftsmanship by Robert Martin: amzn.to/3qVZgNs
- The original Design Principles and Design Patterns article by Robert Martin: fi.ort.edu.uy/innovaportal/fi...
- About singletons in Python: stackoverflow.com/questions/6...
My gear:
📷 Sony A7C - amzn.to/3aShCtw
🎙 Rode Wireless GO - amzn.to/3pSLFVU
💡 GVM RGB LED light - amzn.to/3qRNJid
💡 GVM 100W light with lantern softbox - amzn.to/2NuU8lk
You can find the code I worked on in this episode in my GitHub repository: github.com/arjancodes/betterp...
All parts in this series:
Part 1: Cohesion and coupling - • Cohesion and Coupling:...
Part 2: Dependency inversion - • Dependency Inversion: ...
Part 3: The strategy pattern - • The Strategy Pattern: ...
Part 4: The observer pattern - • Observer Pattern Tutor...
Part 5: Unit testing and code coverage - • 100% CODE COVERAGE - T...
Part 6: Template method and bridge - • Two UNDERRATED Design ...
Part 7a: Exception handling - • Exception Handling Tip...
Part 7b: Monadic error handling - • Monadic Error Handling...
Part 8: Software architecture - • Why You Should Think A...
Part 9: SOLID principles - • Uncle Bob’s SOLID Prin...
Part 10: Object creation patterns - • QUESTIONABLE Object Cr...
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
🏆 Join my Discord server: discord.arjan.codes
👥Twitter: / arjancodes
👥LinkedIn: / arjancodes
👥Facebook: / arjancodes
🔖 Chapters:
0:00 Intro
0:28 Design pattern categories
1:14 The singleton
2:42 Singleton code example
5:38 Why singletons are an anti-pattern
6:47 The object pool
7:33 Object pool code example
11:34 Object pool with context manager
13:20 Limitations and caveats of the object pool
14:32 Gag reel
#arjancodes #softwaredesign #singleton
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
Don't know why you started uploading, how you have so much production quality or how you can explain it so well, but thank you for the amazing content.
That's really kind @Newascap, I appreciate that!
The 3 reasons why singletons are an anti-pattern that were listed:
1) If you inherit from a singleton class, you can create multiple instances of that class;
2) You never know if you're using a new instance or an existing one, which makes testing trickier;
3) They are not thread-safe;
And the 3 limitations of the much more convoluted object pool pattern:
1) You can still create Reusable instances outside of the object pool;
2) You have to make sure to reset the released instance to a fresh state before it's ready to be used again;
3) They are not thread-safe
Crossing the above lists:
1) If you use any of these patterns in a non-intended way, the pattern loses its purpose;
2) If, with an object pool, you need to make sure the instance is in a fresh state before it's ready to be used again, then you could make sure you reset the singleton to a fresh state for testing purposes as well.
3) Both patterns have issues with multithreading.
So it seems like the issues with singletons are not actually solved with the object pool.
Considering that the idea of the singleton is exactly to have a single instance of a class, and that the problems present in the singleton pattern are not actually solved with the object pool pattern, it's not clear to me how the object pool is a proper substitute for the singleton...
Would like to see some comments here. I'm curious too.
Singletons are a "solution" for languages that only have Classes + Objects.
In Python, if you want the same "solution", you just can take modules (that then can have variables).
The original patterns were written for Java, where everything you have are classes and objects. Even though, there are modules, too, modules can only provide Classes where you can instantiate from, but modules per se don't have variables.
In Python there are "global" variables (bound to the module), where other languages have prohibited them, and when you still want such a behaviour, you can achieve it via the Singleton pattern (that's why for Java like, the pattern is needed sometimes).
There are good reasons to avoid global variables, and pretty much the same reasons apply for Singletons.
But of course, there might and will be exceptions, but for usual application programming, they are very rare.
Standard example would be the application configuration (here, the testing issue is not so problematic, as usually you will have one configuration for the application in dev, one for prod mode and another one for testing [that won't change just for every test and doesn't need to be "fresh"] - and if you need to change some parts of it for testing purposes, a singleton framework might offer it, e.g. in Django, you can patch the settings while testing via decorators or context manager).
More fancy examples are if you use e.g. an object to represent physical output gates (like in a Raspberry Pi) or to directly connect to physical RAM, e.g. on a driver card. There, obviously you only can have one instance of it, and if you want to have a pythonic way to represent them (instead of writing it C-like), Singleton-Pattern is useful, there. Indeed, that's not so fancy, but most of the time, you'd more likely use a system programming language instead of Python. But there are valid reasons for Python in such scenarios, too, just as it might allow a much broader class of users to program (children, subject matter experts from other disciplines than computer science, ...)
And yes, object pool pattern share some of the problems, too.
But here, you just can't elegantly solve what you want to achieve by object pool patterns with global variables, so if you have such a need (db connections are the standard example for it, like in the video), it's better to follow the object pool pattern than to create global variables holding the pool instances and fetch them from there and manage them in the variables.
For testing or other purposes, you usually can create different PoolManagers that hold different instances. So, you can always start from "fresh" by just creating another PoolManager instance (what you also couldn't if you use global variables instead). For at least, in theory, in practice you might still run into some issues, because creating these instances is slow and you might physically not have arbitrary many of them (I mean, these are the reasons why you need PoolManagers in first hand). One advantage of PoolManagers is that they can implement some logic to check for it, and in case block the corresponding thread, till some resources are available again, allowing you to write individual tests or functionality running independent and in parallel where you don't have to check in the business or testing logic for such hardware constraints.
You are correct in observing that the object pool pattern is not a direct substitute for the singleton pattern. They serve different purposes and have their own sets of limitations. The key is to understand when to use each pattern appropriately, based on the requirements of the specific problem you are trying to solve.
The singleton pattern is used when you want to ensure that a class has only one instance throughout the application's lifetime. This is useful when you need to coordinate actions across the system or share resources. However, as you've pointed out, singletons have some downsides, such as potential issues with inheritance, testing, and thread safety.
On the other hand, the object pool pattern is designed to manage a set of reusable objects efficiently. This pattern is particularly useful when object creation is expensive (in terms of time or resources), and you want to limit the number of instances created. Object pools can help improve performance by reusing objects and reducing the overhead of object creation and destruction. As you've mentioned, object pools have their own limitations, such as the need to manage object states and potential thread safety issues.
In summary, the object pool pattern is not meant to be a direct replacement for the singleton pattern. Instead, they are two distinct design patterns with their own specific use cases and trade-offs. Choosing the right pattern depends on the requirements of the problem you are trying to solve and the specific constraints of your application.
I don't understand why your subs are not in millions given the production quality and the way you teach.
Just quality content through and through
Thank you Anjul! I’ve been doing this for only 6 months, and RUclips takes time to grow. Millions? Not sure, but let’s see :).
Found this channel yesterday. Can't stop watching.
Glad you enjoy it!
samesies
Teaching abstract concepts like this with concrete code walkthroughs in a clean, clear, and concise format with a simple example is what sets you apart from the others on RUclips. Can't think of a more efficient way to learn/refresh myself on this stuff, especially for my upcoming interviews, thank you
Wow, thank you so much!
Easiest way (and the correct way) to do this is to simply import the object r.
thats it. Because whenever it is imported somewhere else, it returns the same object.
So, the object pool isn't needed at all, since Pythons module architecture takes care of all this.
Really enjoy your videos! Breath of fresh air compared to the re-hashed theoretical examples the plague RUclips.
Glad you like them, Heinrich!
At last the singleton is simple...
Maybe there is some case where a pool makes sense, but if you evee make a size=1 pool, you have essentially create a resetable singleton with needless extra complexity.
But later it will hunt you
I think the problem this solves is being testable. A singleton is not testable and not sandboxed, leaving room for fun side effects
@@dylanjonesSD its no more or less testable then a singleton with a destroy function.
I'm really glad that YT suggested your channel to me. Awesome content and great explanation skills you have! Thank you for all your videos, I learned a lot!
Awesome, thank you!
I watched every videos that you posted to study design patterns and I really enjoyed watching your channel the most because your code examples are really neat and clean and this is a perfect example for me as a person likes to write/work with codes simply does the main purpose. Anyways, I appreciate your time making this video! Liked and Subscribed from LA:)
Thank you Max, glad you liked it!
Great content, Arjan! Keep it coming (as well as the bloopers in the end :D )
This was just top notch quality. Thoroughly enjoyed it. The explanation and quality of code is nothing short of beautiful
Thanks so much Pranav, glad the content is helpful!
You create my favorite content on Python! Thank you!
Thanks man, glad to hear!
Thank you for the clear and concise explanation. Love your videos!! BIG LIKE!!!
Thank you - glad you liked it!
Loved the bloopers at the end!
Another excellent video related to the design patterns. You should really think about creating a course on design patterns.
I need more quality content like this!
Hi Robert, thanks! Yes, I’m seriously considering a more in-depth course. That will probably not be on RUclips though but on a platform more suitable for that kind of material.
Thank you for presenting the object pool to me :) I have never heard about it before.
Regarding your claim, to rather use an object pool over the singleton pattern, i cannot agree:
I use singletons for wrapping external APIs/SDKs. The singleton class/object is in my case a global and static object. It gives me the advantage, that I do not have to pass a reference to the wrapper class through tenths layers of class hierarchies. Instead I can access the external API from any class in my hierarchy. Of course, this has the major drawback of introducing (intransparent) coupling of many classes to this singleton.
The second point is multi-threading: The object pool, seems to be thread safe by design (nice feature). But it may be inefficient, if the wrapped API/SDK handles multi-threading by itself. Either the object pool needs to be way more complex or I have to instantiate a new object for each thread. If I use the singleton pattern, I do not have this "feature".
I enjoy your videos, and you have opened my eyes to new levels of Python programming elegancy! I have been programming for a living for 26 years, and I agree that singletons are rarely something that is a good idea, but I do use them occasionally in C++ when a static member function isn't the better solution. Of course, you need to get an instance of the class before starting 8 threads that use it - otherwise, you are asking for trouble :-) So, I would say it has its (limited) uses. Thank you very much for your videos!
Thanks Jesper, glad that the videos are helpful to you!
This channel is a gem!!, kudos 🙏
Thank you!
Great Series! Thank you! What is the mechanical keyboard that you are using ?
I feel like this glossed over a bunch of things. Like if i want a singleton (eg. Setup is expensive, or a client is handling batching) i don't ever want more than 1 of them.
I feel like the acquire/release pattern addresses concerns related to only having some predictable number of instances concurrently, but it doesn't really address the concern about wanting to reuse the exact same instance throughout the lifecycle of the application. For example, I only want to read ny config from disk one time - I don't want to do disk I/O every time I need a config value. Or if I am instrumenting some kind of batch processing, if each message ended up creating a new instance then I don't get any benefits of batching.
Honestly this example felt like we just made a factory/object pool for the class but now we need a singleton to ensure there is only 1 object pool. We don't want to have to pass that factory around to everything that needs an instance. One of the perks of the singleton pattern is that you can just ask for a reference from anywhere in your complex application without having to first be passed some state/context.
Lastly, one of the arguments that you made against the singleton pattern was related to testing. And I do very much agree that complicates things. You don't want state from previous tests leaking out into subsequent tests and the singleton pattern would absolutely do that by default.
Overall, I think this channel has excellent content and I think there are a lot of great concepts that are well-explained. I think you do a great job at teaching. I would have loved to see:
1. How to access your pool from other modules, ensuring that only 1 gets created
2. How to share the same instance across multiple places that must use it
3. An example of a couple unit tests for the object in a pool that are demonstrated to be improved by using the pool instead of a singleton
3:24 If ever, singletons should be implemented using the __new__().
Yup, i create singletons with new and not face the recreate instance at anywhere which is an singleton pattern insurance I think video figure out the reusable feature in the python.
Great video! I just had a question about the use of object pools. How is that different from simply instantiating the class wherever it is needed and then sending it to the garbage collector? If the pool needs to reset released objects, isn't that functionally the same as creating a new object? You could maintain a class variable that contains the number of objects created and forbid creation in the __new__() method?
seems like you can add a little bit of thread safety by protecting your critical sections -- the acquire/release() ?
love the outtakes!
Thanks!
Kind of rare thing in the wild. Last time I needed a singleton I ended using a module with pack of global variables and setters/getters for them.
Dude, your videos are amazing
Thank you, glad you enjoy the videos!
Funny, I was earlier today goofing around trying to implement signature overloading in python using decorators and I ended up using this a similar pattern. Basically, I had one instance of the decorator class per function in a group of overloadable functions so i simply returned the old one if it existed
Coming from Java, the use of the Singleton pattern isn't as frowned upon, especially when compared to utility classes with only static methods (singletons are easier to test then utility classes). But the reason it is more useful in Java is because everything is a class and therefore you can't just define methods by itself without defining a class. So I see why you wouldn't need them (or even want them) in python.
Just out of curiosity. Have you ever had len function return a negative number when called on a list?
Modules are nice replacement for singletons, but what about defining interface for modules? You can use abc as interface and reusable code provider for different controllers that other can implement, should you use Protocol to define interface for module if you want to allow creating different controllers for your application? Hard to combine it with providing code that should be useful to controller implementator. Do you have any pattern for this case that works with module?
I too would tend to use a module in most situations, but none of the reasons given for avoiding Singletons are very persuasive. The last doesn't even apply (in comparison with an Object Pool pattern that has exactly the same problem). . The 'breaks OOP because inheritance will give surprising results' is just ..oh, boo hoo. if you are worried, seal the class (supported by __init_subclass__ since Python 3.6). The 'no control over creation' objection would also apply to modules - and why would you care so long as the code gives you a squeaky clean object to play with (this only becomes a point if you are trying to implement/not implement lazy creation surely). The testing objection can be easily overcome. So as I say -- not convinced. I would still use a Singleton cheerfully if I felt I did not want to use a module.
ARJAN, please upload a series of class designing and then coding and alternatives
Thanks for your suggestion!
Awesome video!
Thanks, Robert!
I'm more interested in what alternative I can use instead of Singleton. In my way I have connections to DB and RabbitMQ, they both being coded by previous team leader into an singleton, however singleton have a lot of troubles with it. But I also at the same time was unable to refactor it to something that is better, than that.
He mentioned that the import system that provides python has all the benefits and none of the problems that singleton gives.
Thanks!
Thank you too!
I noticed that after you have released the r2 object back to the pool, you can still call it from the r2 variable. Looks like undesirable behavior.
How can we prevent this?
How about keeping single instance via injecting regular instance (not singleton) into some kind of global module and then importing it? For example:
*instances.py*
from typing import Optional
DB_CONN: Optional[DbConnection] = None
*conftest.py*
import instances
def pytest_sessionstart():
instances.DB_CONN= DbConnection()
def pytest_sessionfinish():
from instances import DB_CONN
if DB_CONN:
DB_CONN.quit()
*script.py*
from instances import DB_CONN
def main():
DB_CONN.execute_sth()
Of course, that's example only to show the idea. Do you see any bigger disadvantage of this solution?
Awesome work! 😊 Please record video about DDD or microservices in Python ❤
Thank you for the suggestions!
FIrst of all really great content! I still don't understand how I could share the created object pool among multiple places in my program, without passing a reference to every class in a lower hierarchy. You mentioned it could be imported as a module, but doesn't that just create a new object pool for every instantiation?
In python if you import a module in multiple places its only actually loaded once
Well u can make singleton init threadsafe in C++ and make the class final to avoid anyone extending it. Its only an antipattern if u misuse it - like implementing it in Python instead of just using modules
blooper reel is hilarious
I'm happy I've never learned this kind of wizardry. Thanks for the tip though.
One possible use case of Singleton pattern. Making sure that only one instance of your complete code can run at a time even if you try to call it multiple times. When my code runs, I am writing the process id into a file, if the file already exists, then I will not allow the next process to call the same code. Is this a correct strategy to ensure only a single instance of your code can run at a time? Are there better ways?
I would suggest to use a database for that instead of a file, since reading and writing to a file is generally not atomic so you might still end up with a race condition. You could even consider to make it more generic and use a job scheduling architecture. This allows for more flexibility and you could use something like this: pypi.org/project/schedule/.
Thanks
Thank you so much!
I expect there is a hotkey for running your code. Learning it would save you from moving your cursor to the run button.
I like your explanation and I agree with you that Singletons are an Anti-Pattern, but I'd like to elaborate:
1) Singletons are global state by design. In most cases you do not want to use global state if it ever changes. So classes that only provide information and are not mutable can be Singletons. Be very sure that that is the case however, in most cases this is not the ideal solution.
2) As Arjan said: There is no real need for Singleton classes in Python, because a module is just so similar to that, so in the very few cases that actually have a good use for a Singleton it almost always makes sense to use a module for that.
As a side note: You could have done this:
r = self.free.pop(0)
Instead of retrieving and removing the object.
To create singleton in python you can just use class instead of its objects!!! That simple!
class Logger:
file, prefix, level
@classmethod
def Error(cls, msg):
if level > ERROR:
return
file.Write(prefix + msg)
@classmethod
def Warning(cls, meg):
...
@classmethod
def SetLevel(cls, level):
# do assesment
cls.level = level
Is NoneType a pool or a singleton in Python? If it's a singleton, would it be better if a pool?
Don't know why I didn't use the opportunity of commenting on this channel's great production value and non trivial contents!
Isn't removing from a list a linear time operation? 🤔 Why not use a deque or pop from the end of the list?
Which keyboard you are using?
It’s a Keychron K2, see: amzn.to/3xD7b69.
can not we create constructor with __init__ for singleton ?
The reason I used the metaclass in the example is that it makes sure you're not creating multiple instances by calling the initializer multiple times. If you add __init__ to the class that is supposed to be a singleton, you can create multiple instances because Python doesn't have private access modifiers.
@@ArjanCodes Thanks a lot, got it.
Is the logger class in python not a singleton? I thought it was
I guess it just acts similarly
@@lepidoptera9337 But shouldn't one logger handled everything within a single program? There are handlers that can split different levels of data into different places.
Are imports in Python singletons?
Great series!
Well, modules are, kind of... since there's only one 'instance' of a module that you're using they offer the same functionality as a Singleton class (without all the issues related to singletons).
@@ArjanCodes Not exactly. That depends on how you import the module. Like doing an import from within a package or from outside the packages it gives you two different instances. It ends up having different "namespaces" I suppose. Tripped over this a few weeks ago.
Just noticed the letters in the background.
An Easter egg in plain sight ;).
An object pool is not a replacement for a Singleton. A pool can allow more than one instance, and if it allow only one instance, it is a Singleton.
Can you please zoom in the screen when typing code, it is very hard to see
In more recent videos, this has been fixed.
Just name it __TheThingClass and write a get_the_thing function or a proxy that ensures the behavior you want. We are adults, we know not to Instantiate a "private" class elsewhere.
1:24 jajajajajaj
Why do we need for?
self.free = [Reusable() for _ in range(size)] ?
Why not use pop instead of remove?
r = self.free.pop()
Singletons are bad now? Damn, they were taught to me specifically because they were good for testing. They were the 1st pattern taught in my Design Pattern book
But if I don't need multiple objects, the idea of a singleton fits this purpose
Python: Can't do private.
Also Python: "SINGLETONS BAD!" :q
Angular implements singleton services in a very cleaver way.
If python doesn’t have a “private” modifier, the problem is in python 🤷♂️
that example is not of a singleton, it's of a Borg class
Resistance is futile. ;)
I really dont understand what the fuzz with singletons is about.
From my perspective, a singleton is a struct with no fields. Hence all instances of it are identical. What does this have to do with global state, thread-safely and testing issues? There seem to be different definitions on what a singleton is. When I look at the python code all I see is the usual overcomplicated oop bs
Haha Kotlin object go brrrrrr
Have you used something like an object pool before?
no, never
No
Factories are perfect candidates for singletons. There's no need to create multiple factories.
Factories are great candidates for not being objects in the first place tbh
@@superscatboy highly dependent on your application and the role of the factory. Doing something expensive at startup time that requires a dependency statically is a footgun in Java for example (honestly, a bit of a nightmare anyway, I hate startup time Python things).
Even in python, (almost) everything is an object, so you're just saying don't use factories in python.
@@9e7exkbzvwpf7c Can you give an example of when refactoring a singleton factory into a free function would introduce additional overhead? I'm not seeing it.
Both versions of the code would have to do the exact same thing with an almost identical amount of work, the only real difference being that the singleton version also has the extra overhead of handling all the unnecessary singleton pattern stuff.
I have no idea what's going on here. Metaclasses buried my understanding. But at least I know not to use singletons.
something about ur videos make a program better i wish they were long as hell lmao
Hell is indeed pretty long ;). Glad you're enjoying the vids!
Singleton is fine