Nope, this is not a special syntax, this is __or__ magic methods. For "a | xxx | b" it will create use xxx.__or__(xxx.__or__(a, b), b). And during this "process" it will allocate two complex objects with that magical methods. It's relatively easy to implement, but it's not recommended for perfomance reasons. And this is not a syntax this is used for calling __or__ method. Just in case someone will NOT wait until end
Although your understanding is basically right, your code rephrasing is wrong. It's actually simply xxx.__ror__(a).__or__(b). Also, although you can optimize this a bit to only have a single xxx object, it's really not that important of a thing. Recreating objects is an acceptable performance loss in lots of design patterns (e.g. immutable data structures), and frankly if you're worried about the performance hit of these operations, you probably shouldn't be working in Python. The real reason why I would not want to adopt such a pattern is because although it's syntactically correct, to someone who understands Python semantics, it seems almost nonsensical. In Python, `|` in many contexts is known as the bitwise OR operator. As the name implies, it's most often used to perform bit calculations on the given arguments (e.g. `12 | 4` would give you 12). So seeing something like `10 | mult | 4` is semantically ambiguous at a glance for common python users, since it looks like `mult` is some normal variable that you're including in your bit calculations. So everyone who writes in your codebase needs to be aware of this non-standard use of bitwise OR that doesn't actually do bit calculations, but instead facilitates an infix operation pattern. Basically it's an interesting niche thing that might be cool if you really like how the infix syntax reads, but it's definitely not something you want to introduce outside of your own personal/very small team projects.
@@Indently Yeah 100%, was just responding to the main thread guy. It's always neat to see these niche videos because it highlights one of Python's double edged strengths/weaknesses giving developers significant control over even some fundamental operations that you would not see in many other languages.
@@NerdyStarProductions maybe the fact that "| mult |" means something special as a unit could be clearer if it was written "|mult|" without the spaces? the formatter would rip that apart though... if it's possible to customize that behavior, it might help to treat these special operator keywords as a conceptual unit. Which would look weird at first and raise eyebrows about whether python added new syntax or sth, but is also extremely intuitive after the initial surprise.
My most recent Python geek out was with multiprocessing module: using a process to spawn a process so it could keep status updating without leaving all spawned processes up to the main thread to perform the status checks while it's alive. Process checks when spawned can only be done by the parent so in order to make a process continuously check on progress of a real working process you gotta make another process for the status checks then launch the work process inside of it. I'm not a Python pro so maybe everyone already knows this but I thought it was neat.
I used to define infix operators all the time in R, where i found it very useful, especially with pipes. I've never seen this in Python before, but I can see using it now that I know about it - I'd probably tighten up the spaces from the pipes to the operator, though. I'd be careful about when it breaks, however. It might break when you have several of these chained together in one expression without a mess of parenthesis to assure proper precedence - I haven't tried it.
This is part of the reason why I find R code is hard to read, and I don't like to read sort of 90% of available R codes (fun fact is that most of them are written by researchers, not programmers). R (the language itself and part of it's user base) love to use native math symbols which is weird and hard to follow in a programming point of view.
There are 2 immediate downsides I can see for this: (1) It's going to be slower. So if it's part of a code section that needs to run 100,000 times or more, you'll feel the slowdown. (2) There's a risk people who don't know this trick will have a harder time reading and understanding this type of code. There could be a use case where you have to chain a couple of operations together and an infix operator makes it more readable, but in 7 years I haven't come across such a case.
You dont actually require both arguments for infix operators. You can totally define default behavior or default values. In general, I find parsing the AST itself for defining unique new behavior ends up being cleaner (although defining new "operators" gets messy this way), but infix has always been a neat little trick. Also for anyone reading, any operator with similar presedence and structure will work, > for instance. You might also consider overriding @, as it has no default behavior. Very nice coverage.
Some time ago I - with the help of some kind Redditor - used the same concept to implement function composition, but with the right shift operator cause I thought it looked clearer than the pipe. Cool thing about it is that it allows for the composition of arbitrarily many functions. Looks like so: ```py class Composition: def __init__(self, f): self.f = f def __rshift__(self, g): def wrapper(*args): return g(self(*args)) return Composition(wrapper) def __call__(self, *args): return self.f(*args) @Composition def compose(*args): return args[0] if len(args) == 1 else args # example usage def flatten(*xss) -> list: return [x for xs in xss for x in (flatten(*xs) if isinstance(xs, (list, tuple, set)) else (xs,))] none_nested = compose >> flatten >> any >> (lambda x: not x) print(none_nested([[0, False], [("", )]])) # prints True print(none_nested([0, [1]])) # prints False ```
This would be useful is CPython JIT compiled it in place, but the CPython interpreter does not do that. So, ultimately, this has no performance benefit. Beyond that, as many people have pointed out, it's not intuitive. You better at least remove the white space around the function name, and even then, it's still confusing.
I believe one use case for this is to expand number of mathematical operators without overwriting existing ones or need to create new objects types. This come out during discussions about operators for matrix operators in PEPs. Also some people just find infix operators more redable than normal function calls, especially when experesing mathematical formulas. The reason why exemple on the side presented two operators (|infix|, ) for this, is becouse they have different precedence if you would like to mix them with normal operators while still ensuring right order of operations.
I think it would be cool if you could use infix without the pipe symbols, which basically would be just operator overloading with custom defined operators. That would be pretty cool for things like matrix multiplication where you can just do something like: C = A . B The only downside is how unreadable some code bases would be, but frankly if you're using Python there's a strong chance it's pretty unreadable anyway unless you have a SE/CS background lol
This syntax is used by Langchain in their LCEL : prompt = ChatPromptTemplate.from_template("tell me a short joke about {topic}") model = ChatOpenAI() output_parser = StrOutputParser() chain = prompt | model | output_parser chain.invoke({"topic": "ice cream"})
but that only applies when both operands are dictionaries, so there is no conflict here, since in this implementation one opperand is always of type Infix.
🎯 Key Takeaways for quick navigation: 00:00 🚀 *Introduction to Infix Operators in Python* - The video introduces the concept of infix operators in Python, which allow users to create custom operators with a unique syntax. 00:53 🌟 *Creating Custom Infix Operators* - Demonstrates how to define custom infix operators in Python, using the example of creating a custom multiplication operator. 02:42 🧐 *Understanding Infix, Prefix, and Postfix Operators* - Briefly explains the concepts of infix, prefix, and postfix operators and their order in expressions. 03:24 💡 *Implementing Infix Operators in Python* - Provides a step-by-step implementation of custom infix operators in Python, using the `infix` class and dunder methods. 07:58 🤔 *Practicality and Usefulness* - Discusses the practicality and usefulness of creating custom infix operators in Python, highlighting that it may be more of an "Easter egg" feature and invites viewers to share their thoughts on it. Made with HARPA AI
can anyone think of any example where this is not just a more confusing way to call a simple 2 arg function? I cannot think of anytime where I may want to use x | foo | y instead of foo(x,y)
Can anyone think of any example where this "2+2" is better than just writing "sum(2,2)"? I cannot think of anytime where I'd want to use 2+2 instead of sum(2,2).
You can just do foo.function(bar), I don't think there's that many uses where it's better unless you want to obfuscate code and don't mind the slowdown.
Operator overloading can make some code pretty, but is generally a bad idea, because it's not obvious what the new operators are doing at first glance.
Operator overloading is only bad when the implementation doesn't follow the semantics of the operator. This example is bad because it overrides the bitwise or operator to compute something other that a bitwise or. When implemented correctly (like adding/scaling vector types) operator overloading only makes code simpler and easier to read.
Overloading in C++ looks amazing when you first see it, but then you learn to only use it when needed, which is rare. I must admit the mention of bitwise operators in python would be interesting, but I suspect they would not be efficient enough to warrant it.
Quite a neat trick, although I'd be very hesitant to use it in practice. In most cases, if the same thing can be accomplished using a simple function I'd use it since it's much easier for other people to understand and doesn't require knowledge of Python oddities. There's a quote that I think often applies to cases like this. "Debugging is twice as hard as writing the code. So if you write your code as cleverly as possible, you are by definition not capable of debugging it."
Is this kind of like the #DEFINE macro in C? Like is it a way to reduce rewritten code and a bit of a compiler optimization rather than making a regular function?
its just overloading the bitwise or, so `2 | mul | 3 | add | 6` *should* just return 12. but, also because it is just bitwise or, theres not really a way, in this implementation, to define precedence or associativity drection. so `6 | add | 2 | mul | 3` would give you 24 where `6+2*3` would give you 12, and `2 | pow | 3 | pow | 2` would give you 64 where `2**3**2` would give you 512
this is a very interesting insight on how python actually works in the background. i can imagine that for specific projects this could be used to make a far more readable code.
Python doesn't "allow" this syntax, you created this syntax by implementing the __or__ and __ror__ dunder methods in the infix class Otherwise we could say that Python allows literally any random syntax as long as you implement it
Why just ahh if it was haskell I would have approved it but no This is just .... Eah You know what forget it It's Ok *BUT IF YOU EVER PULL STUNT LIKE THIS AGAAAAIN! I WILL ....*
Federico, that is a truly ugly bit of legal syntax! 😁 Speaking of using operators in unexpected ways, pathlib does some of the same shenanigans with / to allow you to join paths in a convenient and portable manner, which reminds me of this clever little bit of code that IIRC came from StackExchange: config = os.environ.get('APPDATA') or os.environ.get('XDG_CONFIG_HOME') config = Path(config) if config else Path.home() / ".config" That doesn't account for macOS (and yes, Path.home() / 'Library' / 'Preferences' works!), but it's clever and if you'd like to further share it, go right ahead.
Nope, this is not a special syntax, this is __or__ magic methods. For "a | xxx | b" it will create use xxx.__or__(xxx.__or__(a, b), b). And during this "process" it will allocate two complex objects with that magical methods. It's relatively easy to implement, but it's not recommended for perfomance reasons. And this is not a syntax this is used for calling __or__ method. Just in case someone will NOT wait until end
I understand what you’re getting at, but this is still Python syntax, otherwise it would raise a SyntaxError :)
Although your understanding is basically right, your code rephrasing is wrong. It's actually simply xxx.__ror__(a).__or__(b). Also, although you can optimize this a bit to only have a single xxx object, it's really not that important of a thing. Recreating objects is an acceptable performance loss in lots of design patterns (e.g. immutable data structures), and frankly if you're worried about the performance hit of these operations, you probably shouldn't be working in Python.
The real reason why I would not want to adopt such a pattern is because although it's syntactically correct, to someone who understands Python semantics, it seems almost nonsensical. In Python, `|` in many contexts is known as the bitwise OR operator. As the name implies, it's most often used to perform bit calculations on the given arguments (e.g. `12 | 4` would give you 12).
So seeing something like `10 | mult | 4` is semantically ambiguous at a glance for common python users, since it looks like `mult` is some normal variable that you're including in your bit calculations. So everyone who writes in your codebase needs to be aware of this non-standard use of bitwise OR that doesn't actually do bit calculations, but instead facilitates an infix operation pattern. Basically it's an interesting niche thing that might be cool if you really like how the infix syntax reads, but it's definitely not something you want to introduce outside of your own personal/very small team projects.
I think I mentioned in the video, that it was for fun :)
@@Indently Yeah 100%, was just responding to the main thread guy. It's always neat to see these niche videos because it highlights one of Python's double edged strengths/weaknesses giving developers significant control over even some fundamental operations that you would not see in many other languages.
@@NerdyStarProductions maybe the fact that "| mult |" means something special as a unit could be clearer if it was written "|mult|" without the spaces? the formatter would rip that apart though... if it's possible to customize that behavior, it might help to treat these special operator keywords as a conceptual unit. Which would look weird at first and raise eyebrows about whether python added new syntax or sth, but is also extremely intuitive after the initial surprise.
My most recent Python geek out was with multiprocessing module: using a process to spawn a process so it could keep status updating without leaving all spawned processes up to the main thread to perform the status checks while it's alive. Process checks when spawned can only be done by the parent so in order to make a process continuously check on progress of a real working process you gotta make another process for the status checks then launch the work process inside of it.
I'm not a Python pro so maybe everyone already knows this but I thought it was neat.
Nice
very maintainable and not confusing at all
Calling that a syntax is a stretch but ok. You can do it in c++ and any other language with operator overloading.
I used to define infix operators all the time in R, where i found it very useful, especially with pipes. I've never seen this in Python before, but I can see using it now that I know about it - I'd probably tighten up the spaces from the pipes to the operator, though. I'd be careful about when it breaks, however. It might break when you have several of these chained together in one expression without a mess of parenthesis to assure proper precedence - I haven't tried it.
Don't use that, it's obscure and will lead to unreadable code
This is part of the reason why I find R code is hard to read, and I don't like to read sort of 90% of available R codes (fun fact is that most of them are written by researchers, not programmers). R (the language itself and part of it's user base) love to use native math symbols which is weird and hard to follow in a programming point of view.
Man I was waiting in anticipation for the use cases. RIP.
There are 2 immediate downsides I can see for this:
(1) It's going to be slower. So if it's part of a code section that needs to run 100,000 times or more, you'll feel the slowdown.
(2) There's a risk people who don't know this trick will have a harder time reading and understanding this type of code.
There could be a use case where you have to chain a couple of operations together and an infix operator makes it more readable, but in 7 years I haven't come across such a case.
You dont actually require both arguments for infix operators. You can totally define default behavior or default values.
In general, I find parsing the AST itself for defining unique new behavior ends up being cleaner (although defining new "operators" gets messy this way), but infix has always been a neat little trick.
Also for anyone reading, any operator with similar presedence and structure will work, > for instance. You might also consider overriding @, as it has no default behavior.
Very nice coverage.
Some time ago I - with the help of some kind Redditor - used the same concept to implement function composition, but with the right shift operator cause I thought it looked clearer than the pipe.
Cool thing about it is that it allows for the composition of arbitrarily many functions.
Looks like so:
```py
class Composition:
def __init__(self, f):
self.f = f
def __rshift__(self, g):
def wrapper(*args):
return g(self(*args))
return Composition(wrapper)
def __call__(self, *args):
return self.f(*args)
@Composition
def compose(*args):
return args[0] if len(args) == 1 else args
# example usage
def flatten(*xss) -> list:
return [x for xs in xss for x in (flatten(*xs) if isinstance(xs, (list, tuple, set)) else (xs,))]
none_nested = compose >> flatten >> any >> (lambda x: not x)
print(none_nested([[0, False], [("", )]])) # prints True
print(none_nested([0, [1]])) # prints False
```
I think I'm gonna start a charity for all those affected by this. Absolute madness 😆
I will do it next time on the video that causes the trauma xD
This would be useful is CPython JIT compiled it in place, but the CPython interpreter does not do that. So, ultimately, this has no performance benefit. Beyond that, as many people have pointed out, it's not intuitive. You better at least remove the white space around the function name, and even then, it's still confusing.
I believe one use case for this is to expand number of mathematical operators without overwriting existing ones or need to create new objects types. This come out during discussions about operators for matrix operators in PEPs. Also some people just find infix operators more redable than normal function calls, especially when experesing mathematical formulas.
The reason why exemple on the side presented two operators (|infix|, ) for this, is becouse they have different precedence if you would like to mix them with normal operators while still ensuring right order of operations.
I think it would be cool if you could use infix without the pipe symbols, which basically would be just operator overloading with custom defined operators. That would be pretty cool for things like matrix multiplication where you can just do something like:
C = A . B
The only downside is how unreadable some code bases would be, but frankly if you're using Python there's a strong chance it's pretty unreadable anyway unless you have a SE/CS background lol
I think not defining custom operators, but just using dunder methods is more useful, you can see it in pathlib. Path(/root) / usr / path
This syntax is used by Langchain in their LCEL :
prompt = ChatPromptTemplate.from_template("tell me a short joke about {topic}")
model = ChatOpenAI()
output_parser = StrOutputParser()
chain = prompt | model | output_parser
chain.invoke({"topic": "ice cream"})
isn't this just custom pipe operator?
@@hkgx custom bitwise or operator
since python 3.9 (pep 584) the pipe is actually used for "merging" dictionaries like a | b or a =| b for inplace
By “actually” did you mean “also”? Either way, true :)
but that only applies when both operands are dictionaries, so there is no conflict here, since in this implementation one opperand is always of type Infix.
@@Indently yeah, i mean Like Guido initially ment the feature should work, but it's Python, you know... Monkey patching and similar stuff...
@@blanky_napwhat is monkey patching? I have heard abouy it but don't know any details
This is great. Thanks. Just wondering if we can create our own dunder methods and attach it to an external function
This kind of syntax remembers me about the CYPHER language for neo4j knowledge graphs.
Maybe you can improve union and intersection of sets with this
You could probably make a simple interpreter using this method. like maybe an assembly interpreter.
I thought you were gonna implement pipe operator. Pipe is insane. Never knew python can do that.
He does right. The __or__ is just the pipe if it it to the left side of the Infix class, and the __ror__ is for when its to the right
langchain uses it to create pipelines and I think its very neat...
It just looks like regular old operator overloading to me.
Cool. Operator overloading.
🎯 Key Takeaways for quick navigation:
00:00 🚀 *Introduction to Infix Operators in Python*
- The video introduces the concept of infix operators in Python, which allow users to create custom operators with a unique syntax.
00:53 🌟 *Creating Custom Infix Operators*
- Demonstrates how to define custom infix operators in Python, using the example of creating a custom multiplication operator.
02:42 🧐 *Understanding Infix, Prefix, and Postfix Operators*
- Briefly explains the concepts of infix, prefix, and postfix operators and their order in expressions.
03:24 💡 *Implementing Infix Operators in Python*
- Provides a step-by-step implementation of custom infix operators in Python, using the `infix` class and dunder methods.
07:58 🤔 *Practicality and Usefulness*
- Discusses the practicality and usefulness of creating custom infix operators in Python, highlighting that it may be more of an "Easter egg" feature and invites viewers to share their thoughts on it.
Made with HARPA AI
you think if we badger the python foundation enough they'll add a __juxtaposition__ operator for clean infixes?
Probably never going to use it, since I have no idea where the pipe is on my keyboard
can anyone think of any example where this is not just a more confusing way to call a simple 2 arg function? I cannot think of anytime where I may want to use x | foo | y instead of foo(x,y)
Can anyone think of any example where this "2+2" is better than just writing "sum(2,2)"? I cannot think of anytime where I'd want to use 2+2 instead of sum(2,2).
Mathematical operators, text filters, collection filters... there's all sorts of ways it can be used.
You can just do foo.function(bar), I don't think there's that many uses where it's better unless you want to obfuscate code and don't mind the slowdown.
Operator overloading can make some code pretty, but is generally a bad idea, because it's not obvious what the new operators are doing at first glance.
Operator overloading is only bad when the implementation doesn't follow the semantics of the operator. This example is bad because it overrides the bitwise or operator to compute something other that a bitwise or. When implemented correctly (like adding/scaling vector types) operator overloading only makes code simpler and easier to read.
Overloading in C++ looks amazing when you first see it, but then you learn to only use it when needed, which is rare. I must admit the mention of bitwise operators in python would be interesting, but I suspect they would not be efficient enough to warrant it.
@@terra_creepertell that to airflow designers overloading the bitshift operator for dag definitions
Hear me __roar__!
Can we set the priority of the created operator in Python?
"Hah thats gonna leave a mark" - Captain Stenley
Quite a neat trick, although I'd be very hesitant to use it in practice. In most cases, if the same thing can be accomplished using a simple function I'd use it since it's much easier for other people to understand and doesn't require knowledge of Python oddities.
There's a quote that I think often applies to cases like this. "Debugging is twice as hard as writing the code. So if you write your code as cleverly as possible, you are by definition not capable of debugging it."
clever code == easy code to debug
what is pipline in python?
.NET also has a function that does that. It’s very helpful
Ha! You have turned Python into Pearl......
Is this kind of like the #DEFINE macro in C? Like is it a way to reduce rewritten code and a bit of a compiler optimization rather than making a regular function?
It's just operator overloading, so technically it is still a function. It's basically the same as defining custom behavior of +, -, * and /.
This looks excessively complicated and unnecessary
I hope defining 2 dunder methods isn’t considered excessively complicated these days xD
Can someone explain to me why or where this thing is useful?
What about 2 | mul | 3 | add | 6? Or doing apl/haskell kind of stuff
I haven’t tried that! I will try it when I get the chance
its just overloading the bitwise or, so `2 | mul | 3 | add | 6` *should* just return 12. but, also because it is just bitwise or, theres not really a way, in this implementation, to define precedence or associativity drection. so `6 | add | 2 | mul | 3` would give you 24 where `6+2*3` would give you 12, and `2 | pow | 3 | pow | 2` would give you 64 where `2**3**2` would give you 512
Sound only useful for an obfuscation contest.
I don’t see the benefits of infix, especially this implementation- however clever it may be.
this is a very interesting insight on how python actually works in the background. i can imagine that for specific projects this could be used to make a far more readable code.
I’d say this feels like an abuse of magic methods, but I honestly can’t think of any other reason that the __ror__ method would exist. This is wild.
I should be able to use this to chain generators without nesting them.
Cool. Another thing that took Python about 30 years to provide (plus botched lambdas, but I digress). Oh well. :|
So far from daily usage as lambdas would be
That is a monad
what is that title? I didn't even know that syntax existed
Very nice functionality in Python.
It could be more nice if there were no pipelines 😅
But there are still the >> operators which you can use the same way ;)
@@Indently Hmm, I think it is there to district "user defined" and "built in" keywords.
@@dipeshsamrawat7957 No. The Pipe "|" simply means "Bitwise OR". This is an operation which in Python is desugared to a function call to __or__
Cool, now do it with proper typings! 🤣
Man, this looks so ugly. I love it!
that's completely useless, I will use it from now on .
That's the attitude!
Python doesn't "allow" this syntax, you created this syntax by implementing the __or__ and __ror__ dunder methods in the infix class
Otherwise we could say that Python allows literally any random syntax as long as you implement it
Why even bother?? It's a function, this is language bloat by definition.
"Incredibly insane". Not a serious person. Click.
Why just ahh
if it was haskell I would have approved it but no
This is just ....
Eah
You know what forget it
It's Ok
*BUT IF YOU EVER PULL STUNT LIKE THIS AGAAAAIN! I WILL ....*
Federico, that is a truly ugly bit of legal syntax! 😁 Speaking of using operators in unexpected ways, pathlib does some of the same shenanigans with / to allow you to join paths in a convenient and portable manner, which reminds me of this clever little bit of code that IIRC came from StackExchange:
config = os.environ.get('APPDATA') or os.environ.get('XDG_CONFIG_HOME')
config = Path(config) if config else Path.home() / ".config"
That doesn't account for macOS (and yes, Path.home() / 'Library' / 'Preferences' works!), but it's clever and if you'd like to further share it, go right ahead.