I'm glad you're changing the model to center transactions as the source of truth, because there was a problem with your last implementation that wasn't addressed. If you tried to do a transfer from an account with insufficient funds, the undo step would remove funds from the other account while it had never received them in the first place, because the exception of insufficient funds was not handled. This removes that ambiguity.
Nice Arjan, great content, and so much clarity and calm to explain. The transaction approach in this design seems very powerfull and yet simple. Thank you! I did work on the implementation of a 3D based python software in the past and coincidentally I recently throught to add this type of feature to it. If you are wondering, I work in the metrology (spatial, mainly) & alignment group of a synchrotron laboratory here in Brazil, with lots of 3D challenges. Cheers!
Arjan, amazing work and thank you for these videos, they are fantastic. I was wondering if you can add some flowcharts or diagrams with the design patterns to have a bigger picture at the same time of how they flow and work. Thank you again for these really high quality tutorials.
I'd like to add a vote for Event Sourcing in a future video. I've been looking into Event Sourcing and Kafka lately and from what I understand so far, Event Sourcing with Kafka could help address the ledger deletion issue as mentioned in the comments. So it would be great to see this or a similar/extended example implemented with Event Sourcing. Thanks for covering the Command Pattern so thoroughly!
I'm very surprised you didn't mention Redux and other immutable/transaction-based datastores as examples of this pattern getting (huge) traction in the real world. Great video, great topic!
3 года назад+10
I understand the pattern you want to show and I quite liked the 2 part series. Alas, I can not get pass the fact that the example program breaks an important accounting rule: the ledger can only be appended to. Future (not yet booked/processed) transactions can be cancelled, but it should be impossible to remove transactions from the ledger (after they are booked/processed). The closest thing to undoing a transaction is to add a transaction reversal to the ledger.
you are a game changer. thank you for this video that responds exactly to my problem. Design pattern, when you are a junior are very helpfull to avoird reinventing the wheel (translation of a french expression, don't know if it means something in english sorry). And typing Protocol is crazy, i dind't know it !! Thank you Arjan
Mind truly and completely blown. If I had to use one word to describe transaction-based design, it's "elegant". Now I'm thinking how to apply this to game logic / game physics engine type stuff, haha.
I would like to see you write tests for this. For me the most difficult part of any project is to figure out how to write tests. I try to write tests, because I want to make sure that program works correctly, but how to make the test not brake when something small changes.
One game changing trick is to write the test first because then you haven't written code that is hard to test. By having that test written first, you can't write code that is hard to test, because it won't make your tests pass. TDD really is that impactful. Over time (and with refactoring) the class you are testing often becomes a Facade object, so you can change the structure behind the Facade as much as you like, your unittests only test the API of the Facade and your tests don't break.
Really nice video! I think one simple thing to add would be a DAG representation of the transaction instead of a simple deletion of the current branch, in order to have a full overview of transactions history.
Typically you want to use a combination of both, in order to balance performance with granularity/composability. So define a standard milestone, such as midnight, or UTC midnight on the first of the month, etc, where you aggregate your previous balance with your list of recent transactions, then you move your new baseline time marker to that milestone timestamp. Then, the balance will always be your most recent milestone balance plus the sum of all transactions that have occurred since that milestone. [EDIT] Ah yep you mentioned this at the end. Hehe well done.
I imagine you could serialize the transactions list and store it in whatever, JSON for example. You would then try to load the JSON when you open the application so you keep your non destructive changes and persist them.
I really love your videos! Can you do a video on logging. Like in a large, production code base, using the logging module as it was intended. not a simple basic config. Thank you very much for all the high quality content.
It is basically the same way as how git works. It stores the changes instead of the final codes for each commit. Thanks for showing a nice way to achieve such a functionality!
Can we get a video about database patterns? In you videos you usually ignore the getting, creating, saving objects to a database but I’d be interested in how you do it. Does the object know how to get itself? Is there a class that acts as a repository? Cheers, N
Hey @ArjanCodes! Thank you very much for this really helpfully series. It is really well made and good to understand. I had one question at the end: Does it make sense to call `bank.clear_cache()` in `controller.compute_balances`? So we do not accidentally forget to call it. What would be the cons of this approach?
Is there a more “pythonic” way of pointing in the transaction history, and is it worth using? I started thinking about it when I saw the index=0 initialization and thought about undo-ing. I know python has iterators and generators. I’m not sure how they could be used outside of the context of looping.
Q: which object would be responsible for making sure clear_cache is called before every compute_balances? Or maybe more generally, how to make sure a transaction is not executed twice?
It seems to be quite difficult to see whether the transaction is invalid and need to inform the user or not. While if you store the state and immediately execute it, users receive feedback faster.
is there any advantage to doing "del self.ledger[self.current:]" as opposed to "self.ledger = self.ledger[:self.current]" ? I usually just slice the array as i "believe" it would be faster than using del and avoids any indexing errors
I don’t see any particular advantage, except perhaps that with del you’re making clear that you’re deleting things, whereas with the assignment that’s not immediately evident.
The slice approach creates a new list that is a shallow copy of the previous list, and reassigns self.ledger to point to the new list. This can have memory usage disadvantages, especially if any other variables at a higher scope are pointing to the previous list. The del approach just modifies the current list directly in-place.
I am curious on 1 thing So in real life, banks also recompute the whole transactions list? I mean when the number of transactions per day increases and after a couple days, the list could be huge. How do ppl deal with that? Mixing state and transactions so after a period of time they store the state and use it as initial to perform transaction?
I think banks actually do it in batches. I can imagine that there is a daily balance computation that then also fixes the balance, so you only need to keep track of transactions in the current day. But I don’t work at a bank, so I’m not sure.
That’s actually the whole idea of protocols in Python: the duck typing system takes care of checking whether two objects are of the same type, so the inheritance relationship is not needed.
In a sense, SVNs such as git, work using the principles of the command pattern. I know, that's a very simplistic explanation, but that should help you understand.
The main problem with the approach in this video is that you wouldn't execute transactions later. They would have already executed. Imagine if banks (and merchants) only moved money when you told them to. The world would be a mess.
This is not true, no one prevents you from adding a synchronize call to address this. You can even imagine scheduling a async synchronize call that would run at a predefined time intervalls
This implementation will break if another transaction is executed between a register and an undo. It was safer as it was, I guess. Also, no auditor would accept this kind of implementation as you know longer can follow exactly what happened, but then again You said that your not in banking.
💡 Here's my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
Amazing video! Learnt so much here about Command pattern and also how to approach the same problem from different angles!
Wow! Thank you so much Pinaka Dhara! ❤️
I'm glad you're changing the model to center transactions as the source of truth, because there was a problem with your last implementation that wasn't addressed.
If you tried to do a transfer from an account with insufficient funds, the undo step would remove funds from the other account while it had never received them in the first place, because the exception of insufficient funds was not handled. This removes that ambiguity.
i mean, obviously, those are just examples. if done properly both could work but history based seems better yeah
Nice Arjan, great content, and so much clarity and calm to explain. The transaction approach in this design seems very powerfull and yet simple. Thank you!
I did work on the implementation of a 3D based python software in the past and coincidentally I recently throught to add this type of feature to it. If you are wondering, I work in the metrology (spatial, mainly) & alignment group of a synchrotron laboratory here in Brazil, with lots of 3D challenges.
Cheers!
Thanks Rodrigo, glad to hear you liked it and thanks for sharing your background!
Another great video from ArjanCodes !
I have used something similar to the transactions, event sourcing.
Thank you! I didn't know about the transaction based way of thinking and it's beautiful! I will try to keep that in mind in my next projects!
Arjan, amazing work and thank you for these videos, they are fantastic. I was wondering if you can add some flowcharts or diagrams with the design patterns to have a bigger picture at the same time of how they flow and work. Thank you again for these really high quality tutorials.
I'd like to add a vote for Event Sourcing in a future video. I've been looking into Event Sourcing and Kafka lately and from what I understand so far, Event Sourcing with Kafka could help address the ledger deletion issue as mentioned in the comments. So it would be great to see this or a similar/extended example implemented with Event Sourcing. Thanks for covering the Command Pattern so thoroughly!
Hi, Arjan! Nice to see you again!
I'm very surprised you didn't mention Redux and other immutable/transaction-based datastores as examples of this pattern getting (huge) traction in the real world. Great video, great topic!
I understand the pattern you want to show and I quite liked the 2 part series.
Alas, I can not get pass the fact that the example program breaks an important accounting rule: the ledger can only be appended to. Future (not yet booked/processed) transactions can be cancelled, but it should be impossible to remove transactions from the ledger (after they are booked/processed). The closest thing to undoing a transaction is to add a transaction reversal to the ledger.
It’s a good thing I’m not an accountant, haha.
you are a game changer. thank you for this video that responds exactly to my problem. Design pattern, when you are a junior are very helpfull to avoird reinventing the wheel (translation of a french expression, don't know if it means something in english sorry). And typing Protocol is crazy, i dind't know it !! Thank you Arjan
Mind truly and completely blown. If I had to use one word to describe transaction-based design, it's "elegant". Now I'm thinking how to apply this to game logic / game physics engine type stuff, haha.
SOLID. Appreciate it!
Thank you!
Great videos! I am really grateful for all your work. I have learned so much since I've got to know your channel! Kudos!
Glad to hear that the videos are helping you, Philipe!
I would like to see you write tests for this.
For me the most difficult part of any project is to figure out how to write tests.
I try to write tests, because I want to make sure that program works correctly, but how to make the test not brake when something small changes.
Good suggestion, thanks!
One game changing trick is to write the test first because then you haven't written code that is hard to test. By having that test written first, you can't write code that is hard to test, because it won't make your tests pass. TDD really is that impactful.
Over time (and with refactoring) the class you are testing often becomes a Facade object, so you can change the structure behind the Facade as much as you like, your unittests only test the API of the Facade and your tests don't break.
Thank you for your videos, I find them very helpful. Pls make a video on threads, async and await features sometime.
Really nice video! I think one simple thing to add would be a DAG representation of the transaction instead of a simple deletion of the current branch, in order to have a full overview of transactions history.
really nice ty !
for the visual tips, you can remove you circle blur and put or not a border
Thanks Adrien!
Typically you want to use a combination of both, in order to balance performance with granularity/composability. So define a standard milestone, such as midnight, or UTC midnight on the first of the month, etc, where you aggregate your previous balance with your list of recent transactions, then you move your new baseline time marker to that milestone timestamp. Then, the balance will always be your most recent milestone balance plus the sum of all transactions that have occurred since that milestone.
[EDIT] Ah yep you mentioned this at the end. Hehe well done.
Thank you!
You're welcome!
EGGCELENT. Another banger!
Thanks for great video! Would be great if you could explain event sourcing as well
Thanks and noted!
That was so good! What would be the best way to record changes to disk and make it persistent?
Probably SQLite
I imagine you could serialize the transactions list and store it in whatever, JSON for example. You would then try to load the JSON when you open the application so you keep your non destructive changes and persist them.
I really love your videos! Can you do a video on logging. Like in a large, production code base, using the logging module as it was intended. not a simple basic config. Thank you very much for all the high quality content.
Good suggestion, thank you!
It is basically the same way as how git works. It stores the changes instead of the final codes for each commit. Thanks for showing a nice way to achieve such a functionality!
@Terrence-Monroe: Brannon Yes you are right. Git indeed stores snapshots instead of changes. Thanks for pointing out the differences !
Git in essence is an event-sourced application. Commands are not events.
Can we get a video about database patterns? In you videos you usually ignore the getting, creating, saving objects to a database but I’d be interested in how you do it. Does the object know how to get itself? Is there a class that acts as a repository? Cheers, N
Yes, I’m certainly going to cover databases in the future.
@@ArjanCodes Thank you, forgot to say - love your videos. I've watched almost all of them :)
Hey @ArjanCodes! Thank you very much for this really helpfully series. It is really well made and good to understand.
I had one question at the end: Does it make sense to call `bank.clear_cache()` in `controller.compute_balances`? So we do not accidentally forget to call it. What would be the cons of this approach?
Is there a more “pythonic” way of pointing in the transaction history, and is it worth using? I started thinking about it when I saw the index=0 initialization and thought about undo-ing. I know python has iterators and generators. I’m not sure how they could be used outside of the context of looping.
Interesting. I'll give this some thought.
Love it! U so smart... XD
Q: which object would be responsible for making sure clear_cache is called before every compute_balances? Or maybe more generally, how to make sure a transaction is not executed twice?
It seems to be quite difficult to see whether the transaction is invalid and need to inform the user or not. While if you store the state and immediately execute it, users receive feedback faster.
is there any advantage to doing "del self.ledger[self.current:]" as opposed to "self.ledger = self.ledger[:self.current]" ?
I usually just slice the array as i "believe" it would be faster than using del and avoids any indexing errors
I don’t see any particular advantage, except perhaps that with del you’re making clear that you’re deleting things, whereas with the assignment that’s not immediately evident.
The slice approach creates a new list that is a shallow copy of the previous list, and reassigns self.ledger to point to the new list. This can have memory usage disadvantages, especially if any other variables at a higher scope are pointing to the previous list. The del approach just modifies the current list directly in-place.
I am curious on 1 thing
So in real life, banks also recompute the whole transactions list?
I mean when the number of transactions per day increases and after a couple days, the list could be huge. How do ppl deal with that?
Mixing state and transactions so after a period of time they store the state and use it as initial to perform transaction?
I think banks actually do it in batches. I can imagine that there is a daily balance computation that then also fixes the balance, so you only need to keep track of transactions in the current day. But I don’t work at a bank, so I’m not sure.
A simple question, why there is no inheritance of Deposit,Withdraw from Transaction Protocol? Is it a typing miss..?
That’s actually the whole idea of protocols in Python: the duck typing system takes care of checking whether two objects are of the same type, so the inheritance relationship is not needed.
In a sense, SVNs such as git, work using the principles of the command pattern. I know, that's a very simplistic explanation, but that should help you understand.
The main problem with the approach in this video is that you wouldn't execute transactions later. They would have already executed. Imagine if banks (and merchants) only moved money when you told them to. The world would be a mess.
This is not true, no one prevents you from adding a synchronize call to address this. You can even imagine scheduling a async synchronize call that would run at a predefined time intervalls
This implementation will break if another transaction is executed between a register and an undo. It was safer as it was, I guess.
Also, no auditor would accept this kind of implementation as you know longer can follow exactly what happened, but then again You said that your not in banking.
it's just an example, he didn't handle all errors but could