When I started making my own microservices to play around and learn the good/bad practices, I found that focusing on the entities will absolutely lead to a distributed CRUD system. This is a good video for those who haven't yet learned the hard way.
The road leading to Entities and CRUD is thinking about your software as just data that you are just doing operations on. It is easy to fall into this being a technical person who writes the code. You need to learn to zoom out and to look at what you are doing at least partially from the users perspective - so thinking about UX. And then to learn about boundaries and that although concepts are overlapping you don't need to represent or store them as one just because of inheritance. And in particular not in the same boundary. Treat each boundary as you would and external system.
I have a massive issue with this I think. Usually when I think about free-time projects the first thing I do is find logical entities and think about how I'm going to store those in the database. Even when I think about the UX I end up with those entities. Unfortunately the video didn't really help with that :/
@@DomskiPlays It is not like I don't think about entities. I'm just that secure about how I would implement stuff, I have found a structure, that I can care less about the technical details and focus more on modeling the domain and behavior - that enables me to make progress in a very short amount of time.
Well, the fact the original poster is asking for help because they can't easily separate their domain models, is the best indication that Students and Teachers aren't obvious "microservices". I think this is where a lot of the struggle happens when trying DDD. But to be fair, I can't imagine anyone not struggling with the depth of reasoning required to properly model a problem domain. The solution is to avoid being too prescriptive, but also try to leave enough room for change. At the end of the day, our responsibility is to solve problems, not to find solutions.
I should use this as an example and make up my own to define the boundaries. As an example the concept of a "student" is likely an entity in many different logical boundaries.
I think this video is a very good resource to understand the fundamentals of DDD. In my opinion, focusing too much on the transactions themselves leads to loss of context in the long run and very very hard queries to the database. Sticking to the fundamental entities, without too much inheritance (I prefer none at all, prefer composition and aggregation instead) is better in the long run.
Exactly! I am focusing on functionalities, instead of entities, and the way functionalities interact with one another. We're building a cloud SaaS software (monolith for now) that is built from modules (vertical slicing) and will later be split into more services (micro or macro...who knows). We'll need to determine which functionalities to split and that's why good understanding of these interactions is so important.
Somewhat agree. While it is true focusing on entities will basically create a CRUD based system, is not always true it is hard to change, especially if you use things like EF Core, EF Core Power Tools for Reverse Engineering, and C# Source Generators, that way you can pretty much automate most changes, and focus on UI, Business Rules and Automated Testing.
I agree. You can see this perfectly in Unity's DOTS/ECS (Data Oriented Technology Stack/Entity Component System) system. It focuses so much on programming around the data, almost nobody uses it. There are a few use-cases but you could do a custom solution if needed using SIMD... The discussion is quite a bit more in-depth than this, but in general I agree with this for most use-cases
"Almost nobody" uses dots because of breaking changes all over the place before the production-ready release rolled out. It's stable and production-ready for just a couple of months, give it some time. It's fully compatible with gameobjects and really helps to solve performance issues. Why creating your own wheel when there's already production-ready with editor support
I chuckled at "the microservices are fairly obvious" followed by a list of nouns. I wish UML use case modeling had caught on. The use case analysis process solves problems like this. It's good method to achieve DDD.
I worked for two companies early on in my career who designed having a multi purpose transactions table in oracle, catering for different transaction types. There were full blown monoliths. Can see now how pub sub would be so much better around actions and behaviours instead of finding the common denominator.
There are always a ton of ways to do different stuff, and one needs to know a lot about the system, the business, the usage, the purpose, the life-span, etc. before one can make the right design decisions. Microservices are invented to fix scaling issues in larger systems/environments with many development teams. Don't use them if you don't need them. There are no silver bullets out there, just different tradeoffs. The solutions when writing software for a coffee machine is different than a military simulator, and is again different fixing an issue in a procurement module for a 40 year old system.
I think the challenge is when we straddle the boundaries, especially after the fact. One day its "we want the top 3 vendors by market share in each country, but only if they have at least 5 customers with contracts over 10 million and > 100 employees. Oh, and it needs to be paged. What would be an 'obvious' query in a monolith can be incredibly difficult in msa. You may well end up having to do some kind of event-based thing with versioned data just to keep a real time 'projection' of the source data... so you can do this 'trivial' query against it. And that assuming optimistic locking is good enough.
Agree. Depending on the size of the system and how you choose to partition data between boundaries. If all logical boundaries are hosted within the same physical database, a logical boundary could produce a "view" of that data as a contract for reporting purposes where you can then span multiple boundaries for your report/query. The point is you're creating a "view" that's specific for external usage and not internal.
Fully agree on the point that you should define your system around what the functionality is. If I'm nit picky I don't agree on the title of the video because the outcome of focusing on the functionality are still Entities. I'd have rather chosen a wording like "Focusing on "Data-driven Entities" leads nowhere good."
Your example made me wonder. When you have reporting, how do you tend to handle that? I would imagine each boundary would have some level of reporting, but there might also be boundaries that are practically 100% reporting-based.
Thank you for this wonderful content, as always. Your videos has clearly made me think about any application based on its features and capabilities with its clear boundaries. Not perfect though. We had a UI/UX designer prototype a green field project which is a mobile application which was purely CRUD, the capabilities were purely inferred on the client. I had to step in having fully understood the general functionality of the app, we pointed out the features and the various capabilities of the app, it was not easy to accept at first, but after a while moving forward, we are enjoying the process and where it's leading us to. Keep up the good work 👍
CRUD is wrong both in standard entities as well as REST. It forces the business logic up into the client instead of on the server. We are using a ton more gRPC these days for that very reason. Also, I've done CQRS at scale in fortune 5 as chief arch and honestly we had to get rid of it. The problem was that every system that used the events had to also write interpreters for the events that were running through the system. Times 500 systems in our enterprise and it just wasn't worth it so we moved back to event based microservices.
In my experience you can have many non-crud commands for a entity, and they all result in a EntityXyzUpdatedEvent being published. The handlers can figure out how to react based on what changed, or do nothing. This way, the command does not matter and the event handlers can not be skipped accidentally, and there is no duplication in the event handlers. If the command does matter then that information could be added to the event, but it is more rare i think.
I'd say that strongly depends on A: the size of your entity/aggregate (in terms of behavior supported), and B: The overall complexity of your system. In a large/complex system, "crud-events" often leads to dependency hell in order to have enough context to "reverse engineer meaning" in order to decide if/how a generic event should be handled. But for "dumb data" (like documents) I agree - just remember that what might be "dumb data" for one "logical service" can be a deep problem domain in another...
@@juliancyanid Lets say you have X specific events and Y handlers per event. What if you change the entity? Or add a new command/event? Now you need to inspect all X * Y handlers, or add Y new event handlers. It becomes a 'Trust me bro, i thought this through' thing. This sounds like a dependency hell to me. With a EntityXyzUpdatedEvent there is a 1 * Y relationship. Easier to reason about when things change.
I'm not trying to convince anyone, just voicing my own experience. "Event Carried State Transfer" like you suggest here can be great, especially for async readmodels or other needs with less specific business logic in handlers. I've also seen it cause a lot of overhead and coupling because intent was lost, and needed to be deduced retroactively - and such deduction breaks down for the same reasons you mention (someone changed behavior somewhere) - it's just invicible/implicit for consumers because the only event is "X updated"... Explicit events makes stream/topic boundaries both harder and more important to design properly - everything is a tradeoff 🤷♂️ Like most other things in our trade, "it depends" 🫣
@@CodeOpinion Huh, maybe the f/stop (aperture) setting of your camera? That must be it, because if it's set to be automatic, if you add extra light the aperture will become smaller, leading (usually) to sharper pictures and less background blur
This is a great approach, however it sometimes feels this contradicts Restful API design. What are you thoughts on designing restful API's around behavior? For example, Rest exposes resources. so a CRUD operation for a product. However, how would you design an endpoint to handle a "price adjustment" that is within the product resource?
Name it as what it is. Bounded context name, the fact it is a transaction and the transaction (command) name. e.g. POST /api/stock/transactions/adjustprice And then have it return something with the new version number do you know if you’re getting a stale read from your read model. The important thing is that transactions can span more than one single entity and should only be constrained by your domain model boundaries (I.e. your aggregate root) and the transactional capabilities of your primary DB. I see a lot of anecdotal cases of people getting into strife because they’re too focused on individual entities and break things up so much they end up throwing away basic and important capabilities like atomic transactions and don’t compensate for that properly either so fail to achieve guaranteed eventual consistency too. GraphQL principles fit nicely since it splits query and command from the outset and guides people towards thinking about what a does rather than just what it is. If that makes sense.
Here is an idea for a video. If you would pick say 3 github projects that you think have undestood the task and executed it well. Not as templates mind you but more like "This is a Project that gets it right". Just a thought :).
I'm pretty sure that many faults in popularizing entity-based microservices lie on Microsoft with their guidelines that openly say that microservice should be entity-oriented.
Well they should be "entity" oriented however the problem is people think of only data and not behavior. Or think of an entity as a singular thing. When you focus on behaviors you may realize that what you think of as a single entity is really a concept that belongs in many different logical boundaries, based on capabilities it provides in those boundaries.
@@CodeOpinion That is why I prefer to split my microservices on a use-case basis. Microservice is, first of all, an independent scalability unit, and the finest granular portion of scalability is use-case so it naturally fits one another and brings additional advantages like independent deployment and so on.
You should define "Entity" first. For example in the Clean architecture, an entity is a bunch of critical business rules, more general than the use cases which are specific to the app, operating on critical business data. They need not contain that data, and they don't care one bit about how it is stored. Modeling entities based on how the data they operate on is stored is indeed a mistake
I was to get insane in a job where the PM called all features simple and can be done in few hours because are just a CRUD. Not all is read and write to a database, generic code make some things feel faster until you need to refactor the mess.
I find it rather misleading to refer to what is very clearly an Object from OOP as people focusing on "Entities" - due to the ECS (Entity, Component, System) connotations that has. The example you show is a full class of bundled data and functionality - a hallmark of OOP. In ECS an "Entity" is an index or ID, nothing else. Data types do not contain methods as this rapidly pollutes cache space (among other issues) Systems are where the functions live, and contain minimal to no persistent state data of their own. While I don't exactly disagree with the subscriber or event architecture you're talking about, as I personally like using delegates in my own game code, I feel that "Objects" would have been a far more appropriate term for the title of this video - as an ECS entity was not shown at any time, and generally that is what the word Entity would invoke.
When I started making my own microservices to play around and learn the good/bad practices, I found that focusing on the entities will absolutely lead to a distributed CRUD system. This is a good video for those who haven't yet learned the hard way.
The road leading to Entities and CRUD is thinking about your software as just data that you are just doing operations on. It is easy to fall into this being a technical person who writes the code. You need to learn to zoom out and to look at what you are doing at least partially from the users perspective - so thinking about UX.
And then to learn about boundaries and that although concepts are overlapping you don't need to represent or store them as one just because of inheritance. And in particular not in the same boundary. Treat each boundary as you would and external system.
I have a massive issue with this I think. Usually when I think about free-time projects the first thing I do is find logical entities and think about how I'm going to store those in the database. Even when I think about the UX I end up with those entities. Unfortunately the video didn't really help with that :/
@@DomskiPlays It is not like I don't think about entities. I'm just that secure about how I would implement stuff, I have found a structure, that I can care less about the technical details and focus more on modeling the domain and behavior - that enables me to make progress in a very short amount of time.
Well, the fact the original poster is asking for help because they can't easily separate their domain models, is the best indication that Students and Teachers aren't obvious "microservices". I think this is where a lot of the struggle happens when trying DDD. But to be fair, I can't imagine anyone not struggling with the depth of reasoning required to properly model a problem domain. The solution is to avoid being too prescriptive, but also try to leave enough room for change. At the end of the day, our responsibility is to solve problems, not to find solutions.
I should use this as an example and make up my own to define the boundaries. As an example the concept of a "student" is likely an entity in many different logical boundaries.
I think this video is a very good resource to understand the fundamentals of DDD. In my opinion, focusing too much on the transactions themselves leads to loss of context in the long run and very very hard queries to the database. Sticking to the fundamental entities, without too much inheritance (I prefer none at all, prefer composition and aggregation instead) is better in the long run.
Exactly! I am focusing on functionalities, instead of entities, and the way functionalities interact with one another. We're building a cloud SaaS software (monolith for now) that is built from modules (vertical slicing) and will later be split into more services (micro or macro...who knows). We'll need to determine which functionalities to split and that's why good understanding of these interactions is so important.
Somewhat agree. While it is true focusing on entities will basically create a CRUD based system, is not always true it is hard to change, especially if you use things like EF Core, EF Core Power Tools for Reverse Engineering, and C# Source Generators, that way you can pretty much automate most changes, and focus on UI, Business Rules and Automated Testing.
I agree.
You can see this perfectly in Unity's DOTS/ECS (Data Oriented Technology Stack/Entity Component System) system. It focuses so much on programming around the data, almost nobody uses it. There are a few use-cases but you could do a custom solution if needed using SIMD...
The discussion is quite a bit more in-depth than this, but in general I agree with this for most use-cases
"Almost nobody" uses dots because of breaking changes all over the place before the production-ready release rolled out. It's stable and production-ready for just a couple of months, give it some time. It's fully compatible with gameobjects and really helps to solve performance issues. Why creating your own wheel when there's already production-ready with editor support
I chuckled at "the microservices are fairly obvious" followed by a list of nouns. I wish UML use case modeling had caught on. The use case analysis process solves problems like this. It's good method to achieve DDD.
I worked for two companies early on in my career who designed having a multi purpose transactions table in oracle, catering for different transaction types. There were full blown monoliths. Can see now how pub sub would be so much better around actions and behaviours instead of finding the common denominator.
Finally someone said the quite part out load.
The issue presented here is very similar to code over-abstraction. It's domain modeling over-abstracting and unnecessary generalisation.
I really enjoy learning from your videos. You have tremendous knowledge and experience and its good to have an opinion :)
There are always a ton of ways to do different stuff, and one needs to know a lot about the system, the business, the usage, the purpose, the life-span, etc. before one can make the right design decisions.
Microservices are invented to fix scaling issues in larger systems/environments with many development teams. Don't use them if you don't need them.
There are no silver bullets out there, just different tradeoffs. The solutions when writing software for a coffee machine is different than a military simulator, and is again different fixing an issue in a procurement module for a 40 year old system.
Agree'd. Context is King!
I think the challenge is when we straddle the boundaries, especially after the fact. One day its "we want the top 3 vendors by market share in each country, but only if they have at least 5 customers with contracts over 10 million and > 100 employees. Oh, and it needs to be paged.
What would be an 'obvious' query in a monolith can be incredibly difficult in msa. You may well end up having to do some kind of event-based thing with versioned data just to keep a real time 'projection' of the source data... so you can do this 'trivial' query against it. And that assuming optimistic locking is good enough.
Agree. Depending on the size of the system and how you choose to partition data between boundaries. If all logical boundaries are hosted within the same physical database, a logical boundary could produce a "view" of that data as a contract for reporting purposes where you can then span multiple boundaries for your report/query. The point is you're creating a "view" that's specific for external usage and not internal.
Do you think having a read model would be a good solution?
Fully agree on the point that you should define your system around what the functionality is.
If I'm nit picky I don't agree on the title of the video because the outcome of focusing on the functionality are still Entities. I'd have rather chosen a wording like "Focusing on "Data-driven Entities" leads nowhere good."
This is very useful and inspiring video. But it’s too short. I really want to see more use case or more details. Anyway, great works!!!
wayland yutani corp. i see you 🔥
Oh wow, never though I'd see someone point the finger at data and entities for getting out of hand right before pitching event sourcing and CQRS.
Your example made me wonder. When you have reporting, how do you tend to handle that? I would imagine each boundary would have some level of reporting, but there might also be boundaries that are practically 100% reporting-based.
Thank you for this wonderful content, as always. Your videos has clearly made me think about any application based on its features and capabilities with its clear boundaries. Not perfect though.
We had a UI/UX designer prototype a green field project which is a mobile application which was purely CRUD, the capabilities were purely inferred on the client. I had to step in having fully understood the general functionality of the app, we pointed out the features and the various capabilities of the app, it was not easy to accept at first, but after a while moving forward, we are enjoying the process and where it's leading us to. Keep up the good work 👍
CRUD is wrong both in standard entities as well as REST. It forces the business logic up into the client instead of on the server. We are using a ton more gRPC these days for that very reason. Also, I've done CQRS at scale in fortune 5 as chief arch and honestly we had to get rid of it. The problem was that every system that used the events had to also write interpreters for the events that were running through the system. Times 500 systems in our enterprise and it just wasn't worth it so we moved back to event based microservices.
What did you move back to?
@@ejazahmed4609 not sure how it got cut off but I edited my comment to include what we did. sorry about that
In my experience you can have many non-crud commands for a entity, and they all result in a EntityXyzUpdatedEvent being published. The handlers can figure out how to react based on what changed, or do nothing. This way, the command does not matter and the event handlers can not be skipped accidentally, and there is no duplication in the event handlers. If the command does matter then that information could be added to the event, but it is more rare i think.
I'd say that strongly depends on A: the size of your entity/aggregate (in terms of behavior supported), and B: The overall complexity of your system.
In a large/complex system, "crud-events" often leads to dependency hell in order to have enough context to "reverse engineer meaning" in order to decide if/how a generic event should be handled.
But for "dumb data" (like documents) I agree - just remember that what might be "dumb data" for one "logical service" can be a deep problem domain in another...
@@juliancyanid Lets say you have X specific events and Y handlers per event. What if you change the entity? Or add a new command/event? Now you need to inspect all X * Y handlers, or add Y new event handlers. It becomes a 'Trust me bro, i thought this through' thing. This sounds like a dependency hell to me. With a EntityXyzUpdatedEvent there is a 1 * Y relationship. Easier to reason about when things change.
I'm not trying to convince anyone, just voicing my own experience. "Event Carried State Transfer" like you suggest here can be great, especially for async readmodels or other needs with less specific business logic in handlers.
I've also seen it cause a lot of overhead and coupling because intent was lost, and needed to be deduced retroactively - and such deduction breaks down for the same reasons you mention (someone changed behavior somewhere) - it's just invicible/implicit for consumers because the only event is "X updated"...
Explicit events makes stream/topic boundaries both harder and more important to design properly - everything is a tradeoff 🤷♂️
Like most other things in our trade, "it depends" 🫣
Event driven architecture is the silver bullet for most enterprise software
This sounds like CRUD with extra steps. I'm not convinced that anything but large, complex systems would be better off using this approach.
I agree, that's mainly the space is large complex systems.
Also the video looks way sharper now than the previous ones. What did you change? :)
I agree, and I have no idea why :) Same camera, same setup, same video rendering. My only guess was I might have changed my lighting a bit.
@@CodeOpinion Huh, maybe the f/stop (aperture) setting of your camera? That must be it, because if it's set to be automatic, if you add extra light the aperture will become smaller, leading (usually) to sharper pictures and less background blur
Yellow shirt
This is a great approach, however it sometimes feels this contradicts Restful API design. What are you thoughts on designing restful API's around behavior?
For example, Rest exposes resources. so a CRUD operation for a product. However, how would you design an endpoint to handle a "price adjustment" that is within the product resource?
Name it as what it is. Bounded context name, the fact it is a transaction and the transaction (command) name. e.g. POST /api/stock/transactions/adjustprice
And then have it return something with the new version number do you know if you’re getting a stale read from your read model.
The important thing is that transactions can span more than one single entity and should only be constrained by your domain model boundaries (I.e. your aggregate root) and the transactional capabilities of your primary DB. I see a lot of anecdotal cases of people getting into strife because they’re too focused on individual entities and break things up so much they end up throwing away basic and important capabilities like atomic transactions and don’t compensate for that properly either so fail to achieve guaranteed eventual consistency too.
GraphQL principles fit nicely since it splits query and command from the outset and guides people towards thinking about what a does rather than just what it is. If that makes sense.
Restful sucks. Thinking about resources (what is a resource and how) defeats the purpose of what should you do with them, the processes.
Here is an idea for a video. If you would pick say 3 github projects that you think have undestood the task and executed it well. Not as templates mind you but more like "This is a Project that gets it right". Just a thought :).
I'm pretty sure that many faults in popularizing entity-based microservices lie on Microsoft with their guidelines that openly say that microservice should be entity-oriented.
Well they should be "entity" oriented however the problem is people think of only data and not behavior. Or think of an entity as a singular thing. When you focus on behaviors you may realize that what you think of as a single entity is really a concept that belongs in many different logical boundaries, based on capabilities it provides in those boundaries.
@@CodeOpinion That is why I prefer to split my microservices on a use-case basis. Microservice is, first of all, an independent scalability unit, and the finest granular portion of scalability is use-case so it naturally fits one another and brings additional advantages like independent deployment and so on.
Not using Microservices will solve the problem ✅
Whats the link to the sample repository?
github.com/baratgabor/MyWarehouse
It looks like you reinvented bounded context from DDD
Not reinventing, that's what I'm describing. 😂
You should define "Entity" first. For example in the Clean architecture, an entity is a bunch of critical business rules, more general than the use cases which are specific to the app, operating on critical business data. They need not contain that data, and they don't care one bit about how it is stored. Modeling entities based on how the data they operate on is stored is indeed a mistake
An Entity as a DB Table.
I was to get insane in a job where the PM called all features simple and can be done in few hours because are just a CRUD.
Not all is read and write to a database, generic code make some things feel faster until you need to refactor the mess.
Generic code in business applications over time is often death by a 1000 papercuts
Came here just to give a thumbs up. Don’t even need to watch the video, the title says it all.
I find it rather misleading to refer to what is very clearly an Object from OOP as people focusing on "Entities" - due to the ECS (Entity, Component, System) connotations that has.
The example you show is a full class of bundled data and functionality - a hallmark of OOP.
In ECS an "Entity" is an index or ID, nothing else.
Data types do not contain methods as this rapidly pollutes cache space (among other issues)
Systems are where the functions live, and contain minimal to no persistent state data of their own.
While I don't exactly disagree with the subscriber or event architecture you're talking about, as I personally like using delegates in my own game code, I feel that "Objects" would have
been a far more appropriate term for the title of this video - as an ECS entity was not shown at any time, and generally that is what the word Entity would invoke.
ECS != DDD != OOP