In my experience, a set of parallel "model" and "entity" classes aren't even the worst of the problem. I'm a Java developer, but I expect this happens in C# code too: copying everything back to the ORM-managed objects breaks the automatic change-detection logic, you don't always need every field so someone reimplements lazy loading, someone else realizes that you need a way to specify what to load from the database so they reinvent a "criteria" mechanism, you've doubled your memory usage by having "model" and "entity" copies of everything, and someone who is unaware of the first- and second-level Hibernate caches adds another cache at the "model" layer. I've seen these issues across multiple projects. I've also seen projects that use JDBC to talk directly to the database instead of using an ORM, and you must have at least one person who knows JDBC and SQL really well for that to turn out okay. I think people generally understand that, but I believe that it's the same if you use an ORM. You still need at least one person who knows the ins and outs of SQL and the ORM really well, to make sure that you're not shooting yourself in the foot. So why bother? Because it can result in less code and better performance with less effort. To be clear, I don't believe that an ORM is always the right approach. But if you use an ORM, think of it as a commitment rather than an easy way out or it's going to come back to bite you.
Agree that any tool you're using blindly can/will end badly. The debate on ORM vs SQL i think is largely dependent on peoples good/bad experiences with both. For example, I generally never create graph/object hierarchy's with an ORM because it often gets used in a horrific way.
This has to be one of the most influential YT vids on SA I’ve watched in a while. Your take on passing the infrastructure to the domain is great and seems pragmatic. I will say though that using some micro ORMs like JOOQ for getting the aggregate in a handcrafted query and mapping that to the domain can also be great if you want to have a “cleaner” domain. I wouldn’t recommend that with an ORM though! Great vid! Keep it up
I also think that some people forget that actually, outside of a database, your domain does have persistence, and always will have. Whether you recorded transactions in a book before, give your clients a reference number, that’s an ID in itself, composite keys if you need to, I have used domain entities for a while now, I get them as valid as possible, and when I think about ‘mapping’ to the database, I see that as your setup (in my cases using fluent api on entitytypeconfiguration). I really enjoyed this video, we have a laugh a chap and I at work, he’s so against having the entity anywhere near the database and I’m such an advocate for it!
Yes, as you mentioned, IDs have various forms. You can have an external user facing ID (reference number) or some natural ID. But you can also have a purely technical ID that end users don't care about. At the end of the day it's really just about having focused models and realize how you're coupled to your data model and in your context to the degree that's acceptable to be coupled to it.
Using ORM is the problem, or at least the reason for this whole confusion. Say you use Dapper and you write the SQL yourself. If you persist - you just give your domain model to the repository, its job is to know how to persist it, the application layer doesn't care where it's persisted, which table/tables are involved etc. If you query - you define the view model, with types, formatting, property names exactly how you need them for the case, the repository's responsibility is to prepare it for you, the application layer doesn't care how. And you always have great flexibility, total control over the SQL. But you decided to use some opinionated, limited, magical ORM, because "writing SQL takes so much time".
Persistence Ignorance is one of those patterns that have caused much more problems than it ever solved. Adhering to REST is probably the worst example.
Domain models aren't the data model, but I would argue the identity of the object belongs to the domain. You can't expose meaningful behavior if you can't specify what model to operate on. In my experience, it's always been best to isolate the data models in such a way that you simply pass your domain models to that layer and let it be entirely responsible for mapping back and forth between whatever persistence backend you're using. This allows you to change that backend entirely independently of the rest of the system. If you're worried about mapping overhead, don't use an ORM on top of this.
Hello, thanks for your contents. I really appreciate your videos. However, I didnt understand the point here, when you advocate against the use of ID. I mean, by definition, all DDD books I have read says that the main characteristic of an entity is that it has an identity. Even further, Vaugh Vernon in DDD red book make use of and ID as a value object, in order to encapuslates ID details. Please, dont take me wrong, I really want to hear your toughts about my point of viewing. Your videos always give me good insights. Thanks again.
Exactly, if they don't need to be uniquely identified, they shouldn't be entities in the first place. They should be value objects or DTOs or whatever you want to call them.
He's not advocating against the use of IDs - he's making a distinction between Entity Framework data model, which are behaviorless data structures that do have database IDs in them, and the Domain Model, which does not consist of simple data structures, but of objects with behaviors - and in fact, could even just be a bunch of functions, if you're not doing the OO style. It's not a slightly different copy of the data model, instead, it takes in this data and does stuff to it, or delegates to lover level code where needed - but the point is, it controls the business logic flow. It's where the core business logic is encoded, it controls what happens on the high level. He's just saying that you don't need to replicate your data model there, and copy all the data and all the IDs - just figure use what's needed to fulfill the business logic, and design your domain model around that. Also, please note that Entity Framework entities (data model), Clean Architecture Entities (domain model), and DDD Entities are all different concepts. In DDD, a distinction is made between objects that represent things that can mutate state over time, but have a thread of identity that persists beyond the runtime (or possibly even the lifetime) of the application ("entities"), and objects where two instances with the same state are interchangeable because we only care about their value ("value objects"). It's not a statement about database IDs, it's a conceptual distinction. In the original DDD book, Aggregates are not required to reference other objects by ID, they just need to have a well-defined boundary that defines what's encapsulated behind the Aggregate Root, a surface level object that controls access to anything inside. Vaugh Vernon's approach uses IDs to demarcate that boundary, but it's just an approach (albeit a popular one). It's in no way the essence of DDD.
I'm not advocating against IDs. I'm suggesting that the data you care about in your domain model is what revolves around the behavior in your domain model.
I always find creating aggregates with event sourcing as a great example as it shows the only data you actually care about revolves around behaviors in that model. The point is you don't need one model to rule them all.
@@CodeOpinion Cool, yeah I saw your blog, I was reading some other posts meanwhile haha. I think you could go deeper on posts compared to videos, thanks for sharing!
I appreciate this video, but sorry I’m still confused. I wish there was a good real-life example out there that shows more than just the basic examples usually shown.
I may attempt to show something more complex in a video, my guess is I'd lose the audience immediately because having to explain the nuance of the domain and the reasons why the complexity exists isn't trival.
Feel like I am waiting for this videos for years 😂 though I hate that people create useless private setter (most of them are for non related properties) just to make domain entities work with EF, I still could not figure out where I should put those data models. I used to put them in Infras and use some magic from mapping lib to map p, now I got some confidence to just put it in Domain layers to make the mapping more simple 😂
Where I think people get this overblown is trying to go to far so that even the structure isn't understood by the domain model. If you don't want to couple your data model to your domain model, then yes you'd have to do some mapping between the two. But as I always say, the degree to which your coupling matters. If you're data model is concise and you need to change it's structure, how much of an issue is it that you need to adjust how you change the state within the domain model? I assume not much. If you have a monster data model that includes everything under the sun and your domain model is loaded with CRUD for everything, ya this is going to be an issue.
ID isn't really a persistence-level concern, it can be thought of as very much a domain construct. That was not a good example to make your point IMO, although the video was interesting.
I believe most of the pain comes from the fact that people map (public) properties to columns. In JPA for example you can choose between property and field access. If you use latter you can provide custom properties and behaviour, as your data is not visible to everyone. In EF the same should be achievable with backing fields.
@@CodeOpinion what do you mean by that? Navigating deep into the object tree and relying on lazy loading to fetch the requested items or something different? If so, the same rule applies: what you don’t make public, can‘t be navigated.
I think IDs are a persistence concern and Domain Logic is a core concern. When you want to uncouple lower level implementation details from higher level logic you should use an interface to invert the flow of control. The domain model should not have an id. Data models in the application layer should inherit from the domain model and implement an interface IEntity (covariance). Now the infrastructure knows nothing about the domain. Your contracts will protect your domain from being frigid.
I'm not disagreeing with the approach your suggesting, because in a given context, it can make sense. But at the end of the day we're talking about coupling and the degree to which you want to be coupled between those "layers". But the amount of convoluted indirection with little value because of unneeded abstractions kills me.
Many people have experienced what you described here. But there's no much solution to this problem. An ORM solve several problems, which would otherwise be difficult to solve. If you have a graph of objects it's difficult not to use an ORM.
@@CodeOpinion How do you do it instead? Do you have an example how you would work with navigations (in a world without them, assuming you use an id instead)?
I got real example how people has anemic models almost without any logic on it,. never changed persistence type or even vendor but paid full price with overhead of separate domain models with repository layer and mappings of recursive models. Thats a sad but true story. They tried to make everything in a "propper" way from the start.
I was about to comment something along these lines. The concept discussed in the video seems to be solved with anemic "domain" models (which in reality would be just data models) and a lot of repos, interfaces and services to work with them.
I've done videos about this and using transaction scripts with simple data models. The point of using a domain model, in the typical style of an aggregate is because you want a consistency boundary which is more difficult to achieve if you have a pile of trx scripts using the same underlying data model. If you don't have consistency concerns, then yes trx scripts and a data model works fine.
Well I don't now that C# crap 😄, but should be similar to JPA. The problem is that the metadata pollutes the domain model because we all want to use annotations these days. But the truth is that the mapping information doesn't belong to either side - so the old style where the mapping data was kept in a seperate xml file between the database and the domain model was architecturally a better solution.
It's similar idea of annotating when you choose to make your data model the domain model. The issue I have with mapping is trying to bend over backwards because of the ORM. You can do this with a more fluent API in C#/EF but the amount of mapping gymnastics or conceding to the ORM makes you wonder why even bother.
Derek, I can understand the line of thought of having separated Domain and Persistence models, even having ORMs in place, but what do you think about the additional overhead of "converting" in and out between domain and persistence classes? And about the decision of some developers of doing this using mapper tools?
If you have two separate models that you map between, my assumption is you're doing so because you don't want your domain model coupled to your data model. So you have mapping at the infrastructure/persistence level. That's my guess why people do it. My argument for not doing that and having your data model used inside your domain model, is yes you have coupling, but as I always say, the degree of the coupling matters. If you have a focused/narrow domain model and the data model within in changes its shape/structure, sure you'll need to update your domain model internals but since it's so focused, and inherently small, how much coupling do you really have? Also, none of your tests will change in their assertions. But if you have a large data model/structure, you likely also have a huge domain model over it. Which means you'll have to change a lot, which isn't great because you have a high degree of coupling. As always, coupling is what needs to be managed.
@@Rick104547 Well, in DDD an aggregate root, which is a consistency boundary, has an id and there is nothing wrong with it. I mean aggregate roots themselves are also about persistence
@@CodeOpinion Isn't it quite normal that we have persistence concerns in our domain models/aggregates? Aren't aggregates themselves about enforcing business rules, consistency boundaries and persistence?
Some prefer it, some don't. If you're writing SQL, often times this can involve emitting events so that you can translate it to some type of SQL statement you write. Ultimately, it's still mapping.
@@CodeOpinion the power of ORM is not in creating SQL queries but in change tracking and materialization. Domain Model uses encapsulation which usually needs to be hacked around when materializing and persisting data.
This video is so freaking good. A prime example why CodeOpinion is my undisputed favorite YT channel for software architecture and design.
Glad you enjoy them. Appreciate the support.
In my experience, a set of parallel "model" and "entity" classes aren't even the worst of the problem.
I'm a Java developer, but I expect this happens in C# code too: copying everything back to the ORM-managed objects breaks the automatic change-detection logic, you don't always need every field so someone reimplements lazy loading, someone else realizes that you need a way to specify what to load from the database so they reinvent a "criteria" mechanism, you've doubled your memory usage by having "model" and "entity" copies of everything, and someone who is unaware of the first- and second-level Hibernate caches adds another cache at the "model" layer. I've seen these issues across multiple projects.
I've also seen projects that use JDBC to talk directly to the database instead of using an ORM, and you must have at least one person who knows JDBC and SQL really well for that to turn out okay. I think people generally understand that, but I believe that it's the same if you use an ORM. You still need at least one person who knows the ins and outs of SQL and the ORM really well, to make sure that you're not shooting yourself in the foot. So why bother? Because it can result in less code and better performance with less effort.
To be clear, I don't believe that an ORM is always the right approach. But if you use an ORM, think of it as a commitment rather than an easy way out or it's going to come back to bite you.
Yes breaking the automatic ChangeTracker is something I've stumbled over many times as well and never really got around to a good solution.
Agree that any tool you're using blindly can/will end badly. The debate on ORM vs SQL i think is largely dependent on peoples good/bad experiences with both. For example, I generally never create graph/object hierarchy's with an ORM because it often gets used in a horrific way.
This has to be one of the most influential YT vids on SA I’ve watched in a while.
Your take on passing the infrastructure to the domain is great and seems pragmatic.
I will say though that using some micro ORMs like JOOQ for getting the aggregate in a handcrafted query and mapping that to the domain can also be great if you want to have a “cleaner” domain. I wouldn’t recommend that with an ORM though!
Great vid! Keep it up
Thanks for watching and the support!
I also think that some people forget that actually, outside of a database, your domain does have persistence, and always will have. Whether you recorded transactions in a book before, give your clients a reference number, that’s an ID in itself, composite keys if you need to, I have used domain entities for a while now, I get them as valid as possible, and when I think about ‘mapping’ to the database, I see that as your setup (in my cases using fluent api on entitytypeconfiguration). I really enjoyed this video, we have a laugh a chap and I at work, he’s so against having the entity anywhere near the database and I’m such an advocate for it!
Yes, as you mentioned, IDs have various forms. You can have an external user facing ID (reference number) or some natural ID. But you can also have a purely technical ID that end users don't care about. At the end of the day it's really just about having focused models and realize how you're coupled to your data model and in your context to the degree that's acceptable to be coupled to it.
Using ORM is the problem, or at least the reason for this whole confusion.
Say you use Dapper and you write the SQL yourself.
If you persist - you just give your domain model to the repository, its job is to know how to persist it, the application layer doesn't care where it's persisted, which table/tables are involved etc.
If you query - you define the view model, with types, formatting, property names exactly how you need them for the case, the repository's responsibility is to prepare it for you, the application layer doesn't care how.
And you always have great flexibility, total control over the SQL.
But you decided to use some opinionated, limited, magical ORM, because "writing SQL takes so much time".
Yes the ORM in most situations ultimately is the culprit as you need to live within it's guidelines on how you map, etc.
Persistence Ignorance is one of those patterns that have caused much more problems than it ever solved. Adhering to REST is probably the worst example.
Ya there's a lot of dogma unfortunately. "REST" is now "CRUD over HTTP with JSON"
Domain models aren't the data model, but I would argue the identity of the object belongs to the domain. You can't expose meaningful behavior if you can't specify what model to operate on. In my experience, it's always been best to isolate the data models in such a way that you simply pass your domain models to that layer and let it be entirely responsible for mapping back and forth between whatever persistence backend you're using. This allows you to change that backend entirely independently of the rest of the system. If you're worried about mapping overhead, don't use an ORM on top of this.
Identity doesn't mean it's persistence ID. Example: Invoice #.
@@CodeOpinion Agreed.
Hello, thanks for your contents. I really appreciate your videos. However, I didnt understand the point here, when you advocate against the use of ID. I mean, by definition, all DDD books I have read says that the main characteristic of an entity is that it has an identity. Even further, Vaugh Vernon in DDD red book make use of and ID as a value object, in order to encapuslates ID details. Please, dont take me wrong, I really want to hear your toughts about my point of viewing. Your videos always give me good insights. Thanks again.
Exactly, if they don't need to be uniquely identified, they shouldn't be entities in the first place. They should be value objects or DTOs or whatever you want to call them.
He's not advocating against the use of IDs - he's making a distinction between Entity Framework data model, which are behaviorless data structures that do have database IDs in them, and the Domain Model, which does not consist of simple data structures, but of objects with behaviors - and in fact, could even just be a bunch of functions, if you're not doing the OO style. It's not a slightly different copy of the data model, instead, it takes in this data and does stuff to it, or delegates to lover level code where needed - but the point is, it controls the business logic flow. It's where the core business logic is encoded, it controls what happens on the high level. He's just saying that you don't need to replicate your data model there, and copy all the data and all the IDs - just figure use what's needed to fulfill the business logic, and design your domain model around that.
Also, please note that Entity Framework entities (data model), Clean Architecture Entities (domain model), and DDD Entities are all different concepts. In DDD, a distinction is made between objects that represent things that can mutate state over time, but have a thread of identity that persists beyond the runtime (or possibly even the lifetime) of the application ("entities"), and objects where two instances with the same state are interchangeable because we only care about their value ("value objects"). It's not a statement about database IDs, it's a conceptual distinction. In the original DDD book, Aggregates are not required to reference other objects by ID, they just need to have a well-defined boundary that defines what's encapsulated behind the Aggregate Root, a surface level object that controls access to anything inside. Vaugh Vernon's approach uses IDs to demarcate that boundary, but it's just an approach (albeit a popular one). It's in no way the essence of DDD.
I'm not advocating against IDs. I'm suggesting that the data you care about in your domain model is what revolves around the behavior in your domain model.
Yet to see a fully decoupled domain, data model where the promises of doing that are actually needed, and I've been doing this for around 10 yrs
I always find creating aggregates with event sourcing as a great example as it shows the only data you actually care about revolves around behaviors in that model. The point is you don't need one model to rule them all.
Nothing says "Enterprise Software" like mapping data between 2 objects with the same properties and shape but different attributes.
Real life tends to be way more complex than simple tutorial videos
While the Shipment may not need the Shipment Id. Debugging is a hell of a lot easier when it does have it. Fair enough, drop the other fields.
true
That's valid.
Would be nice to have a written version of this video for better analyzing
I generally always have a blog post for every video I publish. Usually comes out awhile later.
@@CodeOpinion Cool, yeah I saw your blog, I was reading some other posts meanwhile haha. I think you could go deeper on posts compared to videos, thanks for sharing!
I wish you also have some practical solutions to the problem which is a very known problem.
The final example of passing the data model to the domain model is a solution.
Do onion architecture, create a service to work with your data model.
Derek, do you have a small repository with sample code which shows this idea? Thanks!
No, not public. The code in the video is more pseudo then a working demo to illustrate.
@@CodeOpinion If it's not public, can I access it maybe if I subscribe to your channel? Thanks!
I appreciate this video, but sorry I’m still confused. I wish there was a good real-life example out there that shows more than just the basic examples usually shown.
I may attempt to show something more complex in a video, my guess is I'd lose the audience immediately because having to explain the nuance of the domain and the reasons why the complexity exists isn't trival.
Feel like I am waiting for this videos for years 😂 though I hate that people create useless private setter (most of them are for non related properties) just to make domain entities work with EF, I still could not figure out where I should put those data models. I used to put them in Infras and use some magic from mapping lib to map p, now I got some confidence to just put it in Domain layers to make the mapping more simple 😂
Where I think people get this overblown is trying to go to far so that even the structure isn't understood by the domain model. If you don't want to couple your data model to your domain model, then yes you'd have to do some mapping between the two. But as I always say, the degree to which your coupling matters. If you're data model is concise and you need to change it's structure, how much of an issue is it that you need to adjust how you change the state within the domain model? I assume not much. If you have a monster data model that includes everything under the sun and your domain model is loaded with CRUD for everything, ya this is going to be an issue.
ID isn't really a persistence-level concern, it can be thought of as very much a domain construct. That was not a good example to make your point IMO, although the video was interesting.
The ID is a persistence concern if your DB is generating it. It might not be if its a more natural ID such as an Invoice #.
I believe most of the pain comes from the fact that people map (public) properties to columns. In JPA for example you can choose between property and field access. If you use latter you can provide custom properties and behaviour, as your data is not visible to everyone. In EF the same should be achievable with backing fields.
It is, however the issue more comes up when people use navigation properties and the relationships as a large object model.
@@CodeOpinion what do you mean by that? Navigating deep into the object tree and relying on lazy loading to fetch the requested items or something different? If so, the same rule applies: what you don’t make public, can‘t be navigated.
I think IDs are a persistence concern and Domain Logic is a core concern. When you want to uncouple lower level implementation details from higher level logic you should use an interface to invert the flow of control. The domain model should not have an id. Data models in the application layer should inherit from the domain model and implement an interface IEntity (covariance). Now the infrastructure knows nothing about the domain. Your contracts will protect your domain from being frigid.
I'm not disagreeing with the approach your suggesting, because in a given context, it can make sense. But at the end of the day we're talking about coupling and the degree to which you want to be coupled between those "layers". But the amount of convoluted indirection with little value because of unneeded abstractions kills me.
Many people have experienced what you described here. But there's no much solution to this problem. An ORM solve several problems, which would otherwise be difficult to solve. If you have a graph of objects it's difficult not to use an ORM.
Interestingly enough, when using an ORM, I generally avoid graphs or a object hierarchy on purpose.
@@CodeOpinion How do you do it instead? Do you have an example how you would work with navigations (in a world without them, assuming you use an id instead)?
I got real example how people has anemic models almost without any logic on it,. never changed persistence type or even vendor but paid full price with overhead of separate domain models with repository layer and mappings of recursive models. Thats a sad but true story. They tried to make everything in a "propper" way from the start.
I was about to comment something along these lines.
The concept discussed in the video seems to be solved with anemic "domain" models (which in reality would be just data models) and a lot of repos, interfaces and services to work with them.
I've done videos about this and using transaction scripts with simple data models. The point of using a domain model, in the typical style of an aggregate is because you want a consistency boundary which is more difficult to achieve if you have a pile of trx scripts using the same underlying data model. If you don't have consistency concerns, then yes trx scripts and a data model works fine.
Well I don't now that C# crap 😄, but should be similar to JPA. The problem is that the metadata pollutes the domain model because we all want to use annotations these days. But the truth is that the mapping information doesn't belong to either side - so the old style where the mapping data was kept in a seperate xml file between the database and the domain model was architecturally a better solution.
It's similar idea of annotating when you choose to make your data model the domain model. The issue I have with mapping is trying to bend over backwards because of the ORM. You can do this with a more fluent API in C#/EF but the amount of mapping gymnastics or conceding to the ORM makes you wonder why even bother.
Derek, I can understand the line of thought of having separated Domain and Persistence models, even having ORMs in place, but what do you think about the additional overhead of "converting" in and out between domain and persistence classes? And about the decision of some developers of doing this using mapper tools?
If you have two separate models that you map between, my assumption is you're doing so because you don't want your domain model coupled to your data model. So you have mapping at the infrastructure/persistence level. That's my guess why people do it. My argument for not doing that and having your data model used inside your domain model, is yes you have coupling, but as I always say, the degree of the coupling matters. If you have a focused/narrow domain model and the data model within in changes its shape/structure, sure you'll need to update your domain model internals but since it's so focused, and inherently small, how much coupling do you really have? Also, none of your tests will change in their assertions. But if you have a large data model/structure, you likely also have a huge domain model over it. Which means you'll have to change a lot, which isn't great because you have a high degree of coupling. As always, coupling is what needs to be managed.
@@CodeOpinion As always, context is king. Thanks!
Hi Derek. Nice video, appreciate it.
Did you mean that shipment id is a persistence concern?
Yes I think he does. The Id property means the domain model is assuming that there is an id which is really only there to make persistence work.
@@Rick104547 Well, in DDD an aggregate root, which is a consistency boundary, has an id and there is nothing wrong with it. I mean aggregate roots themselves are also about persistence
It can be, and in this case it was. Something more natural that you're also using for persistence might not be though.
@@CodeOpinion Isn't it quite normal that we have persistence concerns in our domain models/aggregates? Aren't aggregates themselves about enforcing business rules, consistency boundaries and persistence?
I think i am the only one who does not understand the problem 😢
Your first mistake was to use an ORM....
Some prefer it, some don't. If you're writing SQL, often times this can involve emitting events so that you can translate it to some type of SQL statement you write. Ultimately, it's still mapping.
@@CodeOpinion the power of ORM is not in creating SQL queries but in change tracking and materialization. Domain Model uses encapsulation which usually needs to be hacked around when materializing and persisting data.