I'm landing on the conclusion that 99% of architectural problems is exactly because we're writing data-centric code. It's absolutely mindblowing to me just how many problems this causes. We keep flipping between all these "solutions" to our data-centric implementations instead of fixing the underlying issue which is that the fundamental metaphor is completely wrong. Your table isn't your domain model, the domain lives in the code. The problem is that we're technical people so we tend to think in very technological ways. But problem solving involves a LOT of communication and "soft science". Microservices gave us a way to incorporate strict boundaries, but we didn't have any meaningful boundaries to begin with, so they didn't solve anything and just made our lives harder. So now we're back to monoliths. And just like our jump to microservices didn't solve anything, so will the jump back not solve anything until we get our act together and solve the underlying problems with our domain. And just like we don't have a consistent definition for what a microservice is, we will not have a consistent definition for what a "modular monolith" is. I personally love a well-designed modular monolith, and I'm perfectly happy working with well-designed microservices. I'll take either. But we NEED to fix the metaphor first and foremost.
On a recent contract I was working on a modern appliation but built as a monolith (lasagne stack) ... oh my it was a pleasure. The ease of deployment, not having to worry about microservice versions, the ease of running eveything locally... sure it's mid 2000s ideology but I love simplicity! Only one SSL certificate to remember renewing too! 😋 - Only major con being that even a small change would require deploying the whole thing but the simplicity made it worth it!
IT conferences circle of life: monolith -> microservices -> cloud -> tests -> something between -> repeat ;) As you said, work with monolith is not so bad as they say.
I love these conversations. It's always good to hear from those that have seen things transition from where they once were, and then be able to see patterns in order to give us better perspective about where we are today. So often it's all we can do to try and understand the current trends and trajectories of the industry, and keep with it all. So that's why these types of videos are important, to hopefully help us avoid getting wrapped up in hype or spin our wheels due of FOMO.
There's so many factors involved in making an architecture decision. Project size, foreseeable future growth (or non growth), team size, team skill level, team leadership/personalities (who talks the loudest, who is the most persuasive, who can bullshit the best), business priorities (ease of development vs long term maintainability etc) and probably a dozen other things I can't think of or am ignorant of. And yet it always seems like we're looking at the current trend to solve all of our problems. It'll never happen.
With all the CI/CD tooling we have right now, I would just go with independent micro-services with a shared chassis. You get more specialization by setting the domain boundaries upfront. For front-end, mono-repo makes sense.
Microservices were heavily promoted by big tech companies looking to sell their Cloud products. If you look at the timing of the cycles, it is noticeable that they often correspond with the strategy shift of the large tech companies. That is not surprising because they have to develop their Cloud products in a microservice-like manner. I think there is now a small correction in the trend by developers who are disappointed in all the promises of microservices. But I expect that will be short-lived. Because the main trend will remain Microservices for the time being. Moreover, despite everything said, the type of project and 'context' determines how a system will be built. And Microservice is not unnatural. I always say, look at how the public services work in the real world. There are hospitals, dentists, pharmacies, etc... that all work together in one way or another, exchanging information, expanding if necessary, splitting up, moving, etc... based on the needs of the people. The microservice idea is nothing more than this and occurs everywhere.
Once you could run microservices on every platform it stopped being attractive catch phrase to Cloud providers. They shifted towards serverless and lowcode and now they are trying to laverage AI as the reason to go their cloud. I don't think we are going back to microservices On the other hand, I think about microservices as just reminiscent of SOA, which is older than the modern cloud. The prefix micro- is actually more misleading as it draws attention to the size of a service, which is not the point of the idea
I recently migrated from msa to modular monolith. It’s great no more 5 micro services per person. The decision to go msa should be guided by team structure and deployment unit
I think the issue is its solutions looking for problems to solve only to find out there was no problem. Probably 9/10 projects out there don't need this complication but just simple straightforward solutions. It's the same with ppl dogmatically applying clean architecture while it doesn't make any sense in most situations. I would start out as simple as possible with vertical slicing. Do make sure there is a good test suite. That might be enough already for most situations and in situations where its not more engineering can be done to figure out the right solution. Applying microservices or any other complicated pattern from the start is simply doomed to fail.
+1 ... Don't you just love it when you start a new software engineering gig and discover the architect had a wet dream over including every possible type of component in a given framework because it was "cool" rather than focusing on what was needed. Also an extra kick in the balls when you find out they no longer work at the company.
There is no silver bullet. Each scenario has multiple solutions that qualify as 'best'. Issues almost always occur because software is coded and not engineered. What I mean by that is, in 2024, a solution where all logic is implemented as Oracle Stored Procedures can still be valid IF you have the engineering around it to support it (I would not do this). There are only 5 points you need to satisfy to be successful. 1) Does the code fulfill the business need. 2) How maintainable is the code? ANYONE can create a day 1 solution. That will change, can you keep up? 3) Can we prove that our code does what we say is does. Testing metrics (not just coverage). Coverage is basic (yet someone absent too often). Can you tell me what scenarios are tested? Can you repeat those test EVERY time you do a release? Are you mocking things correctly (proper test actually hit the database you control). 4) Can you release a quality product in a timely manner with the supporting metrics to prove it is a healhty product. 5) Can you monitor the system post release to ensure it is performing. The how (java vs node vs go vs python) is almost irrelevant, as long as you know you can easily get quality people as needed (attrition is a fact of life, prepare for it). When something goes wrong, you must be able to speak to the business side in english, not tech, to explain things, then fix it. At that point, no one cares if your front end is fully responsive and if the backend is fully reactive, they just care that is it not working.
The swings likely stem from the engineering culture in silicon valley companies + FOMO from the rest of the world. This is Conway's Law at work. pre-2010s the software engineers worked on wide varieties business contexts, whatever industry they're in. Cloud services were only beginning to gain popularity so migration was seen more as high risk. High cohesion project makes more sense as it allows for easy navigation cross-domains, hence the monolith project. In the 2010s the preferred work is for the engineers to not just excel at the tech but also be the expert at the domain they're handling as it's thought to boost innovation (IMO it did). Teams had boundaries and their own little governance, hence the microservices. Now domain expertise is burdened to the product team, with a lack of focus from the business as most companies are just trying to hit product-market fit. engineers are demanded to be more flexible, emphasizing on reusable components and multiple business context 'submodules' that use those components. and people start realizing the microservice is expensive too.
Great video! Can you expand on the authorization service? What are the downsides of a dedicated service for RBAC/ABAC? Are there better approaches sometimes? While a separate authorization service adds network traffic, the deployability benefits in Kubernetes for example could outweigh it don't you think?
Could you elaborate some on William's argument that you shouldn't have a dedicated authorization service/module, but rather implement it as part of the other independent modules? --- As I see it, an authorization module could be one of the system's core modules. Rarely changing. I have played around with this recently, and have ended up with an authorization module responsible for providing some of the core capabilities you would expect related to authorization. For example, "grant permission on a resource", "revoke permission on a resource", "retrieve permissions on a resource", "evaluate a user's permission on a resource", and so on, where the "resource" is a loose reference to some resource's primary key, and the permissions can be defined as "anything" as implemented by the other modules. With the heavy lifting done by the authorization module, the other business capability focused modules can use these permission capabilities to implement their own authorization policies. The policies themselves can of course be more complex, and tailored for a particular modules requirements. In other words; permissions are managed by the centralized authorization module, while policies are implemented by other independent modules (and are only relevant there as well).
Wasn't he referring to a separate Microservice for Authorization, where-as you're referring to a separate 'module' (as in a modular monolith possibly). So I don't know if that means you have a logical separation or a physical module separation like a microservice. If it IS a physically separate module, then I think it's talking about a single point of failure and as you add more microservices that depend on it, more network latency/slowness. If you are dealing with microservice separation, instead of rolling your own authorization service, another option could be adding a service mesh like Istio (if using Kubernetes) and using the sidecar patterns it provides, especially around security.
@@chrisl8292 Thanks for answering! I meant "module" as in an separate, independent piece of the system. Whether it is a module or a microservice was not the important part of the question, but rather "how to handle the situation regardless of in-process vs network". Where should authorization ideally fit within this kind of system?
I like your videos either way, but I think conversations like this are more friendly for less senior developers, and therefore I'm more likely to share videos like this one.
I think the problem is poor education of people in higher positions, a lot of untrained, inexperienced and overconfident young developers and a big tendency to consume generic solutions to generic problems rather than think about your actual problems of your software and your development process. I started in Web Development 2018 and had no history before that. I can’t tell if this has always been the case even before RUclips and excessive blogging, but this is my impression of the WebDev-Community. Can anyone relate?
So I have been following this conversation and I am really confused where the idea of micro service been a bad thing is coming from. Do we really have data for this assertion. Companies like Monzo have built the best digital bank in the UK on microservices from ground up. If the folks dumbing microservices for modular monolith will take the time to try and do it right they will reap the true benefits which is not about decouple systems but loosely coupled system. And the inability to orchestrate this business service (and a again it's not about Size but independence) to function as one big single logic unit that d3livers value to the business and it's customers is what is missing.
It feels like this conversation just lowered the value of the channel for me 😁 I came here to hear an interesting opinion about something I don't yet know. But this particular type of misunderstanding of microservices is something I had to deal with at work every single day. I guess anyone can talk about things they don't understand and argue about lack of value these days.
I think microservices overrated in most cases. all pros that microservices provides are most same ones that modular monolith does except a deployment maybe as in microservices we can rolling upgrade and rollback each service separately. Modular monolith can be decomposed by modules (dotNet assemblies, nuget packages, java jar packages) according to ddd bounded context. After that the resulting modules assigned to corresponding teams to be developed. Within of each module the vertical slice (feature-based) architecture can be applied. Modules can interact with each other via in-memory event/command bus. Finally, they are assembled together inside single composition root. Because of above a module monolith seems good start point to advance to microservice in the future
When Microservices became a thing (a decade ago)....it just seemed like a "code smell" that was approaching at a snail pace! Having to maintain the vast amount of Azure Functions you have now created....over a number of massive API's\Controllers!
Part 2 & 3 of this chat with William will be published as well.
Waiting for it
I'm landing on the conclusion that 99% of architectural problems is exactly because we're writing data-centric code. It's absolutely mindblowing to me just how many problems this causes. We keep flipping between all these "solutions" to our data-centric implementations instead of fixing the underlying issue which is that the fundamental metaphor is completely wrong. Your table isn't your domain model, the domain lives in the code. The problem is that we're technical people so we tend to think in very technological ways. But problem solving involves a LOT of communication and "soft science".
Microservices gave us a way to incorporate strict boundaries, but we didn't have any meaningful boundaries to begin with, so they didn't solve anything and just made our lives harder. So now we're back to monoliths. And just like our jump to microservices didn't solve anything, so will the jump back not solve anything until we get our act together and solve the underlying problems with our domain. And just like we don't have a consistent definition for what a microservice is, we will not have a consistent definition for what a "modular monolith" is.
I personally love a well-designed modular monolith, and I'm perfectly happy working with well-designed microservices. I'll take either. But we NEED to fix the metaphor first and foremost.
On a recent contract I was working on a modern appliation but built as a monolith (lasagne stack) ... oh my it was a pleasure. The ease of deployment, not having to worry about microservice versions, the ease of running eveything locally... sure it's mid 2000s ideology but I love simplicity! Only one SSL certificate to remember renewing too! 😋 - Only major con being that even a small change would require deploying the whole thing but the simplicity made it worth it!
IT conferences circle of life: monolith -> microservices -> cloud -> tests -> something between -> repeat ;) As you said, work with monolith is not so bad as they say.
I love these conversations. It's always good to hear from those that have seen things transition from where they once were, and then be able to see patterns in order to give us better perspective about where we are today. So often it's all we can do to try and understand the current trends and trajectories of the industry, and keep with it all. So that's why these types of videos are important, to hopefully help us avoid getting wrapped up in hype or spin our wheels due of FOMO.
There's so many factors involved in making an architecture decision. Project size, foreseeable future growth (or non growth), team size, team skill level, team leadership/personalities (who talks the loudest, who is the most persuasive, who can bullshit the best), business priorities (ease of development vs long term maintainability etc) and probably a dozen other things I can't think of or am ignorant of.
And yet it always seems like we're looking at the current trend to solve all of our problems. It'll never happen.
This channel consistently delivers great content. Thank you!
I appreciate that!
Deep thoughts. Highly appreciated. And good point about business rule on users able to place orders under 2000 dollars.
With all the CI/CD tooling we have right now, I would just go with independent micro-services with a shared chassis. You get more specialization by setting the domain boundaries upfront.
For front-end, mono-repo makes sense.
Microservices were heavily promoted by big tech companies looking to sell their Cloud products. If you look at the timing of the cycles, it is noticeable that they often correspond with the strategy shift of the large tech companies. That is not surprising because they have to develop their Cloud products in a microservice-like manner.
I think there is now a small correction in the trend by developers who are disappointed in all the promises of microservices. But I expect that will be short-lived. Because the main trend will remain Microservices for the time being. Moreover, despite everything said, the type of project and 'context' determines how a system will be built. And Microservice is not unnatural.
I always say, look at how the public services work in the real world. There are hospitals, dentists, pharmacies, etc... that all work together in one way or another, exchanging information, expanding if necessary, splitting up, moving, etc... based on the needs of the people. The microservice idea is nothing more than this and occurs everywhere.
Once you could run microservices on every platform it stopped being attractive catch phrase to Cloud providers. They shifted towards serverless and lowcode and now they are trying to laverage AI as the reason to go their cloud. I don't think we are going back to microservices
On the other hand, I think about microservices as just reminiscent of SOA, which is older than the modern cloud. The prefix micro- is actually more misleading as it draws attention to the size of a service, which is not the point of the idea
I'd definitely like to see more of these conversation videos.
I recently migrated from msa to modular monolith. It’s great no more 5 micro services per person. The decision to go msa should be guided by team structure and deployment unit
I think the issue is its solutions looking for problems to solve only to find out there was no problem.
Probably 9/10 projects out there don't need this complication but just simple straightforward solutions. It's the same with ppl dogmatically applying clean architecture while it doesn't make any sense in most situations.
I would start out as simple as possible with vertical slicing. Do make sure there is a good test suite. That might be enough already for most situations and in situations where its not more engineering can be done to figure out the right solution. Applying microservices or any other complicated pattern from the start is simply doomed to fail.
+1 ... Don't you just love it when you start a new software engineering gig and discover the architect had a wet dream over including every possible type of component in a given framework because it was "cool" rather than focusing on what was needed. Also an extra kick in the balls when you find out they no longer work at the company.
There is no silver bullet. Each scenario has multiple solutions that qualify as 'best'. Issues almost always occur because software is coded and not engineered.
What I mean by that is, in 2024, a solution where all logic is implemented as Oracle Stored Procedures can still be valid IF you have the engineering around it to support it (I would not do this).
There are only 5 points you need to satisfy to be successful. 1) Does the code fulfill the business need. 2) How maintainable is the code? ANYONE can create a day 1 solution. That will change, can you keep up? 3) Can we prove that our code does what we say is does. Testing metrics (not just coverage). Coverage is basic (yet someone absent too often). Can you tell me what scenarios are tested? Can you repeat those test EVERY time you do a release? Are you mocking things correctly (proper test actually hit the database you control). 4) Can you release a quality product in a timely manner with the supporting metrics to prove it is a healhty product. 5) Can you monitor the system post release to ensure it is performing.
The how (java vs node vs go vs python) is almost irrelevant, as long as you know you can easily get quality people as needed (attrition is a fact of life, prepare for it). When something goes wrong, you must be able to speak to the business side in english, not tech, to explain things, then fix it. At that point, no one cares if your front end is fully responsive and if the backend is fully reactive, they just care that is it not working.
Hey this is great content about software architecture
The swings likely stem from the engineering culture in silicon valley companies + FOMO from the rest of the world. This is Conway's Law at work.
pre-2010s the software engineers worked on wide varieties business contexts, whatever industry they're in. Cloud services were only beginning to gain popularity so migration was seen more as high risk. High cohesion project makes more sense as it allows for easy navigation cross-domains, hence the monolith project.
In the 2010s the preferred work is for the engineers to not just excel at the tech but also be the expert at the domain they're handling as it's thought to boost innovation (IMO it did). Teams had boundaries and their own little governance, hence the microservices.
Now domain expertise is burdened to the product team, with a lack of focus from the business as most companies are just trying to hit product-market fit. engineers are demanded to be more flexible, emphasizing on reusable components and multiple business context 'submodules' that use those components. and people start realizing the microservice is expensive too.
Great video! Can you expand on the authorization service?
What are the downsides of a dedicated service for RBAC/ABAC?
Are there better approaches sometimes?
While a separate authorization service adds network traffic, the deployability benefits in Kubernetes for example could outweigh it don't you think?
Could you elaborate some on William's argument that you shouldn't have a dedicated authorization service/module, but rather implement it as part of the other independent modules?
---
As I see it, an authorization module could be one of the system's core modules. Rarely changing.
I have played around with this recently, and have ended up with an authorization module responsible for providing some of the core capabilities you would expect related to authorization. For example, "grant permission on a resource", "revoke permission on a resource", "retrieve permissions on a resource", "evaluate a user's permission on a resource", and so on, where the "resource" is a loose reference to some resource's primary key, and the permissions can be defined as "anything" as implemented by the other modules.
With the heavy lifting done by the authorization module, the other business capability focused modules can use these permission capabilities to implement their own authorization policies. The policies themselves can of course be more complex, and tailored for a particular modules requirements.
In other words; permissions are managed by the centralized authorization module, while policies are implemented by other independent modules (and are only relevant there as well).
Wasn't he referring to a separate Microservice for Authorization, where-as you're referring to a separate 'module' (as in a modular monolith possibly). So I don't know if that means you have a logical separation or a physical module separation like a microservice.
If it IS a physically separate module, then I think it's talking about a single point of failure and as you add more microservices that depend on it, more network latency/slowness.
If you are dealing with microservice separation, instead of rolling your own authorization service, another option could be adding a service mesh like Istio (if using Kubernetes) and using the sidecar patterns it provides, especially around security.
@@chrisl8292 Thanks for answering! I meant "module" as in an separate, independent piece of the system. Whether it is a module or a microservice was not the important part of the question, but rather "how to handle the situation regardless of in-process vs network". Where should authorization ideally fit within this kind of system?
I like your videos either way, but I think conversations like this are more friendly for less senior developers, and therefore I'm more likely to share videos like this one.
Wasn't the monolith modular before? I mean modules are around since the 60s, right?
I have experienced the entity based service delineation and it sure isn't pretty after a while
I think the problem is poor education of people in higher positions, a lot of untrained, inexperienced and overconfident young developers and a big tendency to consume generic solutions to generic problems rather than think about your actual problems of your software and your development process. I started in Web Development 2018 and had no history before that. I can’t tell if this has always been the case even before RUclips and excessive blogging, but this is my impression of the WebDev-Community. Can anyone relate?
So I have been following this conversation and I am really confused where the idea of micro service been a bad thing is coming from. Do we really have data for this assertion. Companies like Monzo have built the best digital bank in the UK on microservices from ground up. If the folks dumbing microservices for modular monolith will take the time to try and do it right they will reap the true benefits which is not about decouple systems but loosely coupled system. And the inability to orchestrate this business service (and a again it's not about Size but independence) to function as one big single logic unit that d3livers value to the business and it's customers is what is missing.
It feels like this conversation just lowered the value of the channel for me 😁
I came here to hear an interesting opinion about something I don't yet know. But this particular type of misunderstanding of microservices is something I had to deal with at work every single day.
I guess anyone can talk about things they don't understand and argue about lack of value these days.
Which misunderstanding?
I think microservices overrated in most cases. all pros that microservices provides are most same ones that modular monolith does except a deployment maybe as in microservices we can rolling upgrade and rollback each service separately. Modular monolith can be decomposed by modules (dotNet assemblies, nuget packages, java jar packages) according to ddd bounded context. After that the resulting modules assigned to corresponding teams to be developed. Within of each module the vertical slice (feature-based) architecture can be applied. Modules can interact with each other via in-memory event/command bus. Finally, they are assembled together inside single composition root. Because of above a module monolith seems good start point to advance to microservice in the future
Nothing has not been done yet ;)
When Microservices became a thing (a decade ago)....it just seemed like a "code smell" that was approaching at a snail pace!
Having to maintain the vast amount of Azure Functions you have now created....over a number of massive API's\Controllers!
This was a conversation between two confused people with no real insight... Which is a great demonstration of where architecture is at right now
Thanks for the great feedback!
The only thing we learnt is that they suck!!!