Those people who complain about "not fast enough talk" are exactly the problem of our time. People rushing through concepts, just understanding it partially and applying it right away, again rushed, without much thinking. That perfectly explains why more than 90% of Microservice architectures are such a gigantic mess and also most of the talks about it. Thanks to Chris we - the "slow thinkers" - still have a chance to turn things around for the better, cleaning up all the BS the "rushing people" left behind
I thought this was great and not sure how people are watching this at a higher speed... I needed to pause this periodically and take in its glory. This was finally proved the event sourcing model for me.
So glad I found this talk. I watched at 1x speed. Hearing someone speak at their pace allows you to hear words being emphasized in ways you will never heard at faster speeds. This was an awesome talk.
I love Chris. Thanks to guys like him there is still hope that simplicity and clean thinking prevails over all that overcomplication and endless confusion of other microservice talks.
The simplest programming paradigms usually win in the long run. Not sure if this event source method will ever become popular when you shift more programming efforts & complexity to the business logic developer. However, this video is worth watching for some of the ideas (such as aggregates and how it relates to transactional semantics) in it.
Perfect example of how you can add accidental complexity in your software solution. For those who are looking for DDD and how it can helps you to split your solution in services, you're in the wrong place.
Completely different architecture style seems very useful but It would have been more beneficial if video also involves a live project. The concept explanation was very good, it just will take some time to accustom with all concepts explained.
Totally enjoyed this talk...lots of valuable information in a short duration...by the way is the Order service going to be in blocked state until it gets an event telling it that Customer credit limit is ok???
No it would send an async message to the message bus, get an OK response back and return control back to the user. Later a new message of credit approval or rejection would be oublished.on the message bus and picked up by the order service. It would the update the order information and inform the user in some way like an email.
A generous talk from a generous person. Thank you! Yet, if I have to turn software and data design practices upsidedown, and if this were the only way, then I will say: thank you! I don't want microservices, at least for now. Once ideas become more mature and once these side effects are either avoided or at least addressed in a transparent way, we can talk. Are you serious!? Would you add all this complexity! Would you start doing journal maintenance as if you were writing the DBMS itself!? And after all, you'll still have to write here (DB) and there (message broker). The 2PC problem still persists, or maybe I missed it. So, again, a generous talk, but a counterproductive, unrealistic solution.
You're not writing to two places within the transaction. You're writing in one place (DB) with the transaction. Then you're sending the message to the broker (at least) once within a transaction. So the idea is that you can commit the message to the outbox so that the entire operation leading up to and including sending that message is atomic. Then you rely on your ability to only delete the message once it's sent (again using a transaction) to ensure at least once message delivery. So you will only modify your state if the resulting message/event was sent at least once. You cannot guarantee this if you try to send the message/event directly to the broker.
There rarely is a need to search for your orders by a keyword. And if that is needed for business add a read model for customer orders with text query support.
Around minute 54, it is mentioned that events are published on kafka as well as the event store. How is consistency achieved between those two "distributed resources"?
It is confusing. Firstly, the terms aggregate is misused here. Actually, you are not aggregating anything. You just locally referencing to the same information that moved out to its own service. You can do that without local referencing as well. (like service call) Therefore, it is just an optimization technic in my view. Another way to implement it via the workflow process. In this case, you do not need any local referencing (AKA: "aggregate")
Eventsource is like how our body works. Brain is an event store, every other cells are like seperate services that do not know each other and react to the signal(event) that brain sends. And if something happens in a cell, that cell sends a signal to the brain. Thats why our body is sourrounded by millions of neurons. One caveat though. If brain (event store) fails to function, the whole system stops functioning.
the customer workflow seems tightly coupled to the customer service, since it uses its code to apply events. Why wouldn't we decouple that workflow into a different processor that sends commands to the user service?
I haven't watched the whole video yet, but looking at the thumbnail, it appears that he's building a monilithic UI rather than a composite UI? Doesn't that just result in a distributed monolith?
Something about event sourcing being a solution for strong consistency for DB Write and Message publish doesnt seem right here. It is perfectly fine to be strongly consistent within one microservice or one DDD Aggregate.
Isn't there a race condition between checking the credit limit and applying a new event as it is shown (by first checking that applying the new reservation)? Would it not make more sense to add a reservation, only then check if the limit is exceeded and if so, revoke the reservation? (Edit: in the example code around minute 57)
No. Applying the reservation and checking happens in the same service, they can e.g. lock the customer row within one transaction, insert the reservation and commit.
Also there is probably no race condition as they might end up reading the events for a certain customer within one consumer and process them in order (see Kafka partitions).
While I agree with the idea of "atomic data", I completely disagree with the suggestion that each service has it's "own database". Here's just a few reasons why: For starters, most companies have legacy databases filled with many years worth of data, stored procedures, etc. Secondly, SQL servers have perfected the concepts of transactions and referential integrity. Why give that up simply to achieve the concept of "autonomy"? Might I suggest a better / simpler solution? Why don't you utilize database SECURITY instead? You can limit each service to their own list of objects. Let's say you have 20 services. Simply create 20 users named after each service. Then, grant specific rights to each of those users. This gives you the benefits of autonomous data management while preserving referential integrity. It also allows you to continue to utilize stored procedures. In other words, the "Orders Process" can call a procedure that updates the customer's credit limit, but the "Orders Service" has no idea that there is a customers table. I find this to be far better than creating a bunch of databases - then replacing the decades long proven transaction capabilities and foreign key constraints of a SQL server with a proprietary blend of "pub / sub" code written by literally anyone. I'd much rather find a "security solution" to ensure "atomic access" to specific objects. It's far better than reinventing the wheel.
This sounds good in theory, if its a very small company and everyone follows the idea it will work. However with most dev team thats not living in reality (at least in big companies). What happens is you won't find out about a schema change until things go down. Even if you make it clear the schema isn't to be touched, 6 months down the line people forget, new teams are given db access by someone who shouldnt give it, and list 100 other possibilities...
@@JamesSmith-cm7sg That's why there's this thing called testing. You don't just roll out a schema change. You run (automated) tests for all your services including processes that span many services and if the schema change breaks something you'll see it. This whole "pure micro services" way of thinking is so ridiculous... What happens if some dev team breaks a micro service that is required for common use cases? Right...
I would say you haven't tried to dive deeper than just complaining... Just saying, if you have such questions and attitude, you clearly seem like you didn't spend enough time on fundamentals about a different approach. Try with this talk first ruclips.net/video/CZ3wIuvmHeM/видео.html, rather than giving salty comments like "whole industry is wrong, just I clearly see how to do it the right way". Btw, stored procedures and the single database won't last long in a big company. That's a path to a failure that was proven by dozens of big companies. Small companies don't usually care much of it. You can do whatever you want without having big issues.
He is right that aggregates are key, but my issue is he incorrectly sliced up the aggregates because his business logic is filtered across the entire system for say placing an order. This DDD anti-pattern is known as the distributed monolith.
He's actually placing the business logic behavior closest to the data the business logic depends on. This follows the OOP principle of cohesion, which is a widely-accepted practice in OOP: en.m.wikipedia.org/wiki/Cohesion_(computer_science)#:~:text=In%20object%2Doriented%20programming%2C%20if,while%20complexity%20is%20kept%20manageable.
It is about using microservices to enable scaling beyond a monolith. You can of course implement microservices with transactional databases and APIs. However, there are a lot of pitfalls that this event driven model avoids, but unfortunately adds some new ones.
Code/ Functionality first approach is best approach in Software industry and in manufacturing industry Model first as their manufacturing process is based on model ( finished or semi finished product). Aggregation is more towards Model first approach , in case you have already defined BC( bounded context ) then no sense of model granularity as It is already handled during identification of BC. By example in case you have develop is software for school you have to focus admission process exam process then accordingly you have to think required model . in most of case process is important . Siddharth Singh (www.linkedin.com/in/singhsiddh/)
I haven't found many talks/examples of DDD-based microservices, if you will, that detail auth. If anybody has any, I'd appreciate some references to them.
I have only one question if we hit the service directly will it redirect to api gateway automatically? if we need to make it to work what should be done?
Optimistic locking have nothing to do with transaction. Also it's not a bad situation, it's a good situation. Normally order processing systems work in a first come-first served manner, so you HAVE to have optimistic locking, but again optimistic locking is only about timestamp comparison prior to updating the data in the db. It doesn't have anything to do with transactions.
Full of waste time trying to solve fundamental issues of micro services by introducing bunch of other new sever issues. You counted how many drawbacks you have and called them simple? Did you try to cover how to solve each or some at least? When you were going with aggregate description as a unit, and then moved to modularized aggregate and granularity level, you lost me again going from aggregate to entity again.
I am not native speaker. For me speed is perfect, I understood each word. Thank you. Great material!
Those people who complain about "not fast enough talk" are exactly the problem of our time. People rushing through concepts, just understanding it partially and applying it right away, again rushed, without much thinking.
That perfectly explains why more than 90% of Microservice architectures are such a gigantic mess and also most of the talks about it.
Thanks to Chris we - the "slow thinkers" - still have a chance to turn things around for the better, cleaning up all the BS the "rushing people" left behind
I'm not a rush individual.. but my company made me become one
I thought this was great and not sure how people are watching this at a higher speed... I needed to pause this periodically and take in its glory. This was finally proved the event sourcing model for me.
Speed is perfect imo, he talks about many complex concepts and to absorb all of that it helps to have the words flow more slowly.
So glad I found this talk. I watched at 1x speed. Hearing someone speak at their pace allows you to hear words being emphasized in ways you will never heard at faster speeds. This was an awesome talk.
I have been on 2x soo much that 1x sounds slo-mo to me now
Set playback speed to 1.25 and enjoy! Complex concepts explained with simple examples, great talk
Best presentation I've seen so far on microservices and eventsourcing.
i agree
I love Chris. Thanks to guys like him there is still hope that simplicity and clean thinking prevails over all that overcomplication and endless confusion of other microservice talks.
in the name of technology everyone is creating their own version of truth and market it the best. each day a new cook with new cutlery
This guy is founder of Cloud Foundry and API Gateway architecture
so the sales pitch eh ?
The simplest programming paradigms usually win in the long run. Not sure if this event source method will ever become popular when you shift more programming efforts & complexity to the business logic developer. However, this video is worth watching for some of the ideas (such as aggregates and how it relates to transactional semantics) in it.
Perfect example of how you can add accidental complexity in your software solution. For those who are looking for DDD and how it can helps you to split your solution in services, you're in the wrong place.
Totally agree. Full of waste time trying to solve fundamental issues of micro services by introducing bunch of other new sever issues.
@@mohyeid1857 how would you solve the transaction issues in micro service ? could you point to some resources ?
Great overview, really enjoyed your presentation! Totally agree that listening is better at >=1.5x speed.
Watching at x2 speed is recommended for this great video
And then try watch it in normal speed.
Lk
Completely different architecture style seems very useful but It would have been more beneficial if video also involves a live project. The concept explanation was very good, it just will take some time to accustom with all concepts explained.
First of all this not a different architecture, its old architecture and discussed before many times. For more, you can refer to the SAGA.
Excellent talk but I'm sure there is an easy way to do this. Complex way to wrap one's head!! :)
Totally enjoyed this talk...lots of valuable information in a short duration...by the way is the Order service going to be in blocked state until it gets an event telling it that Customer credit limit is ok???
No it would send an async message to the message bus, get an OK response back and return control back to the user. Later a new message of credit approval or rejection would be oublished.on the message bus and picked up by the order service. It would the update the order information and inform the user in some way like an email.
I started a small open source project which became CloudFoundry - CR
Excellent talk. This has clarified how DDD is important for designing and developing micro services.
Repository: github.com/cer/event-sourcing-examples
You, dear sir, are a god sent person 🙌
A generous talk from a generous person. Thank you! Yet, if I have to turn software and data design practices upsidedown, and if this were the only way, then I will say: thank you! I don't want microservices, at least for now. Once ideas become more mature and once these side effects are either avoided or at least addressed in a transparent way, we can talk.
Are you serious!? Would you add all this complexity! Would you start doing journal maintenance as if you were writing the DBMS itself!?
And after all, you'll still have to write here (DB) and there (message broker). The 2PC problem still persists, or maybe I missed it.
So, again, a generous talk, but a counterproductive, unrealistic solution.
You're not writing to two places within the transaction. You're writing in one place (DB) with the transaction. Then you're sending the message to the broker (at least) once within a transaction. So the idea is that you can commit the message to the outbox so that the entire operation leading up to and including sending that message is atomic. Then you rely on your ability to only delete the message once it's sent (again using a transaction) to ensure at least once message delivery. So you will only modify your state if the resulting message/event was sent at least once. You cannot guarantee this if you try to send the message/event directly to the broker.
Thank you for the talk. Now I understand better how microservices can persist data without transactions. I like the Event Store approach the most.
Thank you Chris for this excellent presentation!
crazy good presentation!
19:16 ... now allow the customer to search for an order which contained "headphones" and the microservice journey begins...
best comment ever for this vid...
There rarely is a need to search for your orders by a keyword. And if that is needed for business add a read model for customer orders with text query support.
Another service serving this just this view could save a lot of architecture change trouble.
Very Nice description
Good explanation
This is fantastic. Wish I'd found this video much earlier.
Awesome video, Realy it is helpful.
I am expecting the multitenant based microservice architecture for distributed databases.
Great talk , stil very actual , thanks.
Great talk to refresh my theoretical DDD know how. I like the speed.
26:01 between two context boundaries (where no shared distributed DB is used) 2PC cannot be used.
Very well explained, Thank you
Around minute 54, it is mentioned that events are published on kafka as well as the event store. How is consistency achieved between those two "distributed resources"?
It is confusing. Firstly, the terms aggregate is misused here. Actually, you are not aggregating anything. You just locally referencing to the same information that moved out to its own service. You can do that without local referencing as well. (like service call) Therefore, it is just an optimization technic in my view. Another way to implement it via the workflow process. In this case, you do not need any local referencing (AKA:
"aggregate")
Eventsource is like how our body works. Brain is an event store, every other cells are like seperate services that do not know each other and react to the signal(event) that brain sends. And if something happens in a cell, that cell sends a signal to the brain. Thats why our body is sourrounded by millions of neurons.
One caveat though. If brain (event store) fails to function, the whole system stops functioning.
watch at x1.25 speed or you'll be sleeping before the 5th minute
x1 is good for non-English people:)
but these concepts are the important ones to consider when building out.
thats like the first thing i did when i opened the video..
2x
No joke.
Excellent talk
Watched in 1.25x. Best explanation on DDD that I've come across.
Real talk starts at about 5 minutes in.
My lecture of DDD Chris Evans' book worth
I just love this guys man.
the customer workflow seems tightly coupled to the customer service, since it uses its code to apply events. Why wouldn't we decouple that workflow into a different processor that sends commands to the user service?
Great explanation 👏🏼👏🏼
Great talk!
impressive
I am confused at 42:18 , isnt in event sourcing there are only inserts and no updates
Where are those code samples, mentioned in presentation?
I haven't watched the whole video yet, but looking at the thumbnail, it appears that he's building a monilithic UI rather than a composite UI? Doesn't that just result in a distributed monolith?
Good speech, but "transaction that needed to be atomic" is kind of tautology. Transaction is atomic by definition.
Well explained!!
Axon framework is to make Event driven Microservices and Axon server is the Event store, am I correct ?
Something about event sourcing being a solution for strong consistency for DB Write and Message publish doesnt seem right here. It is perfectly fine to be strongly consistent within one microservice or one DDD Aggregate.
He says so in his book too
Aggregates starts at 15:15
Can we've some code to explain this concepts?
Isn't there a race condition between checking the credit limit and applying a new event as it is shown (by first checking that applying the new reservation)? Would it not make more sense to add a reservation, only then check if the limit is exceeded and if so, revoke the reservation? (Edit: in the example code around minute 57)
No. Applying the reservation and checking happens in the same service, they can e.g. lock the customer row within one transaction, insert the reservation and commit.
Also there is probably no race condition as they might end up reading the events for a certain customer within one consumer and process them in order (see Kafka partitions).
brilliant
Great explanation, thank you!
Whats the problem if we make some sort of call to customer service at 28:47. It will reserve the credit same way.
Temporal coupling. The availability of Order Service would depend on the availability of Customer Service.
Event Sourcing looks something similar to Store in ReactJs
Exactly, ReactJS adapted it
1.25 speed, finally normal speed
try 1.5
While I agree with the idea of "atomic data", I completely disagree with the suggestion that each service has it's "own database". Here's just a few reasons why:
For starters, most companies have legacy databases filled with many years worth of data, stored procedures, etc. Secondly, SQL servers have perfected the concepts of transactions and referential integrity. Why give that up simply to achieve the concept of "autonomy"?
Might I suggest a better / simpler solution?
Why don't you utilize database SECURITY instead? You can limit each service to their own list of objects. Let's say you have 20 services. Simply create 20 users named after each service. Then, grant specific rights to each of those users.
This gives you the benefits of autonomous data management while preserving referential integrity. It also allows you to continue to utilize stored procedures. In other words, the "Orders Process" can call a procedure that updates the customer's credit limit, but the "Orders Service" has no idea that there is a customers table.
I find this to be far better than creating a bunch of databases - then replacing the decades long proven transaction capabilities and foreign key constraints of a SQL server with a proprietary blend of "pub / sub" code written by literally anyone.
I'd much rather find a "security solution" to ensure "atomic access" to specific objects. It's far better than reinventing the wheel.
This sounds good in theory, if its a very small company and everyone follows the idea it will work. However with most dev team thats not living in reality (at least in big companies). What happens is you won't find out about a schema change until things go down. Even if you make it clear the schema isn't to be touched, 6 months down the line people forget, new teams are given db access by someone who shouldnt give it, and list 100 other possibilities...
@@JamesSmith-cm7sg That's why there's this thing called testing.
You don't just roll out a schema change. You run (automated) tests for all your services including processes that span many services and if the schema change breaks something you'll see it.
This whole "pure micro services" way of thinking is so ridiculous...
What happens if some dev team breaks a micro service that is required for common use cases? Right...
@@xnoreq
If only...
I would say you haven't tried to dive deeper than just complaining... Just saying, if you have such questions and attitude, you clearly seem like you didn't spend enough time on fundamentals about a different approach. Try with this talk first ruclips.net/video/CZ3wIuvmHeM/видео.html, rather than giving salty comments like "whole industry is wrong, just I clearly see how to do it the right way".
Btw, stored procedures and the single database won't last long in a big company. That's a path to a failure that was proven by dozens of big companies.
Small companies don't usually care much of it. You can do whatever you want without having big issues.
@@xnoreq It's so ridiculous that it works great at amazon for example xD..
I now know to not implement event-source
Where is the code link?
He is right that aggregates are key, but my issue is he incorrectly sliced up the aggregates because his business logic is filtered across the entire system for say placing an order. This DDD anti-pattern is known as the distributed monolith.
He's actually placing the business logic behavior closest to the data the business logic depends on. This follows the OOP principle of cohesion, which is a widely-accepted practice in OOP:
en.m.wikipedia.org/wiki/Cohesion_(computer_science)#:~:text=In%20object%2Doriented%20programming%2C%20if,while%20complexity%20is%20kept%20manageable.
It is about using microservices to enable scaling beyond a monolith. You can of course implement microservices with transactional databases and APIs. However, there are a lot of pitfalls that this event driven model avoids, but unfortunately adds some new ones.
It seems alternate solutions and design patterns just opens up a different can of worms LOL
So microservices call other microservices which are even micro-er?
This event storing to database looks unnecessarily complex.
Code/ Functionality first approach is best approach in Software industry and in manufacturing industry Model first as their manufacturing process is based on model ( finished or semi finished product).
Aggregation is more towards Model first approach ,
in case you have already defined BC( bounded context ) then no sense of model granularity as It is already handled during identification of BC.
By example in case you have develop is software for school you have to focus admission process exam process then accordingly you have to think required model . in most of case process is important .
Siddharth Singh (www.linkedin.com/in/singhsiddh/)
Is there a github for this project?
Look on eventuate.io ! There are eventuate itself and all example projects including this one from video.
The source code from the presentation, can be found here: github.com/cer/event-sourcing-examples
@@octavr2008 Tq
Here is another presentation by Chris with the Sagas solution:
ruclips.net/video/YPbGW3Fnmbc/видео.html
How'z authentication & Authorization implemented in this model ?
I haven't found many talks/examples of DDD-based microservices, if you will, that detail auth. If anybody has any, I'd appreciate some references to them.
If you mean api auth then the api gateway handles it either by itself or via service
Security should externalized if you are architect then you should know this basic thing
I have only one question if we hit the service directly will it
redirect to api gateway automatically? if we need to make it to work what
should be done?
"is that a question? do you have a burning question?".....lawl...dude must of had his hand up for long time interrupting the presentation
21:50 lmao this guy
change speed to 1.25
thanks me later
second half of the video is all about event sourcing: github.com/cer/event-sourcing-examples
Code Smell? Can simple concepts not be explained with convoluted cosmetic English that sound dead boring.
Optimistic locking have nothing to do with transaction. Also it's not a bad situation, it's a good situation. Normally order processing systems work in a first come-first served manner, so you HAVE to have optimistic locking, but again optimistic locking is only about timestamp comparison prior to updating the data in the db. It doesn't have anything to do with transactions.
nope
Aggregation is coupling by definition. Congratulations, you just made an even more brittle solution.
Aggregates are modeling the inherent coupling in the business domain.
But that may not have been explained very well in the presentation.
lol such a funny guy
What a load of trash on so many levels. It is unfortunate that this atrocity has gotten traction on a lot of places.
Very useful video but his voice is so monotonous that puts me to sleep.
Isn't it just SAGA ? Why people are creating their own version ? BTW two time I fell to sleep during this video.
It's TCC
Full of waste time trying to solve fundamental issues of micro services by introducing bunch of other new sever issues. You counted how many drawbacks you have and called them simple? Did you try to cover how to solve each or some at least? When you were going with aggregate description as a unit, and then moved to modularized aggregate and granularity level, you lost me again going from aggregate to entity again.
I can not estimate quality of material, cause speech manner is just awful. Its extremely boring to hear this robo-type voice
very boring talk
Great talk!