Looking for books & other references mentioned in this video? Check out the video description for all the links! Want early access to videos & exclusive perks? Join our channel membership today: ruclips.net/channel/UCs_tLP3AiwYKwdUHpltJPuAjoin Question for you: What’s your biggest takeaway from this video? Let us know in the comments! ⬇
Mr Fowler has that enviable energy of a school principal in a movie who acts serious around the grownups but helps the kids with all the crazy shenanigans behind the scenes.
That's how he counter balances Bob Martin, whose like that uncle who has an awesome story he never has time to finish explaining. still waiting for him to get through all of SOLID in one video
*Timed index of the video* 0:00 What people mean by EDA 00:51 How he came to write down common patterns of EDA and hold this talk 02:45 4 patterns detected People normally call their system "event-driven" if at least one of those patterns is in place 03:20 *Pattern 1: Event Notification* Generic code notifies specific code by events for example GUI elements 08:33 Events vs Commands Events just indicate a change without expecting any particular response Commands are telling some service what to do 11:30 Pro: Decoupling 13:50 Contra: inability to understand what is going on by stepping through the code 14:53 *Pattern 2: Event-carried State Transfer* Subscribers don't ask for additional information after an event occured, all necessary state is given in the events. Subscribers copy whatever they need. Pro: can greatly reduce network traffic. Better availability because of the copied data Contra: consistency It's Less used 20:51 *Pattern 3: Event Sourcing* Ability to rebuild the full state of the system by a persisted log of events Event Sourcing works with your data like version control systems work with your code Pro: "time traveling" like for debugging. use a copy of the system, feed it the events and see what happened 32:11 Can be a very nice system development-wise 33:43 Downside of Event Sourcing - unfamiliar - when I rebuild the state from the log I can't replay answers of external systems unless I record all answers as kind of events - old event schema still has to work somehow with new versions of code - IDs that are generated during replay are probably different than before - Asynchrony is difficult for people (but it's optional. You can synchronously use event sourcing) - Versioning (Snapshots can make that easier) (Conclusion for me: Always remember that replay has to work for everything, otherwise there is no point in using event sourcing) 38:46 Which events to record in the event store? Two events happen for a state change. One shows the intention (command), the other shows all the different changes that were done. Events that show intention often hold specific knowledge, the EventStore normally should stay generic (git only knows txt, not the used programming language syntax) It's important to think about which of those events are important to persist. Probably both. 43:31 *Pattern 4: CQRS* Separate components for updating the system and reading from the system Different views of the data are pretty common (e.g. a reporting database), the important thing here is that the command model is never used for reading from outside 47:39 *Conclusion* : How to use the knowledge about those 4 patterns When you hear about a system being "event-driven" ask questions to find out what patterns they use. Because the term "event-driven" is too imprecise. Have enough knowledge about each pattern to know where the problems are and what consequences they cause.
Mr. Fowler is not only an excellent writer and knowledgeable on the subject, but he is also an excellent speaker and explains many things eloquently. Thank you!
Martin Fowler, thanks a whole lot for your contributions to the Industry. I also love the way you convey knowledge. Love the humor too. Wondering why folks in attendance didn't even giggle. Thanks again and always.
First class thingamajig deserves to be recognized for it's genius. I tend to explain event sourcing to people as similar to a Git repository. I believe that CQRS is useful for optimization, particularly when considering an ORM.
Great talk that puts a structure over the broad topic. There's one minor issue with using Git as example of Event Sourcing: Git actually stores snapshots of the whole state, instead of changes.
34:11 The event schema issue mentioned here goes away once you start using semantic data for recording events (in formats such as JSON-LD), because in case of semantic data there is no schema to modify, only new vocabularies to add to the vocabularies already in use. Software that read/write events only have to add support for the new vocabularies while remaining 100% compatible with the events previously published.
Nowadays when I read blogs about microservices, "event carried state transfer" (with CQRS) is presented as an absolute necessity 95% of the time, which I think is severely violating "right tool for the right job" for the sole purpose of consultants being able to consult on an unnecessarily difficult topic.
This was rather useful. Seems to me like the 1) Even notification and reversal of dependency and the 3) Event sourcing (nothing specific to events actually. Any journaled system does it) .... are the most important here. The state replication use case is definitely something to be careful about... it opens up a can of worms of consistency problems with home-brew distributes state machines. Also ... It seems a lot of systems promoting them selves as event-driven using message-queues and such could really do with ordinary async REST callbacks, since what they are doing is actually "commands" and not "events". The event "queue" model seems most useful when you don't know what should be done about an event or who should do it - or when you have realtime or broadcast aspects where throughput is more important than dropped messages. (event streaming). If you are just sending jobs of to someone to execute, you can really do with just maintaining a "saga" state of the transaction and have it updated by async REST-like HTTP calls between services. No need for a queue.
Nice talk overall, but I would argue on a couple of aspects: - events/commands: the difference between Event and Command is really deeper than just naming. IMO it's the same difference that occurs between "choreography" and "orchestration". Moreover I would say the kind of queues used for the 2 types of "messaging" have different requirements: message-persistence/multiple-consumption in case of Events (eg. I don't know who and when will be interested in this event, an I would use something like Kafka for it), message-acknowledgment/single-consumption in case of Commands (eg. I am giving an instruction right now to a specific executor of that instruction (actually the executor I could not know but the nature of the action I know), and I would maybe use something like RabbitMQ) - all these patterns put their emphasis on different architectural layers: events/commands are about messaging and communication (being it in the UI or on the server), eventsourcing is somewhat more about the data persistency strategy and requirements (having as a possible beneficial side-effect the "messaging" itself, but not necessarily), cqrs is more about the interfaces to the system/domain, about the kind of domain I need to manage and the strategies used to manage it (I see it more as a counterpart to the rest/crud simplification to manage domains where state-change is more complex). So I think one could use any|none|all of the patterns presented, and I would pull up cqrs a lot in the toolbox of the choices, as recognising the complexity of a domain is the first step towards simplifying the way of managing it
mettjus None of what you said seems off to me, but he pretty clearly explained the conceptual difference between them in the talk, according to his own point of view.
You said that you would choose cqrs for most of the time but I would argue with that statement being the right way to go with. Most of the projects don't even fit well and there are some important things that should be considered before using it because the cons and complexity this architecture brings (atomicity, event versioning, failure management, decreased productivity etc. etc.) is not proportional to the benefits it brings..
I've always stayed clear from (2) Event State Transfer because of the issues with not having idempotent behavior. Passing in state to your messages means you need to process them in order. That means you have to be careful about scaling horizontally.
A bit of a necro of your post but I'm not sure that's globally true. Depending on how you publish your state means you dont have to process them in order. For example if you are publishing a snapshot at any given time and you've clustered your subscriptions for say Added/Updated/Deleted and are checking the timestamp on your message - you can pretty much bin the events that came late unless you want them for audit reasons. Better still if you were an event store behind your subscriptions you could inject the message into the stack and reproject/materialise your store into the state that you want. I have used this pattern a lot in the last 10 years and because of the speed you can process messages now, eventual consistency is less and less a concern. We had a lot customers wanting to turn off sections of our app and replacing it with their own so this was pretty much a must because you never knew what was producing events/commands to your service only that you should still be able to function.
The more I study the subject, the more I must conclude that the distinction should not be made along the lines of 'event-driven' and 'not-event-driven'; but rather between 'imperative' and 'reactive' system design. In the 'imperative' corner I would put Explicit Command Execution, Event Notification and Event Carried State Transfer; since the latter are in fact commands in disguise, HOPING to mutate the persisted state of the system. Which leaves only Event Sourcing/CQRS as truly reactive or 'event-driven'; resulting in a stateless system where a certain state is merely the result of a certain interpretation (or projection) of FACTS.
Payroll example is good because of principals of accounting system as such. Balance sheet is a snapshot of financial state of a company. Each business operation is reflected as accounting entries according to the pattern to the debits and credits, which changes the assets and liabilities. Accounting is build with system approach.
versioning of events is an order of magnitude easier than traditional database migrations. Some companies can't even upgrade their excusing customers to their newest versions in traditional systems.
This is a great talk, and has led me to some helpful incites. Like most commenters though, I'm only going to point out what I don't like. I’m having trouble digesting the notion that Command messages are going to make reasoning about the system any easier than Event Notification. I think that has more to do with asynchronous operations than with the names and expected behavior of messages.
Events are Signals that can be consumed by multiple consumers, and where a consumer will typically only source from a single producer. Command are Remote Procedure Calls that are consumed by one consumer and a consumer is typically fed by multiple producers. His example of an Email is a good one, you want 1 email to go out, you have multiple producers that want to send emails to an SMTP process - a command.
Commands are requests for something to happen. An Event is what has already happened. For example: a user might Command the system to save a file. The system might then generate an Event "the file was saved" or an Event "error saving file".
It strikes me that, in terms of the first example of an event-driven architecture, it'd be possible to get around the highlighted weakness with the notion of child events. If an event triggers other events to occur, then the subsequent events could be described as child events (and conversely, if an event was not driven by another event, it's a parent event). This could all be captured in the same event stream (parent events, child creation events and child completion events), so that you could have a fuller picture of what happened as a result of a single action in a software system.
While this is definitely true and possibly a great way to improve visibility. It's basically distributed tracing. However, it does still require all related applications to be running and then executing the flow that you are looking to gain insights into. So not as simple/straightforward as just looking at the code and seeing what is being called.
@@thijsriezebeek790 In the case of distributed services, it should be possible to monitor for any network calls that triggered an event, and assuming you have one central "logging service" that all other services write their logs to, it should also be possible to link this network activity with the parent process. So for example, if you have a service listening on port 5000 on PC 1, if PC 2 makes a network call to PC 1 on port 5000 you can see the both the parent event behind that network call and the child event that resulted from that network call (if the call resulted in a child event being spawned).
Hmm.. this Event carried state transfer or whatever was implemented almost a year ago at our end. You can define the criteria at your model itself which is then checked against the state during RESTful CRUD operation. So the event can be defined for Cr, U or D even apart from just U as in the example here. Of course the attributes can also be defined, so that a copy of it (such as customer's address) is sent along with the event trigger instead of a copy of the entire instance.
Also the 'command' and 'event' naming stuff is unnecessary IMHO. If its a command, then wrap it in the transaction itself for the Aggregate. Else raise the event which is then processed async
I get the feeling that the unspoke thing about event is that it is an output. A system outputs events and another system watch this outputs and then decide what to do (a listener)
An event, it appears to me, is a declaration of an activity. Something has happened. A command is a declaration of a request, something needs to happen. Well, that's the way I see it
There are still teams who are working on new projects that don't know how to use Event-driven Architectures to its fullest. Events are an afterthought when they need to communicate between "microservices" in a decoupled way - when they cannot use Grpc, that is. The only reasonable thing would be to go full Async Messaging.
1) If your event name has "Changed" it is a bit of a worry that you are still in an OO or model centric mindset. Address should be a property of the the actual event (maybe "Client Moved") that the new address comes from. 2) I don't think events should cross domain boundaries as the "event" may have different meanings to different domains - domains should talk to each other by commands (Do [X])/queries (What is [X])
@@lepidoptera9337 Not even close. If it was common sense, or programming 101, then I wouldn't spend my days dealing with coding horrors. Only an idiot would take the subject lightly.
@@7th_CAV_Trooper Admittedly, this idiot has only himself to blame if the code doesn't work. I am a team of one. Always have been, always will be. I wouldn't want it any other way.
Very nice. To me no such thing like CQRS should operate on Event level. CQRS is pure data manipulation, extension of CRUD concept as I see it. It operates within Event, but once Event execution is finished, it is gone and only Event notification is published. CQRS talks about commands and queries (request and responses), but there is no space for any kind of request or responses in event driven design, it is just sequence of independent events. Event driven with event-sourcing is by fun finally really merging together the middleware with BPM engine together :-)) I even design full UI as event driven service (no REST like stuff), things are designed in sequences (supporting spliting, aggregating, merging within service). So during event storming sessions I define business process, while I implement it independently. The only "coupling/contracting" elements are DTOs which represents the dependencies. This is the space for careful process of sharing these "languages" to others.
7:30 "First class thingimijig" is definitely a good name, but a name that already exists for this, which completely captures the semantics, is "data structure". We've turned a function call with transient arguments into a data structure in its own right. 9:00 if we phrase our data structure in the command form (the event content says "requote the customer"), then we have reintroduced the dependency, since the customer system must now know that the quoting system exists in order to be able to send that command. The decoupling only exists if using the pure event system. So this is not just a naming issue (although it is also that). If the reason for the event architecture is to invert the dependency, then commands simply cannot be used.
@@JamesSmith-cm7sg "We not pass a data structure"? Uh, yes we pass a data structure. Any time you have any kind of data in a computer, it's a data structure of some kind. It might be a primitive, like an int, or it might be complex, like a struct, or a class. If you are moving data around, then you are moving data structures around. If you are passing around an event which says "a customer changed and here is the changed data", then the "first class thingimijig" that you are passing around is a data structure which you might call a "CustomerChange". No, the second part of my comment does not say exactly what he said. If you pass a command to a remote system telling it what it needs to do, then you need to know what the remote system needs to do when a certain event occurs, which means you are more tightly coupled to the remote system. If, on the other hand, you merely pass the data which describes the event as I suggested, for example, to some kind of pub/sub broker, and do not pass it as a command directly to a remote system, then you do not need to make any assumptions about the remote system, and so the systems are more loosely coupled.
@@allmhuran What you said is moronic. To raise an event or command you don't always need to provide data, the name / key can often be enough, so calling everything a "data structure" doesn't work. And then you go on to explain the basics of event vs command, which you somehow think he doesn't understand....
I wish he had explained the part about what is recorded as events in Event sourcing (~ from 38:50) and cqrs pattern a bit better - I did not really understand those parts.
Very helpful, contributed a lot to my understanding of event-drive architecture, especially the event-sourcing part. However, I do not see how CQRS is a event-driven pattern. Seems to me this pattern can also be used for a point-to-point / request-response architecture. Am I wrong?
CQRS is all about separating your read model from your write model, typically resulting in multiple read models that live in completely separate data stores. Events are crucial to communicating state changes to the system so that your read models will stay up to date. Event Sourcing (which != EDA) is not technically required for persistence, but I would say is generally preferred.
finally Martin mentions Greg Young with CQRS. though He's still weary of CQRS. sigh. you can "cheat" and use the event stream as a read model. so many issues still with what he presents. I guess he has to be safe so the existing clients of TW don't get spooked.
There's ability for Redux to have event sourcing but basically it's more about a snapshot. What's great is that you can really build a snapshot with the list of actions dispatched.
Redux is command sourcing, not event sourcing. Minor technicality but in a full application is important to understand (because you have to deal with commands being able to fail/rollback whereas events will never change as they are in the past)
Great talk! Learned quite a lot. I think CQRS needs a bit more explaining compared to the reporting database. My understanding was the data source of the read side of the UI will be served via a different view than the one built entirely for the reporting purposes? Because, reports are targeted for different audience who needs it for analysis, export and other purposes.. But the read part of CQRS needs to serve the same screen where the user is interacting with the application.. so I think there is a subtle difference?
The difference is that the write side isn’t read from. So in a CQRS event sourcing architecture, your event store is the only thing being written to. You never read from that event store. Instead you create read views by “fast forwarding” events to get the current state. If you need to separate ui service from reporting, you would create two read views that are each built and updated by pulling new events off the event store.
Awesome. Losing traceablity while making easy to add new functionality is true. Why not Use event ids to link events with the code (and the subsequent events) that execute due to those events?
This will help you during debugging, but it does not help you during design/coding. What he says is: You need runtime information to understand how the overall process/algorithm works. TracingID in Logfiles are exactly that: Runtime information. Without them its relatively hard to figure out what will happen.
He states that a event is something that simple have happened without the emitter knowing what will be done, a command on other hand is the emitter explicit telling what have to be done.
Very good talk. The only issue I have to understand, how to implement Event Sourcing in DB. As Martin said, I should be able to rebuild state from log at any time. This means for me, that I should create new record in log table everytime somethings have changed, and this record should contain all record data. The other approach is to just store event messages like "address created", "address updated" and have one row in db that is being updated, but then there won't be any history available. Any idea on this problem?
It simply means that you create a new row for every incoming request in log table with all data and process that log in synchronous manner to update the actual entity.
It really depends. I may not care about my listeners or I may care and expect a response asynchronously, thus you can still send the message as an event, not as a command: imagine you need to create a payment everytime you create an order, you can notify whenever an order is created and expect someone to listen to it and notify another event when the payment is made. That's up to you if you want to notify as "order created" or "create payment". I really prefer the first approach.
IS CQRS same as Data normalization good for insert/update/delete whereas a materialized predefined refreshed view for queries. That is what Oracle has been doing long ago
I would strongly disagree that events and commands are the same and only differ in their names. Think of Laws, there you have incidents and consequences. Often it is the same in IT systems: the subsystem indicating a situation (event, change, whatever) can not know what the readers of the indication will do or have to do then. The cause can not know the effect, at least not in advance.
#3 seems out of place in the list as it mainly deals with internals: how one particular component/service represents its own data. whereas all the others are strategies for how one component can talk to others.
#2 shows a diagram where multiple events are sourced from different systems to build a local snapshot. #3 is a superset of what kind of event messages those should be. #4 doesn't necessarily apply to Event-Driven as he says in his blog. So there aren't really 4, but they make for good headings to stimulate discussion.
Thanks for your comment. We'd love to see you at one of our 2019 conferences! Chicago - April 28 - May 2; Berlin - Oct 21-25; Amsterdam - June 17-20; Copenhagen - Nov 18-22 See more details about each here: blog.gotocon.com
I would like to really see some real examples about how to monitor fact that the event-drvien platform is healthy. Imagine UI is event driven, on startup it just sends event to bus "Hi, I am new UI instannce in the wild", multiple services are waiting for such events and start pumping Process/Taks/JobViewPreparedEvents (even in form of full data tables to be displayed), I have no problem on event having the data inside. But how to monitor that it is all healthy??? I can imagine only something which knows the full event seqeucne and can monitor for example timeouts based on event database as timeseries data. But I wasn't able to find any kind of pattern or how to do health checks. Thanks for any comments to event driven monitoring
The pattern is to ditch events. As mentioned in the talk, you lose the ability to monitor what's going on when you use the first 2 event types. after all, events are "I don't want to understand what's happening" and what you describe as 'health checks' sounds like "I want to understand what's happening".
How does CQRS fit in here? I don't see a relation to events. There are commands and queries alright but neither of those have any of the properties attributed to events.
Events and Commands the same thing? Wouldn't an event flip the coupling (an event occurred) and a Command tighten the coupling (you do this)? Didn't see below, but I think I agree with mettjus.
Actually it's very clear where the difference is between "event" and "command". An "event" (in terms of event sourcing) is ALWAYS something in the past = it has happened already! Quite contrary to a "command" which means something has to be done (with that command) = it is UPCOMING, and not in the past.
@@heiko3169 In the home-grown SDK that I maintain; I decided that commands and events are internal object to the service, that are streamed to serializable "Hope" and "Fact" objects respectively. Using this nomenclature has the effect that people get a much more clearly defined concept.
Is internal communication within a living organism ( lets say Human body in specific ) based on EDA. If yes how does the 4 patterns described in this video correlate to the communications within a human body.
Talk derived from his talk here: martinfowler.com/articles/201701-event-driven.html What do you mean by “Event-Driven”? Photo of Martin Fowler Martin Fowler 07 February 2017
Looking for books & other references mentioned in this video?
Check out the video description for all the links!
Want early access to videos & exclusive perks?
Join our channel membership today: ruclips.net/channel/UCs_tLP3AiwYKwdUHpltJPuAjoin
Question for you: What’s your biggest takeaway from this video? Let us know in the comments! ⬇
Mr Fowler has that enviable energy of a school principal in a movie who acts serious around the grownups but helps the kids with all the crazy shenanigans behind the scenes.
That's how he counter balances Bob Martin, whose like that uncle who has an awesome story he never has time to finish explaining.
still waiting for him to get through all of SOLID in one video
After watching a bunch of talks where people clearly don't understand these systems, Martin Fowler truly does.
*Timed index of the video*
0:00 What people mean by EDA
00:51 How he came to write down common patterns of EDA and hold this talk
02:45 4 patterns detected
People normally call their system "event-driven" if at least one of those patterns is in place
03:20 *Pattern 1: Event Notification*
Generic code notifies specific code by events
for example GUI elements
08:33 Events vs Commands
Events just indicate a change without expecting any particular response
Commands are telling some service what to do
11:30 Pro: Decoupling
13:50 Contra: inability to understand what is going on by stepping through the code
14:53 *Pattern 2: Event-carried State Transfer*
Subscribers don't ask for additional information after an event occured, all necessary state is given in the events.
Subscribers copy whatever they need.
Pro: can greatly reduce network traffic. Better availability because of the copied data
Contra: consistency
It's Less used
20:51 *Pattern 3: Event Sourcing*
Ability to rebuild the full state of the system by a persisted log of events
Event Sourcing works with your data like version control systems work with your code
Pro: "time traveling" like for debugging. use a copy of the system, feed it the events and see what happened
32:11 Can be a very nice system development-wise
33:43 Downside of Event Sourcing
- unfamiliar
- when I rebuild the state from the log I can't replay answers of external systems unless I record all answers as kind of events
- old event schema still has to work somehow with new versions of code
- IDs that are generated during replay are probably different than before
- Asynchrony is difficult for people (but it's optional. You can synchronously use event sourcing)
- Versioning (Snapshots can make that easier)
(Conclusion for me: Always remember that replay has to work for everything, otherwise there is no point in using event sourcing)
38:46 Which events to record in the event store?
Two events happen for a state change. One shows the intention (command), the other shows all the different changes that were done.
Events that show intention often hold specific knowledge, the EventStore normally should stay generic (git only knows txt, not the used programming language syntax)
It's important to think about which of those events are important to persist. Probably both.
43:31 *Pattern 4: CQRS*
Separate components for updating the system and reading from the system
Different views of the data are pretty common (e.g. a reporting database), the important thing here is that the command model is never used for reading from outside
47:39 *Conclusion* : How to use the knowledge about those 4 patterns
When you hear about a system being "event-driven" ask questions to find out what patterns they use. Because the term "event-driven" is too imprecise.
Have enough knowledge about each pattern to know where the problems are and what consequences they cause.
Thank you 👌
Thank you
merci !
obrigado!
awesome guy
Love listening to this guy. Pure gem. How i wish I would have had the honor of starting my career under his guidance as a direct reportee to him.
Mr. Fowler is not only an excellent writer and knowledgeable on the subject, but he is also an excellent speaker and explains many things eloquently. Thank you!
Martin Fowler, thanks a whole lot for your contributions to the Industry. I also love the way you convey knowledge. Love the humor too. Wondering why folks in attendance didn't even giggle. Thanks again and always.
Autista have a tough time with irony and humor
First class thingamajig deserves to be recognized for it's genius.
I tend to explain event sourcing to people as similar to a Git repository.
I believe that CQRS is useful for optimization, particularly when considering an ORM.
could listen to the guy all day
Great talk that puts a structure over the broad topic.
There's one minor issue with using Git as example of Event Sourcing: Git actually stores snapshots of the whole state, instead of changes.
I love that for the event sourcing part he became Greg Young and started asking "How many of you" for everything, nice touch
I think the talk was pretty clear and enriching. Thank you Mr Fowler.
34:11 The event schema issue mentioned here goes away once you start using semantic data for recording events (in formats such as JSON-LD), because in case of semantic data there is no schema to modify, only new vocabularies to add to the vocabularies already in use. Software that read/write events only have to add support for the new vocabularies while remaining 100% compatible with the events previously published.
Nowadays when I read blogs about microservices, "event carried state transfer" (with CQRS) is presented as an absolute necessity 95% of the time, which I think is severely violating "right tool for the right job" for the sole purpose of consultants being able to consult on an unnecessarily difficult topic.
This was rather useful.
Seems to me like the 1) Even notification and reversal of dependency and the 3) Event sourcing (nothing specific to events actually. Any journaled system does it) .... are the most important here.
The state replication use case is definitely something to be careful about... it opens up a can of worms of consistency problems with home-brew distributes state machines.
Also ... It seems a lot of systems promoting them selves as event-driven using message-queues and such could really do with ordinary async REST callbacks, since what they are doing is actually "commands" and not "events".
The event "queue" model seems most useful when you don't know what should be done about an event or who should do it - or when you have realtime or broadcast aspects where throughput is more important than dropped messages. (event streaming).
If you are just sending jobs of to someone to execute, you can really do with just maintaining a "saga" state of the transaction and have it updated by async REST-like HTTP calls between services. No need for a queue.
6:15 - those are wise words to live by.
Finally, I've got an understanding of this topic.
Found myself going for an applause when he finished 😁
Nice talk overall, but I would argue on a couple of aspects:
- events/commands: the difference between Event and Command is really deeper than just naming. IMO it's the same difference that occurs between "choreography" and "orchestration". Moreover I would say the kind of queues used for the 2 types of "messaging" have different requirements: message-persistence/multiple-consumption in case of Events (eg. I don't know who and when will be interested in this event, an I would use something like Kafka for it), message-acknowledgment/single-consumption in case of Commands (eg. I am giving an instruction right now to a specific executor of that instruction (actually the executor I could not know but the nature of the action I know), and I would maybe use something like RabbitMQ)
- all these patterns put their emphasis on different architectural layers: events/commands are about messaging and communication (being it in the UI or on the server), eventsourcing is somewhat more about the data persistency strategy and requirements (having as a possible beneficial side-effect the "messaging" itself, but not necessarily), cqrs is more about the interfaces to the system/domain, about the kind of domain I need to manage and the strategies used to manage it (I see it more as a counterpart to the rest/crud simplification to manage domains where state-change is more complex). So I think one could use any|none|all of the patterns presented, and I would pull up cqrs a lot in the toolbox of the choices, as recognising the complexity of a domain is the first step towards simplifying the way of managing it
mettjus None of what you said seems off to me, but he pretty clearly explained the conceptual difference between them in the talk, according to his own point of view.
You said that you would choose cqrs for most of the time but I would argue with that statement being the right way to go with. Most of the projects don't even fit well and there are some important things that should be considered before using it because the cons and complexity this architecture brings (atomicity, event versioning, failure management, decreased productivity etc. etc.) is not proportional to the benefits it brings..
I would say the key difference is in how the obligation is created.
You can name the event in a login screen either "LoginButton_Clicked" or "Perform Login".
mettjus Really appreciating this comment
Datomic solves the schema versioning by treating the schema as data: There is only one Universal schema. It's wonderful.
his scarf game is on fleek
I first thought it's a tattoo, until i continued watching on a big screen.
You win the entire thread!
martin would make an excellent 'what a twist?!' evil villian
This needs more likes
His whole outfit matches the presentation theme. I'm worried/impressed that this might not be an accident.
I've always stayed clear from (2) Event State Transfer because of the issues with not having idempotent behavior. Passing in state to your messages means you need to process them in order. That means you have to be careful about scaling horizontally.
Great call out. This should get added to his slides. However, I would argue all async activity suffers from this.
A bit of a necro of your post but I'm not sure that's globally true. Depending on how you publish your state means you dont have to process them in order. For example if you are publishing a snapshot at any given time and you've clustered your subscriptions for say Added/Updated/Deleted and are checking the timestamp on your message - you can pretty much bin the events that came late unless you want them for audit reasons. Better still if you were an event store behind your subscriptions you could inject the message into the stack and reproject/materialise your store into the state that you want. I have used this pattern a lot in the last 10 years and because of the speed you can process messages now, eventual consistency is less and less a concern. We had a lot customers wanting to turn off sections of our app and replacing it with their own so this was pretty much a must because you never knew what was producing events/commands to your service only that you should still be able to function.
The more I study the subject, the more I must conclude that the distinction should not be made along the lines of 'event-driven' and 'not-event-driven'; but rather between 'imperative' and 'reactive' system design. In the 'imperative' corner I would put Explicit Command Execution, Event Notification and Event Carried State Transfer; since the latter are in fact commands in disguise, HOPING to mutate the persisted state of the system. Which leaves only Event Sourcing/CQRS as truly reactive or 'event-driven'; resulting in a stateless system where a certain state is merely the result of a certain interpretation (or projection) of FACTS.
Payroll example is good because of principals of accounting system as such. Balance sheet is a snapshot of financial state of a company. Each business operation is reflected as accounting entries according to the pattern to the debits and credits, which changes the assets and liabilities. Accounting is build with system approach.
versioning of events is an order of magnitude easier than traditional database migrations. Some companies can't even upgrade their excusing customers to their newest versions in traditional systems.
This is a great talk, and has led me to some helpful incites. Like most commenters though, I'm only going to point out what I don't like. I’m having trouble digesting the notion that Command messages are going to make reasoning about the system any easier than Event Notification. I think that has more to do with asynchronous operations than with the names and expected behavior of messages.
Events are Signals that can be consumed by multiple consumers, and where a consumer will typically only source from a single producer. Command are Remote Procedure Calls that are consumed by one consumer and a consumer is typically fed by multiple producers. His example of an Email is a good one, you want 1 email to go out, you have multiple producers that want to send emails to an SMTP process - a command.
Commands are requests for something to happen. An Event is what has already happened. For example: a user might Command the system to save a file. The system might then generate an Event "the file was saved" or an Event "error saving file".
Morten Brodersen the command is also an event - "file save requested"
Martin Fowler, another great insight(s) that you shared here. Thank you
It strikes me that, in terms of the first example of an event-driven architecture, it'd be possible to get around the highlighted weakness with the notion of child events. If an event triggers other events to occur, then the subsequent events could be described as child events (and conversely, if an event was not driven by another event, it's a parent event). This could all be captured in the same event stream (parent events, child creation events and child completion events), so that you could have a fuller picture of what happened as a result of a single action in a software system.
While this is definitely true and possibly a great way to improve visibility. It's basically distributed tracing. However, it does still require all related applications to be running and then executing the flow that you are looking to gain insights into. So not as simple/straightforward as just looking at the code and seeing what is being called.
@@thijsriezebeek790 In the case of distributed services, it should be possible to monitor for any network calls that triggered an event, and assuming you have one central "logging service" that all other services write their logs to, it should also be possible to link this network activity with the parent process. So for example, if you have a service listening on port 5000 on PC 1, if PC 2 makes a network call to PC 1 on port 5000 you can see the both the parent event behind that network call and the child event that resulted from that network call (if the call resulted in a child event being spawned).
What I find really interesting - this is how those of us in process control have been developing software since the early 80s.
does process control involve distributed systems?
I designed an Event-Carried State Transfer system 3 years ago and it works very well until now. ;)
what happened?
@@SpittingMage I think he meant "it has worked fine ever since".
I love the way he describe stuff
Great talk, had some aha moments. 💡
Great. 1st micro service talk which doesn’t just push Java or Kafka !
great and clear explanation, thanks!
Thank you, your talk is really good.
Thanks for this great talk. I could learn a lot from this talk.
Hmm.. this Event carried state transfer or whatever was implemented almost a year ago at our end. You can define the criteria at your model itself which is then checked against the state during RESTful CRUD operation. So the event can be defined for Cr, U or D even apart from just U as in the example here. Of course the attributes can also be defined, so that a copy of it (such as customer's address) is sent along with the event trigger instead of a copy of the entire instance.
Also the 'command' and 'event' naming stuff is unnecessary IMHO. If its a command, then wrap it in the transaction itself for the Aggregate. Else raise the event which is then processed async
I wonder if Martin had a look at Datomic, the (immutable) functional database with various NoSQL DBs as pluggable data store
I "said" Datomic instead of Git. LOL
Shortcomings of event driven application architecture is visible when you see Database triggers rarely being used for serious stuffs.
I get the feeling that the unspoke thing about event is that it is an output.
A system outputs events and another system watch this outputs and then decide what to do (a listener)
Many events from one component are implicitly commands for another component.
An event, it appears to me, is a declaration of an activity. Something has happened. A command is a declaration of a request, something needs to happen. Well, that's the way I see it
There are still teams who are working on new projects that don't know how to use Event-driven Architectures to its fullest. Events are an afterthought when they need to communicate between "microservices" in a decoupled way - when they cannot use Grpc, that is. The only reasonable thing would be to go full Async Messaging.
1) If your event name has "Changed" it is a bit of a worry that you are still in an OO or model centric mindset. Address should be a property of the the actual event (maybe "Client Moved") that the new address comes from.
2) I don't think events should cross domain boundaries as the "event" may have different meanings to different domains - domains should talk to each other by commands (Do [X])/queries (What is [X])
Gem of a talk!
"after all, names in software systems are one of the most important things you have to deal with." - That could be an entire lecture series.
Yes, it's a series called "The idiot's guide to programming". Come on, folks, if you can't get past that level, go back to your marbles.
@@lepidoptera9337 Not even close. If it was common sense, or programming 101, then I wouldn't spend my days dealing with coding horrors. Only an idiot would take the subject lightly.
@@7th_CAV_Trooper Admittedly, this idiot has only himself to blame if the code doesn't work. I am a team of one. Always have been, always will be. I wouldn't want it any other way.
@@lepidoptera9337 even if you work alone you need to.. never mind, you're too far on the spectrum to get it.
@@7th_CAV_Trooper Oh, I get it. I just never had to work with amateurs. Why are you? Are you an amateur yourself?
Very nice. To me no such thing like CQRS should operate on Event level. CQRS is pure data manipulation, extension of CRUD concept as I see it. It operates within Event, but once Event execution is finished, it is gone and only Event notification is published. CQRS talks about commands and queries (request and responses), but there is no space for any kind of request or responses in event driven design, it is just sequence of independent events. Event driven with event-sourcing is by fun finally really merging together the middleware with BPM engine together :-)) I even design full UI as event driven service (no REST like stuff), things are designed in sequences (supporting spliting, aggregating, merging within service). So during event storming sessions I define business process, while I implement it independently. The only "coupling/contracting" elements are DTOs which represents the dependencies. This is the space for careful process of sharing these "languages" to others.
7:30 "First class thingimijig" is definitely a good name, but a name that already exists for this, which completely captures the semantics, is "data structure". We've turned a function call with transient arguments into a data structure in its own right.
9:00 if we phrase our data structure in the command form (the event content says "requote the customer"), then we have reintroduced the dependency, since the customer system must now know that the quoting system exists in order to be able to send that command. The decoupling only exists if using the pure event system. So this is not just a naming issue (although it is also that). If the reason for the event architecture is to invert the dependency, then commands simply cannot be used.
We might not pass a data structure.
And on the second part of your comment, you said exactly what he said.
@@JamesSmith-cm7sg "We not pass a data structure"? Uh, yes we pass a data structure. Any time you have any kind of data in a computer, it's a data structure of some kind. It might be a primitive, like an int, or it might be complex, like a struct, or a class. If you are moving data around, then you are moving data structures around. If you are passing around an event which says "a customer changed and here is the changed data", then the "first class thingimijig" that you are passing around is a data structure which you might call a "CustomerChange".
No, the second part of my comment does not say exactly what he said. If you pass a command to a remote system telling it what it needs to do, then you need to know what the remote system needs to do when a certain event occurs, which means you are more tightly coupled to the remote system. If, on the other hand, you merely pass the data which describes the event as I suggested, for example, to some kind of pub/sub broker, and do not pass it as a command directly to a remote system, then you do not need to make any assumptions about the remote system, and so the systems are more loosely coupled.
@@allmhuran
See previous comment
@@JamesSmith-cm7sg Oh, you're a moron. Nevermind then.
@@allmhuran
What you said is moronic.
To raise an event or command you don't always need to provide data, the name / key can often be enough, so calling everything a "data structure" doesn't work.
And then you go on to explain the basics of event vs command, which you somehow think he doesn't understand....
Loved when he started to run to wake up camera man LOL!
He is great! Thank you for uploading!
I wish he had explained the part about what is recorded as events in Event sourcing (~ from 38:50) and cqrs pattern a bit better - I did not really understand those parts.
Such a great speake...the content is really nice
Insightful video 🙌
very insightful and clear
Extremelly good speech! Keep on doing!
Very helpful, contributed a lot to my understanding of event-drive architecture, especially the event-sourcing part. However, I do not see how CQRS is a event-driven pattern. Seems to me this pattern can also be used for a point-to-point / request-response architecture. Am I wrong?
CQRS is all about separating your read model from your write model, typically resulting in multiple read models that live in completely separate data stores. Events are crucial to communicating state changes to the system so that your read models will stay up to date. Event Sourcing (which != EDA) is not technically required for persistence, but I would say is generally preferred.
@@michaelclark4283 pdf
Vicp so
X o ): a and oo jn knbc PBS NBC bv k nov lpcvoc office
You are correct. Fowler says this in his blog - CQRS is not an event-driven pattern.
finally Martin mentions Greg Young with CQRS. though He's still weary of CQRS. sigh. you can "cheat" and use the event stream as a read model. so many issues still with what he presents. I guess he has to be safe so the existing clients of TW don't get spooked.
Notes to myself : Version Control system uses 'Event Sourcing'
As well as Redux does.
redux doesn't persist events story, just the latest state processed by the reducers
There's ability for Redux to have event sourcing but basically it's more about a snapshot.
What's great is that you can really build a snapshot with the list of actions dispatched.
Redux is command sourcing, not event sourcing. Minor technicality but in a full application is important to understand (because you have to deal with commands being able to fail/rollback whereas events will never change as they are in the past)
Great talk! Learned quite a lot.
I think CQRS needs a bit more explaining compared to the reporting database. My understanding was the data source of the read side of the UI will be served via a different view than the one built entirely for the reporting purposes?
Because, reports are targeted for different audience who needs it for analysis, export and other purposes..
But the read part of CQRS needs to serve the same screen where the user is interacting with the application.. so I think there is a subtle difference?
The difference is that the write side isn’t read from. So in a CQRS event sourcing architecture, your event store is the only thing being written to. You never read from that event store. Instead you create read views by “fast forwarding” events to get the current state. If you need to separate ui service from reporting, you would create two read views that are each built and updated by pulling new events off the event store.
sou brasileiro e acompanho seu trabalho
Great Talk. Thanks
Event happened in the past and command will be fulfilled in future !!
I like simple, and this is simple and nice.
Awesome. Losing traceablity while making easy to add new functionality is true. Why not Use event ids to link events with the code (and the subsequent events) that execute due to those events?
This will help you during debugging, but it does not help you during design/coding. What he says is: You need runtime information to understand how the overall process/algorithm works. TracingID in Logfiles are exactly that: Runtime information. Without them its relatively hard to figure out what will happen.
Thank you
Martin Fowler sure has changed a lot since his Eastenders days.
I far prefer the new one.
Isn't a command something that is suppose to happen but hasn't yet, and an event something that has happened? Technically they're both messages.
He states that a event is something that simple have happened without the emitter knowing what will be done, a command on other hand is the emitter explicit telling what have to be done.
@@ooredroxoo implicit
(ref: procedural programming)
Thank you sir
30:28 legend
Event sourcing -> Redux
Very good talk. The only issue I have to understand, how to implement Event Sourcing in DB. As Martin said, I should be able to rebuild state from log at any time. This means for me, that I should create new record in log table everytime somethings have changed, and this record should contain all record data.
The other approach is to just store event messages like "address created", "address updated" and have one row in db that is being updated, but then there won't be any history available.
Any idea on this problem?
You don't necessary should be using your database for the log. And log is log, it is append only structure, so, new record on when something happens
It simply means that you create a new row for every incoming request in log table with all data and process that log in synchronous manner to update the actual entity.
If a system uses Event Carried State Transfer, then it makes sense to define a Command as a request to change state.
It really depends. I may not care about my listeners or I may care and expect a response asynchronously, thus you can still send the message as an event, not as a command: imagine you need to create a payment everytime you create an order, you can notify whenever an order is created and expect someone to listen to it and notify another event when the payment is made. That's up to you if you want to notify as "order created" or "create payment". I really prefer the first approach.
Gosh he is a very good writer and just an okay talker.
Most important moment: 30:28
IS CQRS same as Data normalization good for insert/update/delete whereas a materialized predefined refreshed view for queries. That is what Oracle has been doing long ago
I dont see how cqrs related to eda. Can anyone explain it to me?
I would strongly disagree that events and commands are the same and only differ in their names. Think of Laws, there you have incidents and consequences. Often it is the same in IT systems: the subsystem indicating a situation (event, change, whatever) can not know what the readers of the indication will do or have to do then. The cause can not know the effect, at least not in advance.
He means when you fire an event from your code, the name of the event will tell the difference of its intention.
#3 seems out of place in the list as it mainly deals with internals: how one particular component/service represents its own data. whereas all the others are strategies for how one component can talk to others.
#2 shows a diagram where multiple events are sourced from different systems to build a local snapshot. #3 is a superset of what kind of event messages those should be. #4 doesn't necessarily apply to Event-Driven as he says in his blog. So there aren't really 4, but they make for good headings to stimulate discussion.
I wish could attend this conference
Thanks for your comment. We'd love to see you at one of our 2019 conferences!
Chicago - April 28 - May 2; Berlin - Oct 21-25; Amsterdam - June 17-20; Copenhagen - Nov 18-22
See more details about each here: blog.gotocon.com
@@GOTO- $ lol. Fly us out, G.
Event sourcing is as old as the market place. It's how ledgers work.
I played back 6:16 more than once :P
And this ruclips.net/video/STKCRSUsyP0/видео.html :)
It makes me think about Datomic.... ;)
I would like to really see some real examples about how to monitor fact that the event-drvien platform is healthy. Imagine UI is event driven, on startup it just sends event to bus "Hi, I am new UI instannce in the wild", multiple services are waiting for such events and start pumping Process/Taks/JobViewPreparedEvents (even in form of full data tables to be displayed), I have no problem on event having the data inside. But how to monitor that it is all healthy??? I can imagine only something which knows the full event seqeucne and can monitor for example timeouts based on event database as timeseries data. But I wasn't able to find any kind of pattern or how to do health checks. Thanks for any comments to event driven monitoring
Dead letter queues
The pattern is to ditch events. As mentioned in the talk, you lose the ability to monitor what's going on when you use the first 2 event types.
after all, events are "I don't want to understand what's happening" and what you describe as 'health checks' sounds like "I want to understand what's happening".
Great, thank you :)
How does CQRS fit in here? I don't see a relation to events. There are commands and queries alright but neither of those have any of the properties attributed to events.
Events and Commands the same thing? Wouldn't an event flip the coupling (an event occurred) and a Command tighten the coupling (you do this)?
Didn't see below, but I think I agree with mettjus.
Actually it's very clear where the difference is between "event" and "command".
An "event" (in terms of event sourcing) is ALWAYS something in the past = it has happened already!
Quite contrary to a "command" which means something has to be done (with that command) = it is UPCOMING, and not in the past.
That claim struck me too.
@@heiko3169 In the home-grown SDK that I maintain; I decided that commands and events are internal object to the service, that are streamed to serializable "Hope" and "Fact" objects respectively. Using this nomenclature has the effect that people get a much more clearly defined concept.
CQRS - haven't we been doing this since the idea of database?
Is internal communication within a living organism ( lets say Human body in specific ) based on EDA. If yes how does the 4 patterns described in this video correlate to the communications within a human body.
remember brought some of his books on topics like design pattern and refactoring when i learnt how to program ... more than 10 years ago i guess
Crux database also solves many of the problems stated here
Talk derived from his talk here: martinfowler.com/articles/201701-event-driven.html What do you mean by “Event-Driven”?
Photo of Martin Fowler
Martin Fowler
07 February 2017
Go to 6:14 and play at 0.5 speed
Another good example of event sourcing is cryptocurrency
"Events, dear boy, events!"
How to use synchronous event sourcing?
28:45 So this is how braid was made?
isn't rdbms under the core an event source system.. transaction log and current state of db and snapshots? everyone has used them ..
Interesting
But there is no statement of behavior even when we don't use event notification and event driven.
He sounds amazingly like bill bailey
I missed the finance at 34:13
but now we have open telemery, so its not really opaque anymore