This is so relevant for me right now and exactly the scenario I'm working on; refactoring fundamental concepts in a system to slowly migrate from quite strict REST which is coupled to entities to a command/query pattern. Thanks!
Yes it is a touchy topic for many developer....but I totally agree with your thought. In real-world GET and POST are Not working as it was told that - GET is just to get/retrieve entity/resource and POST is to save an entity/resource. Simple and very common example is we have to implement search(with multiple filters/or advance search), here we are suppose to get the data but still we have to make a POST API as we cannot pass complex type(object) in GET API. So although we are suppose to get data but we are ending up with POST API :)
Query String is also possible place there for that data rather than the body of the request. Having said that, ultimately it's however you want to define your API.
I was thinking using nouns rather then verbs in the resource naming like: price instead of increase / decrease or availability instead of available/unavailable or quantity instead of inventoryAdjustment and do the decision of ordering supplies with a simple condition based on the new quantity value. The idea of a resource is that it supports standard CRUD operation as opposite to RPC ... Honestly I also like the idea presented here as .. the server dose not have to "guess" things .. based on what was changed on an entire object graph. I have nothing against dispatching specific commands in the system as business logic. Specific changes can be handled on patch verb I think.
I think Resources as entities mostly ends up that way bcos the way front end is designed. Usually for simplicity sake, backoffice applications would have a detail page for an entity with save/delete. So for that scenario, it makes sense to have resources are entities as opposed to capabilities
CRUD and Entity Services make sense often in less-value boundaries in a supporting role. However, in the heart of your system, if it's CRUD, then the workflow and business logic exists purely in your end-users heads.
This is absolutely true. In much of this industry, you just end up with "UX" people and self-styled "designers" (who don't code) meeting with customers/clients and doing nothing more than designing "mock ups" of pages. They end up throwing up everything all over each screen with a bit of fancy styling, but in the end just asking for everything to be editable with one big "save" button. Then the programmers are asked to implement that, so you get one giant anemic domain model with all the fields and a set of CRUD endpoints to manage the whole thing. It very much all starts at the beginning of the design process, none of this works if you're just being handed "stories" in DevOps Azure to implement something already designed by someone else. You need to start by using DDD with something like Event Storming or Event Modeling, working out actual user intent to determine tasks, figuring out your bounded contexts and your aggregates and entities and value objects etc. Many programmers, let alone juniors, never get the luxury to actually be allowed to design software well.
At 3:36 ,are all those API's still considered Restful such as POST /purchasing/products/{sku}/increaseCost. Or are these considered just "HTTP Api's" (if that's the canonical term.)
In the past, I have used JSON-RPC 2.0 for this purpose. However, it has not been widely accepted. Today I build API's exactly as you described in the video.
This was a video I missed before, great content as always. One perspective that I think a lot of people struggle with (sometimes me) is for example the customer side of the sample application. What you have here is perfectly fine when thinking from the administrative perspective, but when a customer will go a site and search for a product I think the data needs to be consolidated before hand. Composing it together in the client or on the server might not be enough. Take search of products for example, when showing the search result you do want to show both the basic information like price and description, but you might also want to show some stock information. In my opinion that needs to be projected into some kind of (search) database and from a customer point of view it should only be one request to get the relevant information. Not doing so and for example solving it in the client could lead to N+1 requests, first a search against the catalogue for example and then one request per sku to get stock or price.
@@CodeOpinion I knew I've seen it before :). I actually think this is one of the topic that people doesn't think too much about when splitting a larger application into "microservices", that they at some point need to consolidate the data to not flood the system with requests.
You're reacting to events from different contexts. For that you need to reference the project containing the event (Inventory Adjusted from Sales is referenced in PurchaseOnLowInventory from Purchasing). What happens if both contexts have to react to each other's events? You can't have cyclic dependencies. So what should we do in that case? Define a separate project for events? How would that fit in?
You spoke a bit about composing queries to "batch" those client requests into one request. What about at the opposite end with a command? What if your save button instead saved everything on the form - rather than having modals pop up that conveniently HTTP call out to that resource? I'm guessing you'd batch again, which is where something like a durable message queue would come into play to make those multiple backend calls transactional. Do you have a preference?
It's interesting if you start thing about the "resources" as the actual commands or queries, as opposed to "things" stored in your database. My leaning then would be, using this apporach, that I'd want to be consistent with the naming and use verbs everywhere in the route, including for retrieving data. But... GET /api/products/{sku}/getProduct is obviously redundant. Perhaps this is why I've actually seen some APIs from some large companies with some sophisticated software basically use POSTs for everything. If you think of a POST as "creating" a "command" to do something. "POST .../getProduct" -- but then does the concept of CQRS break down? Or can my POST to "getProduct" be interpreted as a query on the backend, and the frontend not necessarily concern itself with being tied to CQRS concepts?
You would need to derive events from CRUD. Or create "catch-all" events such as "CustomerUpdated" without really knowing "what" actually updated. Commands lead to explicit actions that can then derive to specific events.
What if you have a query that returns tabular records with data joined from several services? Should that materialized view be in the service responsible for the aggregations (Backend For Frontend)?
It all depends where you want to do that composition. It can be a service specific to do that composition in a type of BFF. Or it can be materialized in a service that is constantly keeping it up to date via events.
@@CodeOpinion This is the last roadblock to my portfolio project I've been contemplating! I kinda wish to use the latter approach whereby there is a materialized view, because I think this would lead to faster results in most cases, provided the materialized views aren't overencumbered with so many DB requests from frontend restful calls. My question here is, aside from the additional data duplication (we have write, read, and now materialized databases/schemas) does having this "materialized" view for the frontend not detract from the core concept of the read API to begin with? In other words, you have this read API, which is set for queries. But now when frontend needs data from several services, it goes to the materialized view, ignoring the existence of all of the individual microservice's read API's. Is this inefficient, and also would it be an okay idea to omit the "read" API and have the materialized view ONLY? Possibly not, because I imagine her that you'd either omit certain "columns in a table" which may be needed for analytics, etc. OR if you include all of the columns, then the materialized views are basically just caches of your event store, containing a ton of columns, with different materialized views duplicating the columns amongst each other :/
Why should commands be resources? In my opinion a REST interface only really work in the REST way if you use a hypermedia format in your request. The most ubiquitous hypermedia format would be HTML (if you like json look at hal / siren / collection+json, hyper-item for analogies). A default pattern would be query a resource: GET /products/{sku} In the response you should just include an HTML-Form for each of your commands that are possible in the given state (this is dynamic in task based UIs). Don't forget that a form can have a hidden parameter that explains which of the commands should be executed (e.g. inventory-adjustment) Executing a command would be: POST /products/{sku} with a payload of cmd=adjust-inventory params= .... This would redirect to Location /products/{sku} where the client will get the updated resource. This would also clear all intermediate caches which would not happen if you'd use the POST /products/{sku}/adjust-inventory URL.
Why should the be resources? Why shouldn't they be resources? All tradeoffs. Put the CMD in the body of the request, put in in the URI, put it in a header... you're putting it somewhere. A resource is whatever you want it to be. Same goes for idempotency. Some ID used for an operation can be a header, query string, body, or apart of the URI. If it's a part of the URI and you're using hypermedia, the client doesn't even need to think about providing it and you're idempotent without them knowing.
I might be wrong, but /sales/products/{sku}/increasePrice doesn't seem to be very much REST-y. Wouldn't the correct endpoint be a PATCH /sales/product/{sku}/price with replace operation? Making API work with CQRS is something different than making REST API work with it as well
To publish the right event with this "PATCH /sales/product/{sku}/price" you'll have to infer the right action from the price value input (price increase, decrease, reset, whatever.) And if that is not obvious, you'll have to add another info to the request, something like and enum {increase = 1, decrease = 2, reset = 3, etc.}.
I don't have any issue with what you're suggesting. However, I would prefer to be explicit about the intent of the request, which it doesn't provide. Are you increasing or decreasing the price? If the route was as you suggested, that logic must live in the code handling the route to make that determination. Also, let's say you also needed to provide a reason code/type for the price increase? Meaning it's not about specific properties but rather it's about commands/tasks/actions.
@@CodeOpinion I don't have issue with your statement, I'm only asking whether or not we are bending "restiness" of our API to fit CQRS. Isn't that more of an RPC thing there, especially having additional context of code/type of price change? On the other hand, if we'd have additional context, we could POST it to a collection /sales/product/{sku}/price with enum values mentioned by @zaki it above. But thats a different topic. :) I understand why you'd need two separate endpoints for increase & decrease, but is it still REST endpoint? Anyway - don't get me wrong ;) I appreciate the knowledge you are sharing, I just wanted to know your opinion on that.
Using "patch" to update the price is still treating the "product" resource as an Entity, to be manipulated with CRUD semantics. But CRUD semantics can only go so far. Consider this: What if we added a requirement that, when changing the price, we must also specify a "reason" for the price change (to be recorded in some audit log)? The "reason" is not a property of the product, so it would not make sense to include it in a PATCH request. But if you have a dedicated HTTP resource for a *command* (setPrice), then you can add whatever parameters to that command you need to. e.g. POST /sales/product/{sku}/setPrice { price: 100, reason: "Holiday Sale" } There is nothing particularly "REST-y" about CRUD over HTTP.
@@krystian5858 You've made very good points. I think that it all boils down to naming and what the industry in general considers good and bad practices. For some weird reason RESTful APIs have became synonym for "good", and any other type of API is considered "not "RESTful" (i.e. not good). In reality however task-based APIs don't go well with REST (in purist terms), and that's absolutely fine. As the result, a well implemented task-based API is not RESTful, and that's deliberate. I think the only mistake that Derek made in this video is continuing referring to REST, hence why it might confuse some people. The bottom line is- with task based APIs we don't do REST.
I am wondering if you have a code sample I could look at that could help illustrate these principles? Or an example of that I can look at of our to structure a CQRS project. Love your videos btw 🙂
I post the source I use in my videos to the Developer members of my channel. You can get more info here: ruclips.net/channel/UC3RKA4vunFAfrfxiJhPEplwjoin
You should probably have named it 'Is HTTP API with cqrs possible?' :) It looks like RPC over HTTP to me. Creating a resource represented by a verb via POST is not something I got used to. However, I could get REST wrong myself:)
I haven't touched the topic about REST too much on RUclips because most folks are so confused about what REST is. The current definition most people have, that is repeated so often is misguided. I recommend checking out this 20 min video which I think is a good primer about misconceptions. ruclips.net/video/QIv9YR1bMwY/видео.html
@@CodeOpinion as I understand *common* conventions, a resource is a noun. A POST creates a new subordinate represented in a plural form. However, it doesn't say that one has to follow this until the whole team agrees on its own convention:)
Hey Derek, How do you handle returning of ID's that are generated inside domain from a create command? The Id is not Guid, it has complex rules for creation inside domain but CQRS people suggest you shouldn't return anything from commands. Thanks
It's not a hard and fast rule. As I hope I illustrate in my videos, be pragmatic. Returning an ID that's generated via a command isn't much different than returning a Status or an Acknowledgement.
Good question, I'll be doing a video soon about this. You send Commands, Publish Events. Commands are sent from other services and are consumed by the owner of the command. Events are published by the owner of the event and consumed by other services. Commands have one consumer. Events have zero or many consumers.
In a scenario when we will have a mysql database (example: client_service) to store events on command side and a similar data model is used for query side database and events are managed asynchronously using a message broker. When the rest api inserts data in database on the command side and posts the event on message broker thereafter, I felt, to use an internal microservice to subscribe to that channel on the message broker, get the event data and populate the read model (another mysql database, example: client_service_q). For all queries, we will have a microservice that does select queries on the read model (i.e.query side mysql database, on the client_service_q) as per the example. is this correct way? or do you think, I did not follow database per service design pattern correctly w.r.t. query side microservice as I tried to populate data into client_service_q mysql database, using internal microservice, please share your inputs Derek, thank you Note: I am intentionally trying mysql database for read model for this experiment usecase as I wish to try elasticsearch etc as per need in a later time, trying to see if the thought process is right!!!
Why not just use your client_service database for queries? If your model is drastically different, then yes, you could update your read model via an event which will be asynchronous. That has some implications obviously around the users expectations about data being consistent.
@@CodeOpinion if i use that, the options will be materialized views for query know, any other option without violating database per service design pattern? Forgot to mention, I wish to use Event Sourcing design pattern in sync with CQRS & Database per Service design patterns
Because I didn't think the example needed another layer of a domain model. I often times just use a transaction script and then move to a domain model when logic becomes more complex. I don't start there.
is crud mandatory for REST API? if one is about to use CQRS REST and event sourcing together, can the one do it without saving changes to database? and for the query side it just load events of that entity from event store and apply events to rebuild the entity and finally return it the client.
well, your videos are treasures.for anyone who has the same question. i just found out how event sourcing and cqrs can be applied to api. but i am still a bit not sure if such api would be RESTFUL in the video CQRS & Event Sourcing Code Walk-Through "ruclips.net/video/5aznkIEvkKc/видео.html&ab_channel=CodeOpinion" by the same author
CRUD has nothing to do with REST. Yes, you don't have to have a projection/readmodel in Event Sourcing. You can use the event store and event streams for the UI. Creating projections just makes querying easier for various use cases.
CQRS works awesome when you're dealing with a RESTful API. However what most people call REST isn't REST. It's CRUD around Entities and JSON over HTTP.
Good explanation of how broadly applying the concept (REST: 'resource' "any information that can be named can be a resource") makes it useful as a style for event-driven working (while it is much less so with the rigid 'resource= entity' interpretation). But in addition there is the different interpretation of the REST constraint 'uniform interface'. If you look at en.wikipedia.org/wiki/Richardson_Maturity_Model your way of describing resources would be seen as belonging to 'The Swamp of POX'. Not a problem for me, because the forced use of entity names as a resource leads to strange and undesirable effects in event-driven work. It remains a challenge to properly interpret and apply REST as a style because the thought that REST says 'resource must be entity' is very widespread.
While I try and keep videos relatively short, a lot of them relate to each other. In this video, you can see I take it a bit farther and star leveraging Hypermedia. ruclips.net/video/OcWa0WJBF2U/видео.html
Hi, I am following all your videos. In task based ui , how to create new resources i.e product.. how sku will be generated.. and who owns the product? is it catalogue,sales or other bounded context??
@@CodeOpinion If it's in different databases, should the product get created in every other database? Or the catalog service message a ProductCreated which will be consumed by services that need this information and other services will wait until they need to store data about this new product?
Depends on what you want to do. Task-based UI simply means try to capture user intent at the high-level and present it in the UI. It doesn't tell you what the tasks are, that's for you to figure out based on your use needs and problem domain. Maybe you have a "create product" task that creates a product, clicking that button issues the SKU and takes you to the page with the SKU populated and presents tasks to set the description, adjust the price, adjust the quantity on hand etc. Maybe to create the product you need to provide a description and a price etc, in which case perhaps there could be a command to create the product with all those details and then the catalogue service could create the product and issue the command event with the details on price and quantity on hand etc.
I am wondering why we are sticking to REST at all. To me, gRPC with CQRS makes much more sense. From what I understand, gRPC and javascript in the browser are not best friends, but isn't that a matter of time? I look forward to your perspective on this.
If you use Hypermedia in your HTTP APIs along with CQRS, then it's a wonderful match. I'll cover it more in an upcoming video. Hypermedia never really caught on in the HTTP API space, which blows my mind.
Glad you enjoy them. If they're about a specific video, just leave a comment at the video. I try to reply to all of them. If it's more in general, you can post over on a subreddit www.reddit.com/r/CodeOpinion/
You might as well call this by what it really is: RPC. For a task based UI RPC fits better and there's nothing wrong with that. No need to pretend it is REST
What I'm describing is an HTTP API in this video. I might not of made that clear in the beginning. Most people equate REST with Resources as Entities and and HTTP methods as CRUD. I used REST in the title because that's what the term has turned into. I do plan on creating a video about how hypermedia and CQRS fit extremely well together.
@@CodeOpinion Once you put the method/function name in the URL and provide the arguments using the body or the query string it does look suspiciously like an RPC implementation :) Anyway, there's no real disagreement from my part and my comment was mostly about the title. By the way this is one of my favorite channels and I like to think I am learning a lot from your videos. I appreciate all the work you put into it.
As long as the description of how to perform a request is given in the media type (either as OOB description or in the response of a previous request), you’ve got a hypermedia driven API and whether the requests look like RPC doesn’t matter anymore. It’s the hypermedia part that most profoundly separates RPC from REST.
Good explanations. Keep going :) P.S. Seeing DB context operations inside the controller runs shivers down my spine. Please don't show such bad software design as an example. I assume you do that to keep your explanations simple ... but ..don't.
I used to think of it as a bad practice as well but nowadays I'm not that sure anymore. What is the point of moving that code to an abstraction if there is no value in this abstraction because it's just doing a Crud operation? Keeping things simple at the beginning and just encapsulating/abstracting them when there is a need for it is something that I think is a reasonable approach after seeing a lot of overengineered code in the past years.
@@markuss.6981 This is in no way anything near over-engineering. Where do you see any over-engineering by just move the code from the controller to a repository class?? It's just about "separation of concerns" and DRY ("don't repeat yourself"). And of course, it helps with the unit testing of your components. Do it in the controller might be okay, for some hobby projects - but only there. ...and even there you should actually do it properly. But that's up to you. P.S. If someone applies for a job, and does this in his demo/portfolio projects, I wouldn't hire him. Because he obviously misses the basics of software development.
@@heiko3169 Well, introducing abstractions that provide no value could be seen as over-engineering. I see your point regarding SoC. Also one could say, for consistencys sake we only want data access through repositories in the whole project. Fair enough. In this example i see that there are mostly CRUD operations in the Controller. So i see no value in moving the code to a Repository. How would this help unit testing? What would you test in the Controller when there is only one line in there delegating to the Repository class? Completely abstracting persistence is beneficial when the core domain / logic in this area is complex enough to test it in isolation. If the logic only consists of CRUD, there is no value in carrying this extra weight. When using the Repository Pattern in a dogmatic way (and without accompanying patterns as the Specification Pattern) i often see performance problems and dependency hell (as people inject a repository for every table and this can explode when the use case has multiple tables involved). So i think starting simpler and abstracting when there is a need is a better strategy. I hardly believe in absolute solutions when it comes to Software Architecture. For almost every decision there are pros and cons.
@@markuss.6981 Regarding your question about "how this helps unit testing": Well, how would you unit test your controller functions with Unit Tests without the ability to mock the datalayer?!
Any new architectural approach can be considered controversial for a person seeing if for the first time. The most important thing is to avoid being stuck in a comfort zone and appreciate that there are different ways of doing things. Not good, not bad, but different. And which one we use should depend on the context and specific system requirements.
@@CodeOpinion yeah I meant controversial in a good way. The only part I struggle with is where you said commands and queries can be resources and have identifiers. Posting a command does not represent a state transfer for me, I am also not so sure they have identifiers. I am not sure it is a resource. I like the general approach (and use CQRS over HTTP apis myself on a daily basis). Just difficult for me to still call it REST. It is just RPC over HTTP or poor man's gRPC or whatever you want to call it:) What's in a name. I highly appreciate the task based approach and sentiment around UX. Thanks for another great video!
@@Mvision123 I used "REST" in the title because I think that's what most people equate to an HTTP API and not really what REST was defined as (architectural constraints). I do plan on creating another video that continues this one but uses hypermedia. I might in that video address some misconceptions about REST. Appreciate the comment.
You have no idea how much this example helped to clarify a few points I had about CQRS. I never really understood its benefits until now. Great Job!
Glad it helped!
This is so relevant for me right now and exactly the scenario I'm working on; refactoring fundamental concepts in a system to slowly migrate from quite strict REST which is coupled to entities to a command/query pattern. Thanks!
Hopefully it helped!
Task-based UI and REST are perfectly matching as you just demonstrated. Great video! Waiting for the next episodes :)
More to come!
Yes it is a touchy topic for many developer....but I totally agree with your thought.
In real-world GET and POST are Not working as it was told that - GET is just to get/retrieve entity/resource and POST is to save an entity/resource.
Simple and very common example is we have to implement search(with multiple filters/or advance search), here we are suppose to get the data but still we have to make a POST API as we cannot pass complex type(object) in GET API.
So although we are suppose to get data but we are ending up with POST API :)
Query String is also possible place there for that data rather than the body of the request. Having said that, ultimately it's however you want to define your API.
I was thinking using nouns rather then verbs in the resource naming like: price instead of increase / decrease or availability instead of available/unavailable or quantity instead of inventoryAdjustment and do the decision of ordering supplies with a simple condition based on the new quantity value. The idea of a resource is that it supports standard CRUD operation as opposite to RPC ... Honestly I also like the idea presented here as .. the server dose not have to "guess" things .. based on what was changed on an entire object graph. I have nothing against dispatching specific commands in the system as business logic. Specific changes can be handled on patch verb I think.
I think Resources as entities mostly ends up that way bcos the way front end is designed.
Usually for simplicity sake, backoffice applications would have a detail page for an entity with save/delete. So for that scenario, it makes sense to have resources are entities as opposed to capabilities
CRUD and Entity Services make sense often in less-value boundaries in a supporting role. However, in the heart of your system, if it's CRUD, then the workflow and business logic exists purely in your end-users heads.
This is absolutely true. In much of this industry, you just end up with "UX" people and self-styled "designers" (who don't code) meeting with customers/clients and doing nothing more than designing "mock ups" of pages. They end up throwing up everything all over each screen with a bit of fancy styling, but in the end just asking for everything to be editable with one big "save" button. Then the programmers are asked to implement that, so you get one giant anemic domain model with all the fields and a set of CRUD endpoints to manage the whole thing.
It very much all starts at the beginning of the design process, none of this works if you're just being handed "stories" in DevOps Azure to implement something already designed by someone else. You need to start by using DDD with something like Event Storming or Event Modeling, working out actual user intent to determine tasks, figuring out your bounded contexts and your aggregates and entities and value objects etc. Many programmers, let alone juniors, never get the luxury to actually be allowed to design software well.
"A resource is not necessarily an entity" +100!
It's still confusing to me how we got into the path of CRUD over HTTP against "entities" that are really just data models.
At 3:36 ,are all those API's still considered Restful such as POST /purchasing/products/{sku}/increaseCost. Or are these considered just "HTTP Api's" (if that's the canonical term.)
In the past, I have used JSON-RPC 2.0 for this purpose. However, it has not been widely accepted. Today I build API's exactly as you described in the video.
This was a video I missed before, great content as always. One perspective that I think a lot of people struggle with (sometimes me) is for example the customer side of the sample application. What you have here is perfectly fine when thinking from the administrative perspective, but when a customer will go a site and search for a product I think the data needs to be consolidated before hand. Composing it together in the client or on the server might not be enough. Take search of products for example, when showing the search result you do want to show both the basic information like price and description, but you might also want to show some stock information. In my opinion that needs to be projected into some kind of (search) database and from a customer point of view it should only be one request to get the relevant information. Not doing so and for example solving it in the client could lead to N+1 requests, first a search against the catalogue for example and then one request per sku to get stock or price.
I talk about composition in this video which is a bit newer: ruclips.net/video/ILbjKR1FXoc/видео.html
@@CodeOpinion I knew I've seen it before :). I actually think this is one of the topic that people doesn't think too much about when splitting a larger application into "microservices", that they at some point need to consolidate the data to not flood the system with requests.
You're reacting to events from different contexts. For that you need to reference the project containing the event (Inventory Adjusted from Sales is referenced in PurchaseOnLowInventory from Purchasing). What happens if both contexts have to react to each other's events? You can't have cyclic dependencies. So what should we do in that case? Define a separate project for events? How would that fit in?
Take a look at one of my most recent videos where I show the structure of this even within a monolith: ruclips.net/video/VGShtGU3hOc/видео.html
Man you deserve a medal for this video. Thank you.
Thanks!
You spoke a bit about composing queries to "batch" those client requests into one request. What about at the opposite end with a command? What if your save button instead saved everything on the form - rather than having modals pop up that conveniently HTTP call out to that resource? I'm guessing you'd batch again, which is where something like a durable message queue would come into play to make those multiple backend calls transactional. Do you have a preference?
It's interesting if you start thing about the "resources" as the actual commands or queries, as opposed to "things" stored in your database.
My leaning then would be, using this apporach, that I'd want to be consistent with the naming and use verbs everywhere in the route, including for retrieving data.
But... GET /api/products/{sku}/getProduct is obviously redundant.
Perhaps this is why I've actually seen some APIs from some large companies with some sophisticated software basically use POSTs for everything. If you think of a POST as "creating" a "command" to do something. "POST .../getProduct" -- but then does the concept of CQRS break down? Or can my POST to "getProduct" be interpreted as a query on the backend, and the frontend not necessarily concern itself with being tied to CQRS concepts?
I'm not seeing what CQRS adds at the end, how would triggering additional actions would have been more problematic without?
You would need to derive events from CRUD. Or create "catch-all" events such as "CustomerUpdated" without really knowing "what" actually updated. Commands lead to explicit actions that can then derive to specific events.
@@CodeOpinion this actually became apparent in your video about event centric UI.
What if you have a query that returns tabular records with data joined from several services? Should that materialized view be in the service responsible for the aggregations (Backend For Frontend)?
It all depends where you want to do that composition. It can be a service specific to do that composition in a type of BFF. Or it can be materialized in a service that is constantly keeping it up to date via events.
@@CodeOpinion This is the last roadblock to my portfolio project I've been contemplating! I kinda wish to use the latter approach whereby there is a materialized view, because I think this would lead to faster results in most cases, provided the materialized views aren't overencumbered with so many DB requests from frontend restful calls. My question here is, aside from the additional data duplication (we have write, read, and now materialized databases/schemas) does having this "materialized" view for the frontend not detract from the core concept of the read API to begin with? In other words, you have this read API, which is set for queries. But now when frontend needs data from several services, it goes to the materialized view, ignoring the existence of all of the individual microservice's read API's.
Is this inefficient, and also would it be an okay idea to omit the "read" API and have the materialized view ONLY? Possibly not, because I imagine her that you'd either omit certain "columns in a table" which may be needed for analytics, etc. OR if you include all of the columns, then the materialized views are basically just caches of your event store, containing a ton of columns, with different materialized views duplicating the columns amongst each other :/
Why should commands be resources? In my opinion a REST interface only really work in the REST way if you use a hypermedia format in your request.
The most ubiquitous hypermedia format would be HTML (if you like json look at hal / siren / collection+json, hyper-item for analogies).
A default pattern would be query a resource: GET /products/{sku}
In the response you should just include an HTML-Form for each of your commands that are possible in the given state (this is dynamic in task based UIs).
Don't forget that a form can have a hidden parameter that explains which of the commands should be executed (e.g. inventory-adjustment)
Executing a command would be: POST /products/{sku} with a payload of cmd=adjust-inventory params= ....
This would redirect to Location /products/{sku} where the client will get the updated resource.
This would also clear all intermediate caches which would not happen if you'd use the POST /products/{sku}/adjust-inventory URL.
Why should the be resources? Why shouldn't they be resources? All tradeoffs. Put the CMD in the body of the request, put in in the URI, put it in a header... you're putting it somewhere. A resource is whatever you want it to be. Same goes for idempotency. Some ID used for an operation can be a header, query string, body, or apart of the URI. If it's a part of the URI and you're using hypermedia, the client doesn't even need to think about providing it and you're idempotent without them knowing.
When you update the price why send back NoContent and the fetch for the updated content instead of just sending back the updated info in one request?
I might be wrong, but /sales/products/{sku}/increasePrice doesn't seem to be very much REST-y. Wouldn't the correct endpoint be a PATCH /sales/product/{sku}/price with replace operation? Making API work with CQRS is something different than making REST API work with it as well
To publish the right event with this "PATCH /sales/product/{sku}/price" you'll have to infer the right action from the price value input (price increase, decrease, reset, whatever.) And if that is not obvious, you'll have to add another info to the request, something like and enum {increase = 1, decrease = 2, reset = 3, etc.}.
I don't have any issue with what you're suggesting. However, I would prefer to be explicit about the intent of the request, which it doesn't provide. Are you increasing or decreasing the price? If the route was as you suggested, that logic must live in the code handling the route to make that determination. Also, let's say you also needed to provide a reason code/type for the price increase? Meaning it's not about specific properties but rather it's about commands/tasks/actions.
@@CodeOpinion I don't have issue with your statement, I'm only asking whether or not we are bending "restiness" of our API to fit CQRS. Isn't that more of an RPC thing there, especially having additional context of code/type of price change?
On the other hand, if we'd have additional context, we could POST it to a collection /sales/product/{sku}/price with enum values mentioned by @zaki it above. But thats a different topic. :)
I understand why you'd need two separate endpoints for increase & decrease, but is it still REST endpoint?
Anyway - don't get me wrong ;) I appreciate the knowledge you are sharing, I just wanted to know your opinion on that.
Using "patch" to update the price is still treating the "product" resource as an Entity, to be manipulated with CRUD semantics. But CRUD semantics can only go so far.
Consider this: What if we added a requirement that, when changing the price, we must also specify a "reason" for the price change (to be recorded in some audit log)? The "reason" is not a property of the product, so it would not make sense to include it in a PATCH request. But if you have a dedicated HTTP resource for a *command* (setPrice), then you can add whatever parameters to that command you need to. e.g. POST /sales/product/{sku}/setPrice { price: 100, reason: "Holiday Sale" }
There is nothing particularly "REST-y" about CRUD over HTTP.
@@krystian5858 You've made very good points. I think that it all boils down to naming and what the industry in general considers good and bad practices. For some weird reason RESTful APIs have became synonym for "good", and any other type of API is considered "not "RESTful" (i.e. not good). In reality however task-based APIs don't go well with REST (in purist terms), and that's absolutely fine. As the result, a well implemented task-based API is not RESTful, and that's deliberate. I think the only mistake that Derek made in this video is continuing referring to REST, hence why it might confuse some people. The bottom line is- with task based APIs we don't do REST.
I am wondering if you have a code sample I could look at that could help illustrate these principles? Or an example of that I can look at of our to structure a CQRS project. Love your videos btw 🙂
I post the source I use in my videos to the Developer members of my channel. You can get more info here: ruclips.net/channel/UC3RKA4vunFAfrfxiJhPEplwjoin
How to write test for this endpoint?
great video again, maybe a slightly random question, but with boundaries, would you have a separate DB per boundary or are data boundaries different?
DB per boundary/service. Check out my Loosely Coupled Monolith where I cover this a bit: ruclips.net/video/48C-RsEu0BQ/видео.html
You should probably have named it 'Is HTTP API with cqrs possible?' :) It looks like RPC over HTTP to me.
Creating a resource represented by a verb via POST is not something I got used to. However, I could get REST wrong myself:)
REST isn't CRUD over HTTP. Resources are not Entities. They can be, but a resource is anything with an identifier.
I haven't touched the topic about REST too much on RUclips because most folks are so confused about what REST is. The current definition most people have, that is repeated so often is misguided. I recommend checking out this 20 min video which I think is a good primer about misconceptions. ruclips.net/video/QIv9YR1bMwY/видео.html
@@CodeOpinion as I understand *common* conventions, a resource is a noun. A POST creates a new subordinate represented in a plural form. However, it doesn't say that one has to follow this until the whole team agrees on its own convention:)
Just wanted to say thanks for the videos. Very impressive!
Thanks! Hope they are helpful
Excellent illustration, thank you very much
Many thanks
Hey Derek, How do you handle returning of ID's that are generated inside domain from a create command? The Id is not Guid, it has complex rules for creation inside domain but CQRS people suggest you shouldn't return anything from commands. Thanks
It's not a hard and fast rule. As I hope I illustrate in my videos, be pragmatic. Returning an ID that's generated via a command isn't much different than returning a Status or an Acknowledgement.
Thanks for the video.
I got a bit confused regarding usage of Publish and Send. When do we use publish and when we use send?
Good question, I'll be doing a video soon about this. You send Commands, Publish Events. Commands are sent from other services and are consumed by the owner of the command. Events are published by the owner of the event and consumed by other services. Commands have one consumer. Events have zero or many consumers.
@@CodeOpinion Thanks a lot for the explanation, looking forward to see the video!.
Aggretating multiple requests on server side is API Gateway pattern
I've covered this to some degree in other videos related to mentioning View/UI composition.
can be POST /api/v1/products/{sku}/price-increase (note the nouns) but not necessary
In a scenario when we will have a mysql database (example: client_service) to store events on command side and a similar data model is used for query side database and events are managed asynchronously using a message broker.
When the rest api inserts data in database on the command side and posts the event on message broker thereafter, I felt, to use an internal microservice to subscribe to that channel on the message broker, get the event data and populate the read model (another mysql database, example: client_service_q). For all queries, we will have a microservice that does select queries on the read model (i.e.query side mysql database, on the client_service_q) as per the example.
is this correct way? or do you think, I did not follow database per service design pattern correctly w.r.t. query side microservice as I tried to populate data into client_service_q mysql database, using internal microservice, please share your inputs Derek, thank you
Note: I am intentionally trying mysql database for read model for this experiment usecase as I wish to try elasticsearch etc as per need in a later time, trying to see if the thought process is right!!!
Why not just use your client_service database for queries? If your model is drastically different, then yes, you could update your read model via an event which will be asynchronous. That has some implications obviously around the users expectations about data being consistent.
@@CodeOpinion if i use that, the options will be materialized views for query know, any other option without violating database per service design pattern? Forgot to mention, I wish to use Event Sourcing design pattern in sync with CQRS & Database per Service design patterns
Wow, eye-opener
Graphql would be better??
So why is your business logic inside your Mediatr handler? How can it be enforced if it's here?
Because I didn't think the example needed another layer of a domain model. I often times just use a transaction script and then move to a domain model when logic becomes more complex. I don't start there.
is crud mandatory for REST API? if one is about to use CQRS REST and event sourcing together, can the one do it without saving changes to database? and for the query side it just load events of that entity from event store and apply events to rebuild the entity and finally return it the client.
well, your videos are treasures.for anyone who has the same question. i just found out how event sourcing and cqrs can be applied to api. but i am still a bit not sure if such api would be RESTFUL
in the video CQRS & Event Sourcing Code Walk-Through "ruclips.net/video/5aznkIEvkKc/видео.html&ab_channel=CodeOpinion" by the same author
CRUD has nothing to do with REST. Yes, you don't have to have a projection/readmodel in Event Sourcing. You can use the event store and event streams for the UI. Creating projections just makes querying easier for various use cases.
can i get where to clone this sample project?
Check the community tab. I should have it listed there.
CQRS Maps better when the api that's beign mapped is follows RESTful principles.
CQRS works awesome when you're dealing with a RESTful API. However what most people call REST isn't REST. It's CRUD around Entities and JSON over HTTP.
@@CodeOpinion So true it hurts
@@CodeOpinion I believe they refer to those types of REST APIs as Pure or Fully REST APIs.
Good explanation of how broadly applying the concept (REST: 'resource' "any information that can be named can be a resource") makes it useful as a style for event-driven working (while it is much less so with the rigid 'resource= entity' interpretation). But in addition there is the different interpretation of the REST constraint 'uniform interface'. If you look at en.wikipedia.org/wiki/Richardson_Maturity_Model your way of describing resources would be seen as belonging to 'The Swamp of POX'. Not a problem for me, because the forced use of entity names as a resource leads to strange and undesirable effects in event-driven work. It remains a challenge to properly interpret and apply REST as a style because the thought that REST says 'resource must be entity' is very widespread.
While I try and keep videos relatively short, a lot of them relate to each other. In this video, you can see I take it a bit farther and star leveraging Hypermedia. ruclips.net/video/OcWa0WJBF2U/видео.html
Amazing content
Hi, I am following all your videos. In task based ui , how to create new resources i.e product.. how sku will be generated.. and who owns the product? is it catalogue,sales or other bounded context??
The SKU would be generated and owned by one service. It could be the catalog.
@@CodeOpinion please cover a video on task based UI/UX to create a new product and how to generate a new sku ,
@@CodeOpinion If it's in different databases, should the product get created in every other database? Or the catalog service message a ProductCreated which will be consumed by services that need this information and other services will wait until they need to store data about this new product?
@@Draziak You could publish an event. Another option is to simply create the product in other boundaries when it's first command comes in.
Depends on what you want to do. Task-based UI simply means try to capture user intent at the high-level and present it in the UI. It doesn't tell you what the tasks are, that's for you to figure out based on your use needs and problem domain.
Maybe you have a "create product" task that creates a product, clicking that button issues the SKU and takes you to the page with the SKU populated and presents tasks to set the description, adjust the price, adjust the quantity on hand etc.
Maybe to create the product you need to provide a description and a price etc, in which case perhaps there could be a command to create the product with all those details and then the catalogue service could create the product and issue the command event with the details on price and quantity on hand etc.
I am wondering why we are sticking to REST at all. To me, gRPC with CQRS makes much more sense. From what I understand, gRPC and javascript in the browser are not best friends, but isn't that a matter of time? I look forward to your perspective on this.
If you use Hypermedia in your HTTP APIs along with CQRS, then it's a wonderful match. I'll cover it more in an upcoming video. Hypermedia never really caught on in the HTTP API space, which blows my mind.
@@CodeOpinion Awesome. I’m looking forward to your videos. I love how well prepared they are.
hi guy, can we get source code ?
All the source is available to Developer members of my channel. For more info: ruclips.net/channel/UC3RKA4vunFAfrfxiJhPEplwjoin
@@CodeOpinion how can be free ???
Hi, I'm a big fan of your videos. Hope i can contact you i have some questions 😁
Glad you enjoy them. If they're about a specific video, just leave a comment at the video. I try to reply to all of them. If it's more in general, you can post over on a subreddit www.reddit.com/r/CodeOpinion/
You might as well call this by what it really is: RPC.
For a task based UI RPC fits better and there's nothing wrong with that.
No need to pretend it is REST
What I'm describing is an HTTP API in this video. I might not of made that clear in the beginning. Most people equate REST with Resources as Entities and and HTTP methods as CRUD. I used REST in the title because that's what the term has turned into. I do plan on creating a video about how hypermedia and CQRS fit extremely well together.
@@CodeOpinion Once you put the method/function name in the URL and provide the arguments using the body or the query string it does look suspiciously like an RPC implementation :)
Anyway, there's no real disagreement from my part and my comment was mostly about the title.
By the way this is one of my favorite channels and I like to think I am learning a lot from your videos. I appreciate all the work you put into it.
@@GlediCaushaj Appreciate the comment. Don't be hesitant to comment if you disagree!
As long as the description of how to perform a request is given in the media type (either as OOB description or in the response of a previous request), you’ve got a hypermedia driven API and whether the requests look like RPC doesn’t matter anymore. It’s the hypermedia part that most profoundly separates RPC from REST.
Good explanations. Keep going :)
P.S. Seeing DB context operations inside the controller runs shivers down my spine. Please don't show such bad software design as an example. I assume you do that to keep your explanations simple ... but ..don't.
It is due example. But ya, agreed within a certain context.
I used to think of it as a bad practice as well but nowadays I'm not that sure anymore. What is the point of moving that code to an abstraction if there is no value in this abstraction because it's just doing a Crud operation? Keeping things simple at the beginning and just encapsulating/abstracting them when there is a need for it is something that I think is a reasonable approach after seeing a lot of overengineered code in the past years.
@@markuss.6981 This is in no way anything near over-engineering. Where do you see any over-engineering by just move the code from the controller to a repository class??
It's just about "separation of concerns" and DRY ("don't repeat yourself"). And of course, it helps with the unit testing of your components.
Do it in the controller might be okay, for some hobby projects - but only there.
...and even there you should actually do it properly. But that's up to you.
P.S. If someone applies for a job, and does this in his demo/portfolio projects, I wouldn't hire him. Because he obviously misses the basics of software development.
@@heiko3169 Well, introducing abstractions that provide no value could be seen as over-engineering.
I see your point regarding SoC. Also one could say, for consistencys sake we only want data access through repositories in the whole project. Fair enough.
In this example i see that there are mostly CRUD operations in the Controller. So i see no value in moving the code to a Repository. How would this help unit testing? What would you test in the Controller when there is only one line in there delegating to the Repository class? Completely abstracting persistence is beneficial when the core domain / logic in this area is complex enough to test it in isolation. If the logic only consists of CRUD, there is no value in carrying this extra weight. When using the Repository Pattern in a dogmatic way (and without accompanying patterns as the Specification Pattern) i often see performance problems and dependency hell (as people inject a repository for every table and this can explode when the use case has multiple tables involved). So i think starting simpler and abstracting when there is a need is a better strategy.
I hardly believe in absolute solutions when it comes to Software Architecture. For almost every decision there are pros and cons.
@@markuss.6981 Regarding your question about "how this helps unit testing":
Well, how would you unit test your controller functions with Unit Tests without the ability to mock the datalayer?!
Very controversial!
It is, if your view of an HTTP API is built around HTTP methods as CRUD and Entities/Objects as Resources, which is the predominant advocated way.
Any new architectural approach can be considered controversial for a person seeing if for the first time. The most important thing is to avoid being stuck in a comfort zone and appreciate that there are different ways of doing things. Not good, not bad, but different. And which one we use should depend on the context and specific system requirements.
@@CodeOpinion yeah I meant controversial in a good way. The only part I struggle with is where you said commands and queries can be resources and have identifiers. Posting a command does not represent a state transfer for me, I am also not so sure they have identifiers. I am not sure it is a resource. I like the general approach (and use CQRS over HTTP apis myself on a daily basis). Just difficult for me to still call it REST. It is just RPC over HTTP or poor man's gRPC or whatever you want to call it:) What's in a name. I highly appreciate the task based approach and sentiment around UX. Thanks for another great video!
@@Mvision123 I used "REST" in the title because I think that's what most people equate to an HTTP API and not really what REST was defined as (architectural constraints). I do plan on creating another video that continues this one but uses hypermedia. I might in that video address some misconceptions about REST. Appreciate the comment.