Clean Message Bus Consumers with MediatR in .NET

Поделиться
HTML-код
  • Опубликовано: 8 сен 2024
  • Enroll to "Cloud Fundamentals: AWS Services for C# Developers" for FREE: bit.ly/3XKUBOH
    Get the source code: github.com/Elf...
    To get $25 worth of free AWS credits email dotnet-on-aws-feedback[at]amazon[dot]com with subject line "AWS CREDIT NICK CHAPSAS".
    This video is sponsored by AWS
    Hello everybody I'm Nick and in this video I will show you how you can build clean and elegant message consumers for queues, topics and event streams using ASP.NET Core and MediatR.
    Don't forget to comment, like and subscribe :)
    Social Media:
    Follow me on GitHub: bit.ly/ChapsasG...
    Follow me on Twitter: bit.ly/ChapsasT...
    Connect on LinkedIn: bit.ly/ChapsasL...
    Keep coding merch: keepcoding.shop
    #csharp #dotnet #aws

Комментарии • 149

  • @nickchapsas
    @nickchapsas  Год назад +20

    Since many of you prefer loading the MessageTypes using scanning, I updated the code to use it too.
    You can find the updated code here: github.com/Elfocrash/aws-videos/blob/master/MessageProcessingMediatR/Customers.Consumer/QueueConsumerService.cs#L24

    • @kevinlloyd9507
      @kevinlloyd9507 Год назад

      Another vote for scanning. Forcing one namespace prevents you from colocating message types if using Vertical Slice Arch.

    • @nickchapsas
      @nickchapsas  Год назад +1

      @@kevinlloyd9507 The namespace of a type is usually the same with the folder but doesn't have to be. That's a common .NET misconception.

    • @kevinlloyd9507
      @kevinlloyd9507 Год назад

      @@nickchapsas I'm aware, but IDEs usually make it painful if they don't match :)

    • @ConstantinStoica
      @ConstantinStoica Год назад

      In a real application wouldn't it be better to have separate consumers for each message type ? So that the load can be better distributed.

    • @pilotboba
      @pilotboba Год назад +1

      @@ConstantinStoica I think consumer per aggregate or maybe per queue topic is fine. Having a consumer for each message in my opinion is much too much. But, YMMV.

  • @connorradeloff5597
    @connorradeloff5597 Год назад +14

    One thing I might change about the type resolution system is instead of using the direct magic string in Type.GetType(), I believe it would be better to create a message type resolver which is added to DI which scans over the assembly at startup and maps the TypeName -> Type using a dictionary. This way, you're not reliant on the messages living within a specific namespace of the project to resolve types, and can even get fancy using attributes or additional properties to do things like override the name or do conditional type resolution based on values within the type if you need to do something funky. But that's very niche.
    Great video as always!

    • @nickchapsas
      @nickchapsas  Год назад +9

      I have used both approaches and both have advantages and disadvantages. For this scenario I prefer not having the scanning because it’s way more understandable of an approach and there is not magic that the developer needs to be aware of. In fact, when I showed that very approach (scanning) in a previous video, many people comments negatively about disliking it. In any case, I updated the code with the scanning approach too just so people have a choise to go with whatever they prefer. Check the pinned comment

    • @connorradeloff5597
      @connorradeloff5597 Год назад +1

      ​@@nickchapsas Both methods have their merits! Assembly scanning is how *I* would have done it, but we all know software is a tool and you use it to best suit the use case. Thanks for updating the code.

    • @robsosno
      @robsosno Год назад +1

      Yet another way - in similar cases I'm usually using Dictionary.

    • @nickchapsas
      @nickchapsas  Год назад +2

      @@robsosno Yeap, check the pinned comment

  • @majidmajid2192
    @majidmajid2192 Год назад +1

    Couple days ago i create a consumer but in the bad way without mediatr...
    Thanks for sharing for using mediatr. It is nicely fit for this usecase

  • @metaltyphoon
    @metaltyphoon Год назад +4

    There is always a “switch" somewhere, its just a matter of if you want to see it in your code or not ☺️.
    One thing that was not touched here would be tracing. This worker service wouldn’t show up on a distributed tracing (unless proxy sidecars were used)

    • @nickchapsas
      @nickchapsas  Год назад +15

      It’s goto statements all the way down

  • @mackie1001
    @mackie1001 Год назад +1

    The 2 arrows going to DB and queue in your diagram raised an eyebrow. I see lots of people make that mistake so a demonstration of how to deal with multiple resources without 2PC would be a nice topic to cover.

  • @Rajeshsingh-ws5th
    @Rajeshsingh-ws5th Год назад +4

    Great one, specially to have dynamically handler to be passed into scoped services. Thanks keep posting good videos.

  • @Dhaiky
    @Dhaiky Год назад +8

    I would love to see a video about Vertical-Slice architechture

  • @mikem4008
    @mikem4008 Год назад +1

    I know this focuses more on the consumer side of thing, but the producer is also a huge topic. I've normally used CDC events to handle this type of thing so that you don't have to worry about synchronizing the DB transaction with SQS. I haven't looked to see if AWS SQS supports integration with AWS DMS. I use DMS CDC events with Kafka, although your consumer now needs to know a little more details about the DB and not just the event type. Another approach is the outbox pattern for proper synchronization.

    • @nickchapsas
      @nickchapsas  Год назад +3

      CDC is how you should be doing this. That’s why I didn’t focus on the publisher in this video, because I’m "hacking" it with not using CDC to simplify the video. I’ve already made a video on CDC with DynamoDB streams and I have a dedicated section on it in my free AWS course.

  • @sushilb7994
    @sushilb7994 Год назад +1

    Excellent video Nick. A very clean way of handling the messages. Loved it! Thank you

  • @romanciesielski10
    @romanciesielski10 Год назад

    Thanks for good video! I want to share one issue which I see with solution.
    In code you check if you have message implementation of message name which received and then call Mediator handles. Issue here that you will not have handlers for all message in real scenario. Some messages you can have only to inform another service. So Mediator will have exception trying to get handler for message which you received, but not have implementation.
    Thanks again for you work!

  • @nove1398
    @nove1398 Год назад

    This is the typical flow I use more or less, really nice explanation of scoping also. Great!

  • @isnotnull
    @isnotnull Год назад

    In such cases I prefer to create my own Mediator implementation which works as a pipe. It is really simpler then using IMediatr and the only overhead is calling every handler when handle the message (every handler processes own message type and declines other types). No magic and may be easily tested

  • @CobraTrainer
    @CobraTrainer Год назад

    Just created a consumer a couple of months back and used the switch way you showed as bad practice lol. Wish i had seen this vid before, having everything in its own handler is pretty awesome. I can always refactor i guess :)

  • @bloopers2967
    @bloopers2967 Год назад +2

    Can you do the same video with azure service bus? Please

  • @andrewogu6451
    @andrewogu6451 Год назад +1

    Always love your content, Nick. Well done.

  • @AdamWeigert
    @AdamWeigert Год назад +1

    I used MediatR for a similar concept with our event-driven apps based on Azure Event Grid, get the message and parse it and publish.

  • @PaulVanGundy0
    @PaulVanGundy0 Год назад +1

    Hi Nick, can you talk a little about how you deploy the message service to production. Is there any special configuration needed to keep it alive in IIS?

  • @onedev7316
    @onedev7316 Год назад

    Another excellent video. Thank you. Could you please do a video on User Impersonation using JWT/Identity Thanks.

  • @Vityhard
    @Vityhard Год назад

    As always clear explanation. Thanks. Can't wait when you drop some DDD course :)

  • @zglmorph
    @zglmorph Год назад

    I'd love to see more about how to unit test code like this. Do you basically just write the message loop once, get it working, and then not worry about testing it further, and only focus on testing the handlers? Or do you get tests around the message loop too? It seems like the message loop would need some refactoring in order to unit test it, since the code you wrote in this video depends so directly on the handlers.
    I'd also love to hear whether you've used contract tests like Pact. It seems like it could be useful here, to make sure that the sender's JSON matches all the properties you're expecting to receive here, even without putting the types in a shared assembly.

  • @yoanashih761
    @yoanashih761 Год назад

    Thanks Nick. If your course website could support dark mode that would be great.

  • @plamenstanchev21
    @plamenstanchev21 Год назад

    verry elegant

  • @user-sv9fe5qf5h
    @user-sv9fe5qf5h Год назад

    Great approach! Thank you!

  • @KojiKazama
    @KojiKazama Год назад

    I love your videos. You make things look clear;

  • @adrianmustelier2870
    @adrianmustelier2870 Год назад

    Thanks for the great work as always. Why not use a Factory pattern?

  • @pilotboba
    @pilotboba Год назад +1

    Should the message types be in a shared project so they can be treated as contracts between the producer and the consumer?

    • @nickchapsas
      @nickchapsas  Год назад +1

      Yes, absolutely. They should live in a single package that the publisher owns, and shares with the consumers

    • @qwer1234cvb
      @qwer1234cvb Год назад

      @nickchapsas, but with this approach we are adding a dependency to MediatR library to the consumer contracts, thus forcing producers to have this dependency too, aren't we? And if producers are using MediatR already, but of differerent versions - there's going to be a problem.

    • @qwer1234cvb
      @qwer1234cvb Год назад

      @@nickchapsas but with this approach we are adding a dependency to MediatR library to the consumer contracts, thus forcing producers to have this dependency too, aren't we? And if producers are using MediatR already, but of differerent versions - there's going to be a problem.

  • @ak_nex
    @ak_nex Год назад

    Hi, Nick. Thanks for the video)
    The approach you showed is applicable but only in straightforward scenarios.
    Other commentators rightly noticed, that message classes can be placed in different assemblies to be reused as Nuget packages. The best solution is to pass somehow the types to the consumer.
    Apart from that every message type more often belongs to its own queue, so in this example we need to duplicate consumers for each queue.
    Both of these problems make consumer configuration much more complicated.
    And the last but not the least point is to handle a message context (envelope pattern) which is often transferred in headers. In the context there can be a lot of useful information like trace id or request id when request/response pattern is used. Generic abstract class doesn't seem like a good idea, since a wrapped message class might have property name collisions in this case. So, MediatR has its downsides as well.
    Nevertheless, i really wish i had seen this video before dived deep into the topic)
    P.S.: really waiting for ConfigureAwait() video

    • @nickchapsas
      @nickchapsas  Год назад +1

      I don't think that's true. If you want to dynamically load your types, simply assembly scan them based on the interface, add them in a dictionary and load them like that. Then you don't have to worry about where they are located.
      I also explained in the video that if extra message metadata matters to you, add those fields in the ISqsMessage interface and map them from the SqsMessage object before you end it to MediatR. Both extremely simple and straightfoward changes.

    • @ak_nex
      @ak_nex Год назад

      @@nickchapsas yeah, agree, it makes sense to handle additional fields in the cosumer without passing them down to the mediator.
      Scanning is not really necessary here because message class type and related queue must be linked, likely in DI, and then these links are passed to the consumer

  • @dkaps06
    @dkaps06 Год назад

    Another excellent video. Thanks for sharing!

  • @SunriseTequila523
    @SunriseTequila523 Год назад

    Hi @Nick Chapsas, thanks for the video and i wanted to ask what tablet or Wacom board? (I guess) do you use to draw while you are explaining anything?

  • @escobar4599
    @escobar4599 Год назад

    For learning, I might like to take full control of things. In the early 90s I think I must have written a client and server implementation of every RFC network protocol I could find good use for, and it was a great way to understand how things really work down to the protocol level.
    But for a pro/team project where the skillsets can vary and change over time as people come and go, I skip right past libs like Masstransit and go right to Dapr. Once you have done the pub/sub by hand once there is no value to do it a second time - so why tie yourself to a vendor/sdk? Besides, the handler that Dapr invokes can be a web method, and so I still get to use MediatR ;)

    • @nickchapsas
      @nickchapsas  Год назад +3

      Having been working with queues, pub/sub, messaging etc for the past 7 years in more than 200 microservices dealing with thousands of messages per second, I never needed to switch either the SDK or the vendor (or any of the others teams for that matter) and I've worked with both major cloud providers. The lie that "you might need it in the future" is nice, but I've worked long enough to know its a lie. Direct control of the SDK and the pipeline in advanced scenarios can't be matched by libaries like MassTransit. MassTransit makes a lot of assumptions and has a lot of opinions that as a poweruser, I don't like. It a cool library, but for the stuff I'm working on, I prefer control.

    • @nickchapsas
      @nickchapsas  Год назад +8

      As a matter of fact, I'll make a video on it explaining in detail.

    • @ivanpavlovnorth
      @ivanpavlovnorth Год назад

      ​ @Nick Chapsas Video about your opinion and experience of using Masstransit/ NServiceBus or any wrappers will be very useful for many developers!
      I did not find any decent video about criticism of these libraries over a “raw” client.

  • @dennycrane2938
    @dennycrane2938 Год назад

    Here's an opinion question regarding onion/clean architecture -- I typically put my message handlers for mediatr in my application layer because it essentially covers the use cases. But what about an external bus like sqs, rmq, etc? It feels like infrastructure, but then you end up having external bus handlers in one layer that basically just takes the external message and translates it into a command to hand off to the application layer. This is actually what I do and I'm ok with it (it does make sense in terms of separation of concern) but I admit it feels redundant. I could just orchestrate the application and domain layers in that handler, but then you have use cases handled in external handlers and others in mediatr handlers and it should probably be more consistent. Thoughts?

  • @oriRelastwish
    @oriRelastwish Год назад +1

    Great video Nick, again.
    A question if you may.
    What happens in cases where you have multiple handlers for the same message, is this approach still viable?
    Would you then write a single handler and within it call the other handlers (which do not implement IRequestHandler so they wont subscribe to the message as well)?
    Note that if you have multiple handlers implementing the same IRequestHandler, naming conventions will be lost and chaos will start to reign.
    Moreover, I understand you support this mediator approach, but is it good to have implicit code executions in big projects? whats your point on that?

    • @JavierAcrich
      @JavierAcrich Год назад

      I think you should use SNS, instead of SQS for that. SNS allows you to configure multiple subscriptions to a single SNS message.

    • @oriRelastwish
      @oriRelastwish Год назад

      @@JavierAcrich SNS is for when you have multiple subscribed applications. This is a subscription inside an application, but it might have more than a single handler, that’s different.

    • @btastic2
      @btastic2 Год назад

      I am currently designing a similar system. I am currently evaluating whether I could just use MediatR Notifications and use multiple NotificationHandlers. Thoughts on that approach?

  • @MinhNguyen-zx8me
    @MinhNguyen-zx8me 3 месяца назад

    I can't run the Consumer project. I am getting the "No RegionEndpoint or ServiceURL configured" error. Please tell me how I configure those urls/endpoints. Thank you

  • @rasimismatulin1400
    @rasimismatulin1400 Год назад

    Some advantages can help simplify code.
    1. Mediatr have nice IPipelineBehavior. It can use for exception catching and logging.
    2. Instead of return Unit, create base types Result. And in ExceptionHandlingBehavior in catch block do logging and create Result.AsFailed(ex).
    3. In worker service use if (!result.IsSuccess) continue;

  • @stanislav7228
    @stanislav7228 Год назад +2

    Thanks for the video!
    What do you think about the MassTransit library? It supports all the staff you showed under the hood.
    It has both functionality similar to MediatR and the support of the most popular service buses, such as Amazon and Azure ones.

    • @nickchapsas
      @nickchapsas  Год назад +1

      Mass transit is fine, I just prefer having full control of the pipeline and the code that runs on it. MassTransit is like bringing a bazooka in a watergun fight

    • @coderider3022
      @coderider3022 Год назад

      Mass transit really needs producers and consumers to run it, it’s overkill for most things.

    • @nickchapsas
      @nickchapsas  Год назад +7

      Maybe making a video comparing the performance of using MassTransit vs a hand rolled processing pipeline would be interesting.

    • @JollyGiant19
      @JollyGiant19 Год назад +1

      @@nickchapsas I agree because I’m looking into that very same comparison. Hand rolling or MassTransiting my project.

    • @TimGrannen
      @TimGrannen Год назад +1

      Been investigating MassTransit recently for a client. It certainly has a learning curve to solve this simple problem, but the robust error handling ability (retries with backoff, kill switch, etc) is really what my project needs. And I'm definitely a fan of not having to roll my own for that.

  • @BonBaisers
    @BonBaisers Год назад +1

    (Senior Architect) Well Nick, it's not that *clean*. The message attribute is deeply coupling your code to the emiter of the message. Having a contract on both (common message model) kills the main objective of a Bus imho. Emiters should be free of any contract while consumer must implement a specific handler. Emiters can be black box solutions where u have no control anyway. I would have use the mediator pattern, the approach is perfect, but introduce an IHandler interface with a CanHandle(IMessageAttributes) and Handle(IMessage) method, register the handlers with IoC or a custom mediator that will call the fist or all handlers who respond true to CanHandle.

  • @Sky4CE
    @Sky4CE Год назад

    this one is really good, thank you

  • @ArgenisVillarroel
    @ArgenisVillarroel Год назад

    Great implementation for MediatR!!! Thanck crack

  • @dariuszlenartowicz
    @dariuszlenartowicz Год назад

    Foe every message which is published I use DataContract attribute. Thanks to that I can change/correct name of message class or move it to different namespace - name from DataContract remains the same and app works like nothing changed.

    • @nickchapsas
      @nickchapsas  Год назад

      Spending time reading attributes is wasteful. These are contracts. Names and namespace changes are breaking changes that have no reason to happen. Especially considering that you, as the consumer, don’t own them in the first place

    • @MistyKu
      @MistyKu Год назад +2

      @@nickchapsas You could add some cache there. You can use attributes or just search for implementations of a particular interface which I think is cleaner, it can even be cached on startup. The problem with your approach is that after a couple months or years, someone will come and refactor the code. If I was doing a refactor, I wouldn't expect that someone hardcoded a namespace somewhere in the code, I think it's a bad practice.

    • @nickchapsas
      @nickchapsas  Год назад

      @@MistyKu You don't refactor contracts. Your logic assumes that the developer does something fundamentally wrong.

    • @MistyKu
      @MistyKu Год назад

      @@nickchapsas Why is namespace part of a contract? It's just a location where your dto resides. I'd say that contract is the shape of dto (message), including type name but not the namespace itself

    • @nickchapsas
      @nickchapsas  Год назад +1

      @@MistyKu We agree on that. I guess when I've worked with contracts in teams it was never something I was controlling. All I was given is a nuget package and that's that. I can't change it, and really, I shouldn't because it is someone else's. Even if I controlled it, I still wouldn't use the data contract attribute. I'd just use assembly scanning to load the types in a dictionary and resolve from that. Check the pinned comment to see what I mean.

  • @lukassinkus6162
    @lukassinkus6162 Год назад

    How would you handle a one to many event (INotification)? I find it hard figuring out how to ensure retries that don’t retry handlers that have succeeded

  • @anujguleria869
    @anujguleria869 Год назад

    Hi Nick, If we use aws lambda , would it be more cheap as you dont need the consumer always running?

    • @nickchapsas
      @nickchapsas  Год назад +1

      Depends on how much you’re processing. For smaller throughput, yes

  • @alexbagnolini6225
    @alexbagnolini6225 Год назад

    What about using source generators to generate the MessageType -> Type switch?

  • @JavierAcrich
    @JavierAcrich Год назад +1

    I think this can also be written without breaking the Open Close Principle using MassTransit.

  • @ArnonDanon
    @ArnonDanon Год назад

    Quick question, if you have to perform a task that takes longer than a second per message to complete would you still use that approch?

    • @nickchapsas
      @nickchapsas  Год назад

      Yeah the await is asynchronous so you will wait for however long your processing is plus one second

    • @ArnonDanon
      @ArnonDanon Год назад

      @@nickchapsas i meant, await the proccessing of a message which takes one second and not second between each message proccessing.

    • @nickchapsas
      @nickchapsas  Год назад +1

      @@ArnonDanon Yes, I understood what you meant. The approach doesn't change no

  • @R0ckyRocket
    @R0ckyRocket Год назад +1

    Great video!
    When do u think its more suitable to use MediatR vs MassTransit?
    Ive used MassTransit and it very good to consume messages from SQS, I wonder if there are any benefits for MediatR?

    • @nickchapsas
      @nickchapsas  Год назад +2

      MassTransit is good but I prefer the simplicity of simply having a single queue consumer and mediatr to do all the fanning out and that's it.

    • @andreireinus9026
      @andreireinus9026 Год назад

      @@nickchapsas
      The idea is correct. Mediatr is an entry to my application, and everything around is an IO layer.
      First, you could create a single consumer that implements all the message contract consumption, and all they do is send messages to MediatR.
      However, usually, the message contracts are coming from other services, and then your deserialization logic grows out of hand and you will be missing a ton of features of MassTransit.

    • @CarmenSantiNova
      @CarmenSantiNova Год назад

      ​@@andreireinus9026 We use MassTransit in the way @nickchapsas describes. The biggest reason is that we want to be able to easily switch between queue implementations.
      But we don't use MassTransit directly but rather behind an custom abstraction to avoid Nuget hell (we don't want to have MassTransit references all over the place).
      There are however a lot of features in MassTransit that you wouldn't want to use in production such as retry policy and circuit breaking as these concerns are for i.e. infrastructure layers such as service meshes, etc.
      So in that sense I agree that MassTransit might be on the heavy side...

    • @zabustifu
      @zabustifu Год назад +1

      I'm used to MassTransit too. I must be missing something, because I see no real advantage in using MedatR here. Having to manually build the namespace seems very clunky to me, and will cause extra maintenance when the namespace changes later. Also, I'm not sure how you're supposed to handle a type that might exist in multiple namespaces (of course, you could avoid that scenario, but namespaces were created to avoid such conflicts).

    • @nickchapsas
      @nickchapsas  Год назад

      @@zabustifu Check the pinned comment

  • @joshstather3543
    @joshstather3543 Год назад

    Hey Nick, what's your thoughts on AWS Event Bridge? I've used it in a production context with pretty good results, and IMO it tends to be more applicable than SQS in many of the systems I find myself needing to build... especially for CQRS type systems. I also noticed that it is not covered in your AWS course. Do you have experience using it, would you advise against it? Thanks

    • @nickchapsas
      @nickchapsas  Год назад +1

      I'll make a video on EventBridge at some point. It's really good but a message and an event is a bit of a different concept so I don't consider it a direct replacement. I use both in my systems for different reasons

  • @condar15
    @condar15 Год назад

    So the example you're using has the message type matching the name of the request message that's handled, however I've seen different naming formats for message type in the wild, e.g. "customer-created", or "Customer.Created".
    How would you go about handling this sort of problem, my gut is to use a message type registration like you've got in your pinned comment mixed with the static interface members from C# 11 so each message would define it's own message type. Alternatively you could achieve similar behaviour in a less strict manner by the usage of attributes.

    • @nickchapsas
      @nickchapsas  Год назад

      Yeah, if the publisher uses weird names in the message type then you could take those and map them to types you own and so on. I’ve only seen this be an issue with the publisher is written in a different programming language

  • @pramod.kulkarni9607
    @pramod.kulkarni9607 Год назад

    Great learning tq...

  • @PetarPetrov_hh
    @PetarPetrov_hh Год назад +1

    where did you learn all this stuff? Your content is really advanced and also clear for junior devs! Amazing keep it up

    • @lukkkasz323
      @lukkkasz323 Год назад +1

      A lot of practice most likely. I agree, he's explaining these topics very well for beginners.

  • @ilyahryapko
    @ilyahryapko Год назад

    Nice video!
    What about EventNameAttribute approach or Name-Class mapping? I've seen a problem about type Name or FullName used as identifier: we can't move to different assembly or rename code class

    • @nickchapsas
      @nickchapsas  Год назад +1

      You wouldn’t rename the class since you are the consumer and you have no say to what it’s called. It’s coming from the publisher. There is no reason why you’d also change the namespace. In a real proper setup, these types are just given to you through a nuget package and that’s it

    • @alzaimar
      @alzaimar Год назад

      @@nickchapsas Why is there no reason to change the namespace? What if your service grows and somebody else (as always) decides to clean up the message namespace structure. Of course, the classes would not be renamed (you'd have a unit/architectual test for that, right?) but namespaces can change easily. So either add a unit/architectual test for this to make sure nobody accidentally changes stuff like that or make your software immune to this kind of changes.
      Never really argue:"This will not happen, as there is no reason".
      Well, and never say never :-)

    • @nickchapsas
      @nickchapsas  Год назад

      @@alzaimar The namespace doesn't have to change if you move the class around. Folder structure and namespace are different things. It is a good practice (and I am biased because I've only worked in companies that do this properly) to not change namespaces when you are serving your consumer with a package. If there is a change, it is being communicated before hand :)

  • @btastic2
    @btastic2 Год назад

    Hello Nick. We are doing something similar right now, however, one key question we had is, what do we do when we need something from another microservice when handling the consumers? E.g. We read the "OrderCreated" event, but now we need more customer information from the customer that is executing the order, from the customer microservice? Thank you!

    • @nickchapsas
      @nickchapsas  Год назад +1

      You usually use the customers api to get more information using the customer id from the message

    • @btastic2
      @btastic2 Год назад

      @@nickchapsas that's what we do right now. Feels a bit weird jumping between systems a lot. Thought there was a better way. I assume grpc is the method of choice here? Thank you

    • @volodymyrusarskyy6987
      @volodymyrusarskyy6987 Год назад

      @@btastic2 it depends how you implemented your app. gRPC won't help you for sure, it wasn't meant to solve this problem.
      What I typically see is a CQRS-based apps that have Command Handlers and Event Handlers (command will be "CreateOrder", event(s) will be like "OrderCreated", "OrderNotificationSent" etc). Command handlers receive incomplete information (for example, ShoppingCartId) prepare the events (contain all the required info like user's full name, address, total price, etc), emit them and event handlers do the processing. If you have something similar and fetch the data inside an **event handler**, then you are doing it wrong because an event handler must be as lightweight as possible and throw exceptions in extremely rare cases (for example, DB is down). Otherwise you will end up implementing sagas which I can assure you isn't so easy ))) Fetching data from an external service definitely increases your chances to fail 1000x times, so it is for sure a "no go". BUT you can move this fetch operation to the command handler and add all the required information into your "OrderCreated" event.
      If you have a single level implementation (just event handlers) then it again depends on the implementation. I would try to eliminate external calls at all (basically, an event emitter provides me all the required information) but in your case it might be simply not possible (for example, an event is too large and can't be sent over the transport layer).

  • @ib0018
    @ib0018 Год назад

    I suggest using CAP. Much cleaner and barely need any configuration

  • @meskinsenad
    @meskinsenad Год назад

    Nice video, thanks.
    I'm not familiar with SQS but I have one question: if you don't delete the message will it be in the next "_sqs.ReceiveMessageAsync" message list?

    • @nickchapsas
      @nickchapsas  Год назад +2

      After its visibility timeout elapses, yes

    • @meskinsenad
      @meskinsenad Год назад

      ​@@nickchapsas so if the visibility timeout is long enough list could be overwhelmed with the messages that get ignored and no real messages would be processed. :( Thanks.

    • @nickchapsas
      @nickchapsas  Год назад +1

      @@meskinsenad Wait what? No it wouldn't. Messages can still be read while others are being processes. This isn't a FIFO queue. It's exactly the same as RabbitMQ, Azure Service Bus etc

  • @JoeEnos
    @JoeEnos Год назад

    Since you dequeue the message, then work it, then delete it - seems there’s a possibility of pulling the same message twice if you have two parallel consumers, or if there’s a catastrophic failure after the message is processed but before it’s deleted.
    Is there a pattern to deal with that? Seems a simultaneous pull+delete would solve the first but not the second one.

    • @nickchapsas
      @nickchapsas  Год назад

      Yes there is. It's called idempotency. Effectively you design your operation so that no matter how many times you process the message, the result is the same. There are many techniques to do this usually involving event sourcing or optimistic concurrency or write conditions etc

    • @JoeEnos
      @JoeEnos Год назад

      @@nickchapsas Gotcha. I guess it makes sense that I’d have to do that on my side rather than relying on the library.

  • @kostyainfine5528
    @kostyainfine5528 Год назад

    if I change the message's namespace this approach would fail. normally you don't want to hardcode your namespaces

  • @koldolmen5837
    @koldolmen5837 Год назад

    Great video Nick! I've got a question that perhaps you could answer. At work we have multiple function/api projects with their own program.cs (startups). Some of them share the same services so we have to add the service in program.cs for dependency injection. Every now and then a developer forgets to add a service in one of more of the projects and stuff starts crashing. Do you have any good solution for this, like some way that we wouldnt need to add the services in every project? I was thinking of perhaps creating an extension method that adds services that multiple projects use, so that those projects that share services can use this extension method? Would it be bad if projects added services that they don't need? There might be services that 2/4 projects use but all of them will add it because of the extension method. Help a junior developer out pls :D

    • @bartvankeersop8218
      @bartvankeersop8218 Год назад

      Just run the service and you will get an error that it can’t resolve a certain dependency. Don’t make a giant extension method, that’s not modular nor maintainable. You should be able to catch errors like that quickly if you run the service locally, which you should do anyways before checking it in to master.

    • @bartvankeersop8218
      @bartvankeersop8218 Год назад

      Note that you can and should group dependencies together. If you have a NuGet package that exposes a SetviceXClient that requires a TokenClient, Timer and HttpClient for example, you can put all those behind an extension method called AddServiceXClient. That way you don’t have to setup the whole dependency chain every time.

    • @faress6934
      @faress6934 Год назад

      You can use reflection to find all the services (that implement some common interface for example) automatically and register them in DI.

  • @JamesSecretofski
    @JamesSecretofski Год назад

    They just released MediatR v12 and it broke everything. :(

  • @venud1800
    @venud1800 Год назад

    Can we expect similar video about Kafka consumer using Mediatr?

  • @TheSimonDavidson
    @TheSimonDavidson Год назад

    Do you see any issues using this alongside a Channel Writer / Reader?

    • @nickchapsas
      @nickchapsas  Год назад

      I don’t see any issues but I also don’t see any reason to in this particular scenario

    • @TheSimonDavidson
      @TheSimonDavidson Год назад

      @Nick Chapsas a factory spins up multiple readers to run in multiple threads so concurrent messages can be processed.

    • @nickchapsas
      @nickchapsas  Год назад

      @@TheSimonDavidson But that might not be what you want. Especially with queue consumers, it is generally better to scale using competing consumers instead of multithreading in a single service. You get better scaling, better resilience and better processing since SQS treats the services differently. Maybe if you are processing something like Kafka, it makes more sense

    • @The_ideal_optimum
      @The_ideal_optimum Год назад

      @@nickchapsas Hey Nick, thanks for the awesome vid. can you please elaborate on that. I might just not understand what you meant. Do you mean one should process a single request at a time in a consumer service? I can see that multithreding can be a mess in a complex service with lots of logic, especially when an error can occur. honestly I never worked closely with queues to understand how they work (e.g. if I use multithreading and one read a message, does that mean that message is "tied" to the consumer up until the broker get an ack message that the process is completed, but what if the consumer fail and crash, the message is "unavailable" to other consumers? thanks for clarifying.

    • @nickchapsas
      @nickchapsas  Год назад +1

      Hey @matan3611. No I did not mean that. You can scale your in-app processing by using multithreading with multiple consumers, but you don't need to use Channels for that. Channels are meant for high performance scenarios, processing messages buses where you iterate 10 messages that will be bottlenecked on IO (reading and deleting them) will make the gain of using channels, basically 0. Reading an event stream is a way more high throughput scenario, which in that case, will have visible benefits to use channels.

  • @adrian_franczak
    @adrian_franczak Год назад

    What about worker Project template?

    • @nickchapsas
      @nickchapsas  Год назад +1

      Not a fan of that approach for consumers

    • @ogy23
      @ogy23 Год назад

      @@nickchapsas Any particular reason?

    • @nickchapsas
      @nickchapsas  Год назад +2

      @@ogy23 Because it uses the Host class as the application abstraction which I don't like. Like I said in the video, you will always have healthcheck endpoints and metric ones, so using the WebApplication abstraction is just more appropriate

    • @ogy23
      @ogy23 Год назад

      @@nickchapsas Makes sense, thanks for answer!

  • @TechySpeaking
    @TechySpeaking Год назад +1

    First

  • @denielalain5701
    @denielalain5701 6 месяцев назад

    Hello!
    I do not aggree to a solution being "cleaner" because the same spagetti code might run in the background of mediator, and the fact that i can not see its actual code makes me even more distrustful. You have indeed shown how does it work, you have even referenced the subject in which you are teaching yourself personaly, but even then you result to "use this easy solution instead". You are teaching indeed, but i know well, that behind mediator there might be a similar spagetti code you are showing as a bad example, which i can not even see due to it being a nuget behind the "mediator scenes". You are teaching indeed, but i am either not on the level of knowledge, or even though you have shown everything, you still conclude to "use this easy solution instead" anyway, and that makes me not very happy with this video. Just think about your approach, and show how would you go about it, with mentioning that mediator is a good solution as well, instead of showing mediator right out of the bat, saying the "easy" solution is this, because whatever can i see in your video does not make me smarter yet. You have a bunch of good info of how things work, but this video looks like an advertisement of mediator for me personaly. I have noticed that most of your videos are teaching to use -bugets- nugets effectively, which is super handy to me, but in some other videos, like this, you actualy attempt to teach other things beside it, and the two just does not fit together well i think. I do not know my exact issue, i can feel that this video is a little bit off due to being informative, but it also feels like you try to prove that mediator is worthy to use instead of showing your approach. More over we can see a "bad" approach at around 12:13 as far as i am understanding. Could you teach with a good approach, please? Mentioning bad approaches is okay to me, but not mentioning any good approaches, and even cuting it down to use nugets instead is just not very feasable to me. Sorry. It is very likely that i miss a lot of knowledge, and information, and i am telling all my comment as somebody not very experienced in this.

  • @Paul-uo9sv
    @Paul-uo9sv Год назад

    Thanks dude

  • @pramod.kulkarni9607
    @pramod.kulkarni9607 Год назад

    Do we have any webhooks options from the azure service bus queue for the consumer when u get message into queue that webhook url will be triggered if we have this i can remove background service in my system completely