Yesssssssss, been waiting for this, I've done a few implementations in my projects but I've never been fully happy with it. Would love to see more videos on proper async data pipelines/flows👍 thanks for doing these videos man, I dropped out in first year of my Cs degree and while I can code, I always feel lacking on the infrastructure/architecture side. Your courses and videos have been a massive help for me🙏
I would love to see different techniques how to tackle idempotency! I've seen many different approaches and would be great to compare with what you've seen.
The message received by SQS has one property messageId and that id is unique no metter how many times this message is delivered, a simple way to deal with this idempotency is saving this Id
Yo Nick! I'm Xamarin dev that switched to backend half a year ago. Old viewer of yours, but now - your channel is a real GEM for me, every new video I watch is exactly what I need, just started my AWS journey and the timing of this one is magic. Thank you for your content! Also I really want to see a bonkers approach with Handlers and more AWS content overall.
I've been using this approach for processing messages via the azure service bus since generic host (background service) released in .net core 2.1. Thanks for producing these videos, I find the content very useful and practical.
Great video and pretty slick use of reflection to handle registering your services in the service container. Do you have a video that would utilize SNS and also display how the messages are deleted from the Queue? Also, would be nice to see the use of the DLQ for failed messages.
So this is a fascinating idea. The way you described Standard AWS QUeue sounds like a prioritized asynchronous queue. I can imagine a queue where all of these items are waiting to be processed, and as product becomes available it can be filled by the first order that asks for it, maybe. But its potentially optimizable because if you have someone in slot 3 asking for grapes, but slot 2 wants grapes and pomegranates but the pomegranets are delayed (and maybe you optimize to try to fill what you can complete, and if it ends up getting stuck then getting intervention somehow to break the queued item up. AWS could optimize to look for the first item in the queue that can be filled and dispatched. So I think I really like this idea, because its not like you have competing tasks waiting to be filled and potential contention that way. Pretty cool!
Hey Nick. Thanks for the video. I have a question though. If the published message is subscribed by multiple applications (or I can say users), how do we make sure all the users see the message before it gets deleted? (If we don't delete the message, one user might end up seeing the message multiple times)
I tried to run this project in dotnet 7 but it fails because IMessageHandler can be used as a type. Then I downgrade to dotnet 6 but I can use abstract in the property IMessageHandler.MessageType so I have to initialize that property with typeof(IMessageHandler), it is good ?
Hey Nick, awesome video, thanks so much for the great content and for sharing. Would be interested in how to deal with idempotency too. Looking forward to the solution architecture video too.
Your comment about the messages being in a shared project instead of duplicating the files is a good one. However when you used the executing assembly it will not find those messages, instead typeof(IMessage).Assembly would be better. Same concept for the IMessageHandler.
@@nickchapsas Now that I think about it more, like you said most developers would have understood to use typeof(IMessage) instead of typeof(Program) if they moved the files to a separate project.
Hey Nick, great video, please do more about queues and how to efficiently publish/consume messages 🙏. Kind of unrelated, but it would make sense to put together handler's registration and populating MessageDispatcher's dictionaries? (to avoid forgetting registering them in Program.cs) Many thanks!
heh, I decided to implement MessageDispatcher with generics, and it was the worse decision for the last few days) Could you please make a video\another branch in github with implementation in the way with generics/variance stuff.
Hey mate, your videos are so useful, can you please make a video on comparison between Entity Framework latest vs Dapper. Why we should use dapper over entity framework and use cases.
For the dispatcher, I'd create an IEnumerable IMessageHandler can contains { string Command {get => nameof(MessageType)} Type MessageType {get;} Task HandleAsync (); } Then register all handlers as IMessageHandler Finally inject IEnumerable in the constructor Using linq var handlers = msgHandlers.Where(mh => mh.Command == input.commans ) foreach(var handler in handlers) await handler.HandleAsync(input.Data);
Unnecessarily instantiating handlers aside, i like this approach. I would probably introduce an abstract base class like this: public abstract class MessageHandler : IMessageHandler where TMessage : IMessage { public async Task HandleAsync(IMessage message) { if (messageis TMessage m) { await DoHandleAsync(m); } } protected abstract Task DoHandleAsync(TMessage m); } so my MessageDispatcher is nice a simple like this: public class MessageDispatcher { private readonly IServiceScopeFactory scopeFactory; public MessageDispatcher(IServiceScopeFactory scopeFactory) { this.scopeFactory = scopeFactory; } public async Task Dispatch(IMessage m) { using var scope = scopeFactory.CreateScope(); foreach (var handler in scope.ServiceProvider.GetServices()) { await handler.HandleAsync(m); } } } This way you can also write handlers which work off a base message type for instance: public class AllMessagesHandler : MessageHandler { protected override Task DoHandleAsync(IMessage m) { Console.WriteLine(m.GetType()); return Task.CompletedTask; } }
@Nick Chapas, How can i rate limit a specific api which url contains and require int value. E.g api/account/details - This is like you explained - which is working but what in case if url is like this? api/account/{accountId:int}/details Please help me. im using webapi2
I never understand the obsession with adding `Async` to methods where you don't provide a sync version of it. It's just fluff. Everyone knows a task can be awaited
@@nickchapsas I definitely don’t. In fact calling it Async may be a lie. See MemoryStream for example. I also don’t add access modifiers just because; only when necessary.
@@paulkoopmans4620 Microsoft don’t even follow their own conventions on their asp code base 😂. Async is just noise as is typing private for private members.
@@metaltyphoonnot sure which occurrences you are thinking of at this moment. But... no framework and/or programming is ever 100% correct. That does not mean however that "standards" and trying to follow them is bad. Quite the opposite IMHO. I will admit that I come back to my code at times thinking... wt... why did I just not follow the coding style guide or why I not do this or that. But I am a human and so are they. But if nothing would have existed at all.... oh boy... I can just imagine how terrible that must be. Property names will all kinds of different casing. Terrible method names like b_convstr() instead of TryConvertString(), just because, you know. No!!! No!!! No!!! Even though never perfect, millions of developers following a "standard" is better than nothing. And again; I personally believe in intent of code. If a member is a private member it should be marked as such. If it were up to me I would say the implicit access modifier behaviour should have never existed. It serves no purpose! the "choice" of private as the default access modifier is as terrible of a choice as any of the other ones. My classes are usually sprinkled with lots of public methods, public properties, protected, etc, as well. I think I may usually have more of any of the other ones than private. Maybe public should have been the default :).
While I agree with you, I believe that regular developers don't need to worry about it as much as our clients - they're the ones that amazon might refuse service. We can only advise them not to use services from providers who might be unreliable like that.
@@marcinpawelw The problem arises when it's your own AWS-backed service that a client is using. They can wrongthink your entire service in to the ground. Not a liability I want to take on
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
This is the best C# channel on RUclips x 10. Sometimes it's like Nick has access to my backlog and makes videos just for me.
A simpler way to have it working nice is with Mediatr, the background service does de parse and dispatch of the messages
Your channel is amazing. It's perfectly in-depth enough to lean and start doing. As requested, more AWS + .NET Architecture videos please.
I would like to see the covariance and contravariance one!
This is honestly my favorite video. So many good patterns and examples. Hats off.
Yesssssssss, been waiting for this, I've done a few implementations in my projects but I've never been fully happy with it. Would love to see more videos on proper async data pipelines/flows👍 thanks for doing these videos man, I dropped out in first year of my Cs degree and while I can code, I always feel lacking on the infrastructure/architecture side. Your courses and videos have been a massive help for me🙏
Nick your videos are just what is required, a true scientist... well done man.
I like the fact that your videos and Topics are getting "bigger" and more detailed! thx
I would love to see different techniques how to tackle idempotency! I've seen many different approaches and would be great to compare with what you've seen.
The message received by SQS has one property messageId and that id is unique no metter how many times this message is delivered, a simple way to deal with this idempotency is saving this Id
Yo Nick! I'm Xamarin dev that switched to backend half a year ago. Old viewer of yours, but now - your channel is a real GEM for me, every new video I watch is exactly what I need, just started my AWS journey and the timing of this one is magic. Thank you for your content! Also I really want to see a bonkers approach with Handlers and more AWS content overall.
I've been using this approach for processing messages via the azure service bus since generic host (background service) released in .net core 2.1. Thanks for producing these videos, I find the content very useful and practical.
I love how you're releasing these AWS videos a few months after I had to figure it all out myself lol
Cool video! I’d probably prefer to use a message bus like mass transit to abstract most of this out for me, but still fun to see it built too
Congratulations Nick, as always a good video.
Eyy, this was super helpful and very easy to follow. Thank you! 🙏
Great info... having used MSMQ for over 10 years for Process Control.. nice to see a modern implementation of a message dispatcher
I mean even 10 years ago MSMQ wasn't "modern". Wondering why it was ever picked after 2008.
Hi Nick, great video as always. Is there any chance you could do a video on Amazon Cognito? i.e. basic setup and walkthrough
Can't wait for the architecture video ! Keep up the good work !
Great video and pretty slick use of reflection to handle registering your services in the service container. Do you have a video that would utilize SNS and also display how the messages are deleted from the Queue? Also, would be nice to see the use of the DLQ for failed messages.
I'd love to see a video for the generic message handling. That scenario comes up way too often.
Hey Nick, great video! Thank you bringing such quality content!
So this is a fascinating idea.
The way you described Standard AWS QUeue sounds like a prioritized asynchronous queue.
I can imagine a queue where all of these items are waiting to be processed, and as product becomes available it can be filled by the first order that asks for it, maybe.
But its potentially optimizable because if you have someone in slot 3 asking for grapes, but slot 2 wants grapes and pomegranates but the pomegranets are delayed (and maybe you optimize to try to fill what you can complete, and if it ends up getting stuck then getting intervention somehow to break the queued item up. AWS could optimize to look for the first item in the queue that can be filled and dispatched. So I think I really like this idea, because its not like you have competing tasks waiting to be filled and potential contention that way. Pretty cool!
Looks like you ended up not using the static abstract interface members feature as the reflection scanning would work regardless. Right?
As usual, awesome content. 🙏
Can one use NServicebus with SQS and not worry about implementing a custom dispatcher logic?
Hey Nick. Thanks for the video. I have a question though. If the published message is subscribed by multiple applications (or I can say users), how do we make sure all the users see the message before it gets deleted? (If we don't delete the message, one user might end up seeing the message multiple times)
Awesome video, although the last part 'Automatic registration refactoring' is a little bit hard and rush.
I tried to run this project in dotnet 7 but it fails because IMessageHandler can be used as a type. Then I downgrade to dotnet 6 but I can use abstract in the property IMessageHandler.MessageType so I have to initialize that property with typeof(IMessageHandler), it is good ?
Thanks for sharing all those details. I am also using SQS very intensively and love the service for its simplicity. Keep it up
Hey Nick, awesome video, thanks so much for the great content and for sharing. Would be interested in how to deal with idempotency too. Looking forward to the solution architecture video too.
Really nice video! Congrats! And as always, keep them coming… 🎉
Your comment about the messages being in a shared project instead of duplicating the files is a good one. However when you used the executing assembly it will not find those messages, instead typeof(IMessage).Assembly would be better. Same concept for the IMessageHandler.
Yeah I did mention that you would use a type or an assembly if you had it in a separate project
@@nickchapsas Now that I think about it more, like you said most developers would have understood to use typeof(IMessage) instead of typeof(Program) if they moved the files to a separate project.
Hey Nick, great video, please do more about queues and how to efficiently publish/consume messages 🙏.
Kind of unrelated, but it would make sense to put together handler's registration and populating MessageDispatcher's dictionaries? (to avoid forgetting registering them in Program.cs)
Many thanks!
No need for dictionaries, just use something like Scrutor to automatically register stuff
Hi Nick. Maybe in the near future you could create video about SAGAs?
Hi Nick! Do you use Azure (in this case the service bus) as well and can you make a comparison, or are you only on AWS?
I’ve used both in high scale production environments for 3 years each and I prefer AWS.
@@nickchapsas interesting, i would love to see a video about the reasons some day :)
Hey Nick, you don't have a link in the description below for Queues.
Sorry added now
Did you just write your own MediatR? =)
Great video, reduce font size please
Very nice video. Any chance of making a similar vid with MassTransit?
I plan to make an expansion on this video to show how MassTransit can make all this better
Great vid bro... can you do some on Dapr?
Amazing content and great explanations. But for a more pleasant reception, I would advise 10% slower. It isn't a chase :).
heh, I decided to implement MessageDispatcher with generics, and it was the worse decision for the last few days) Could you please make a video\another branch in github with implementation in the way with generics/variance stuff.
I’m planning to use this example in a future video to explain the runtime problems (which you probably encountered), variance and alternatives
@@nickchapsas That would be awesome. Thanks
Hey mate, your videos are so useful, can you please make a video on comparison between Entity Framework latest vs Dapper. Why we should use dapper over entity framework and use cases.
For the dispatcher, I'd create an IEnumerable
IMessageHandler can contains
{
string Command {get => nameof(MessageType)}
Type MessageType {get;}
Task HandleAsync ();
}
Then register all handlers as IMessageHandler
Finally inject IEnumerable in the constructor
Using linq
var handlers = msgHandlers.Where(mh => mh.Command == input.commans )
foreach(var handler in handlers) await handler.HandleAsync(input.Data);
This approach is problematic because it instantiates all handlers to just use one. waste of speed and memory
Make sense, that's why you created a dictionary
Unnecessarily instantiating handlers aside, i like this approach. I would probably introduce an abstract base class like this:
public abstract class MessageHandler : IMessageHandler where TMessage : IMessage
{
public async Task HandleAsync(IMessage message)
{
if (messageis TMessage m)
{
await DoHandleAsync(m);
}
}
protected abstract Task DoHandleAsync(TMessage m);
}
so my MessageDispatcher is nice a simple like this:
public class MessageDispatcher
{
private readonly IServiceScopeFactory scopeFactory;
public MessageDispatcher(IServiceScopeFactory scopeFactory)
{
this.scopeFactory = scopeFactory;
}
public async Task Dispatch(IMessage m)
{
using var scope = scopeFactory.CreateScope();
foreach (var handler in scope.ServiceProvider.GetServices())
{
await handler.HandleAsync(m);
}
}
}
This way you can also write handlers which work off a base message type for instance:
public class AllMessagesHandler : MessageHandler
{
protected override Task DoHandleAsync(IMessage m)
{
Console.WriteLine(m.GetType());
return Task.CompletedTask;
}
}
sadly we cant use aws services in iran,thats just make me feel depressed
@Nick Chapas, How can i rate limit a specific api which url contains and require int value.
E.g
api/account/details - This is like you explained - which is working
but what in case if url is like this?
api/account/{accountId:int}/details
Please help me. im using webapi2
I never understand the obsession with adding `Async` to methods where you don't provide a sync version of it. It's just fluff. Everyone knows a task can be awaited
Then in the code that you write you should definitely not use that suffix
@@nickchapsas I definitely don’t. In fact calling it Async may be a lie. See MemoryStream for example. I also don’t add access modifiers just because; only when necessary.
@@ShiyalaKohny To clarify, I never meant to be rude and if that’s how it came across, I want you to know it wasn’t the case @Nick Chapsas.
@@paulkoopmans4620 Microsoft don’t even follow their own conventions on their asp code base 😂. Async is just noise as is typing private for private members.
@@metaltyphoonnot sure which occurrences you are thinking of at this moment. But... no framework and/or programming is ever 100% correct. That does not mean however that "standards" and trying to follow them is bad. Quite the opposite IMHO.
I will admit that I come back to my code at times thinking... wt... why did I just not follow the coding style guide or why I not do this or that. But I am a human and so are they.
But if nothing would have existed at all.... oh boy... I can just imagine how terrible that must be. Property names will all kinds of different casing. Terrible method names like b_convstr() instead of TryConvertString(), just because, you know. No!!! No!!! No!!! Even though never perfect, millions of developers following a "standard" is better than nothing.
And again; I personally believe in intent of code. If a member is a private member it should be marked as such. If it were up to me I would say the implicit access modifier behaviour should have never existed. It serves no purpose! the "choice" of private as the default access modifier is as terrible of a choice as any of the other ones.
My classes are usually sprinkled with lots of public methods, public properties, protected, etc, as well. I think I may usually have more of any of the other ones than private. Maybe public should have been the default :).
👏🏽👏🏽👏🏽👏🏽👏🏽
Yeah, get on AWS, so they can turn you off if you have the "wrong ideas". No thanks.
While I agree with you, I believe that regular developers don't need to worry about it as much as our clients - they're the ones that amazon might refuse service. We can only advise them not to use services from providers who might be unreliable like that.
@@marcinpawelw The problem arises when it's your own AWS-backed service that a client is using. They can wrongthink your entire service in to the ground. Not a liability I want to take on
5:13 Am I only one person who expected a random id as 69
god forbid any poor soul has to write such C#
I like your videos but you speak so fast. Could you speak more slowly, please?