Hi Arpit , Amazing content and explanation :). I have a query regarding one of your synchronous calls example - Payment services. I think i have seen asynchronous calls being used for Payments now a days (Example - Amazon) since PGs have higher latencies so its better to process the payment/order initially for user experience and then use a webhook for updated status later. Thats why Amazon gives a message after order completion incase payment failed from PG and we need to re-do the payment.
Aprit, I am curious to know how Async communication can make the loose coupling between the services. Because in that scenario, if anything changes, you need to inform the caller, Whether it is schema changes, API changes, or event definition changes.
But so long as communication interface does not change each service can independently be developed. This is loose coupling. Go through my Microservices playlist to understand it in depth
thanks for the quick reply. Your video was really good and informative. I understand the loose coupling concept. But I am just curious to know how async communication makes things loosely coupled.
I feel this info is widely known by software engineers. Can you please make a video on blocking vs non-blocking call ? I think many people confuse it with sync/async.
@Arpit Bhayani For Asynchronous communication if multiple services are dependent on one microservice is it better to use multiple message queues for each microservice or use pub/sub mechanism for communication
Aript , let's suppose if any service consume the message with in time even after retry , and we pushed those message in back in queue , can we use cron job so that if let's suppose those all transaction which status is not chanage with in 1 hr they can again process ?
Reaction service will be under load balancer. What was mentioned is, if we use aysnc communication, we do not need load balancer for notification service.
Arpit, can we have a video on components generally used in system design with pros & cons of each component so that we know when to use Kafka and when to use RabbitMQ
You are blending two very different concepts here. Unfortunately both are named Asynchronous. Async processing mentioned in the video is about delegating actual task processing. In case of async http, it is about not letting your process get blocked on getting the response.
We return response to user only after getting acknowledgment from the queue, how to handle if there is some issue with the message broker and it's crashing, in this case should we return 200 to user or 500? Can't we avoid waiting for an acknowledgement from queue before returning response to user, and manage error in queue(while putting message) by monitoring logs, since the reaction is already registered in db?
Can we use redis cache in reaction service to store messages in case notification service is down? This will avoid the need to have queue in between. Could you talk about the pros and cons of this approach?
dude , redis cahce is no use here , if notif service is down , then the message will be there in queue until it is consumed and processed successfully . if there is issue in processing in notif , then as per design if it will retry maybe 3 times , still no success , then the message would be in queue , and not evicted. This depends on which message broker , then developer/infra will have to fix the defect in the notification service if any causing it or due to infra issues.
Thanks for the amazing content as always, just one query regarding async communication: Do we always need a queue? Event loop in js and reactive programming in spring, do they also fall under the category of async communication ?
Yes. We always need a queue for async communication between services. Event loop is async intra-process although similar concept but different usecases.
sir, i had a doubt... in real life how this is implemented ? multiple processes consuming from a single queue / message broker ? suppose only one service is consuming from the queue, then after consuming, the queue can probably discard that... in case of multiple services, what pattern or technique is used ?
![Summer Walker - Heart Of A Woman [Official Lyric Video]](http://i.ytimg.com/vi/SFddctbZzw8/mqdefault.jpg)
Beautifully Explained
Your content is a goldmine for backend engineers. Keep posting more detailed videos. Thanks
Hi Arpit , Amazing content and explanation :).
I have a query regarding one of your synchronous calls example - Payment services.
I think i have seen asynchronous calls being used for Payments now a days (Example - Amazon) since PGs have higher latencies so its better to process the payment/order initially for user experience and then use a webhook for updated status later. Thats why Amazon gives a message after order completion incase payment failed from PG and we need to re-do the payment.
Yes. The payment processing can also be done asynchronously. Great observation.
Best explanation, thanks Arpit
Bhai what an explanation kudos 👏
very well explained....thank you
fantastic explanations
How I did not come to know about your channel before :) . Nice bro.
Aprit, I am curious to know how Async communication can make the loose coupling between the services. Because in that scenario, if anything changes, you need to inform the caller, Whether it is schema changes, API changes, or event definition changes.
But so long as communication interface does not change each service can independently be developed. This is loose coupling.
Go through my Microservices playlist to understand it in depth
thanks for the quick reply. Your video was really good and informative. I understand the loose coupling concept. But I am just curious to know how async communication makes things loosely coupled.
What do you mean by , because of network partitioning the service is not available?
correction at 18:32 synchronous instead of asynchronous.
I feel this info is widely known by software engineers. Can you please make a video on blocking vs non-blocking call ? I think many people confuse it with sync/async.
@Arpit Bhayani For Asynchronous communication if multiple services are dependent on one microservice is it better to use multiple message queues for each microservice or use pub/sub mechanism for communication
Pubsub/message stream would be a better choice.
@@AsliEngineering pubsub or different topic consume should also works cmiiw here.
Aript , let's suppose if any service consume the message with in time even after retry , and we pushed those message in back in queue , can we use cron job so that if let's suppose those all transaction which status is not chanage with in 1 hr they can again process ?
Amazing video but one doubt ..if we have multiple reaction-service nodes then how the request forwarding would happen without a load-balancer?
Reaction service will be under load balancer. What was mentioned is, if we use aysnc communication, we do not need load balancer for notification service.
Arpit, can we have a video on components generally used in system design with pros & cons of each component so that we know when to use Kafka and when to use RabbitMQ
Something I cover in my course. Would not be possible to cover it on YT. Sorry. But you can definitely find a ton of resources on this topic.
how is the communication over HTTP between two services Synchronous if we make a call and Await the response using Promises?
You are blending two very different concepts here. Unfortunately both are named Asynchronous.
Async processing mentioned in the video is about delegating actual task processing.
In case of async http, it is about not letting your process get blocked on getting the response.
We return response to user only after getting acknowledgment from the queue, how to handle if there is some issue with the message broker and it's crashing, in this case should we return 200 to user or 500? Can't we avoid waiting for an acknowledgement from queue before returning response to user, and manage error in queue(while putting message) by monitoring logs, since the reaction is already registered in db?
No you should not. If you are unable to persist in queue then return an error to user
Hi arpit iam a py dev, for backend which is the best framework for learning lld Or hld
Can we use redis cache in reaction service to store messages in case notification service is down? This will avoid the need to have queue in between. Could you talk about the pros and cons of this approach?
but then the implementation logic for retrying falls under each service. everyone will needs to implement it and make sure it will definitely retry.
dude , redis cahce is no use here , if notif service is down , then the message will be there in queue until it is consumed and processed successfully . if there is issue in processing in notif , then as per design if it will retry maybe 3 times , still no success , then the message would be in queue , and not evicted. This depends on which message broker , then developer/infra will have to fix the defect in the notification service if any causing it or due to infra issues.
Thanks for the amazing content as always, just one query regarding async communication: Do we always need a queue? Event loop in js and reactive programming in spring, do they also fall under the category of async communication ?
Yes. We always need a queue for async communication between services. Event loop is async intra-process although similar concept but different usecases.
@@AsliEngineering Got it, thanks !
sir, i had a doubt... in real life how this is implemented ? multiple processes consuming from a single queue / message broker ?
suppose only one service is consuming from the queue, then after consuming, the queue can probably discard that...
in case of multiple services, what pattern or technique is used ?