Good content. Thanks, Narendra. Here are some ideas for other viewers about handling failures. Say a worker reads a message from the queue. The worker starts processing the message, but then the worker crashes. The message is then lost, because it was removed from the queue. One solution is for the message queue to expect confirmation when a message is handled. If a worker takes a message, but doesn't tell the queue "the message is handled" within some time limit, then the message queue software puts the message back in the queue. In this case, it's important for the work to be idempotent or for workers to anticipate race conditions, since a worker might time out some task even if the worker does not crash. If a message is re-queued too many times, then the message is discarded or sent to a "poison queue". Other workers watch the poison queue and decide how to handle the failed tasks.
Generally you would not have just one queue for this kind of stuff.. You will have multiple queues. However, coming back to your question, queues are inherently fault tolerant, meaning they log everything and if they fail they create the last state. So if you happened to send a message to queue, and it crashes after accepting the message, in that case, when it is comes back online, it will recreate its last state from the logs and pass the message to the threads. Hope this helps
Awesome, I have few question. Here (ruclips.net/video/BFcNDPt6SlE/видео.html) you told that we need to pass video reference not the video and send the video in queue is not good practice. Will you clear me how we could send the video reference in the queue? If we save the video another place that also need some time to store the video and how we could optimize that also?
HI Naren, I need a small help with the system design question i'm trying to solve. It would be helpful if we can have a char or if you share your email. Looking forward. Thank you
Great video Narendra :)
The common pattern for async programming:
1. Callbacks
2. Placeholder style (future, promise and deferred)
3. Queue
4. Signals
Great Explanation !!
this is really good!
Great content as always.
Thanks Varun :)
this is one of your best videos.
AWESOME
Beautifully explained Naren!
Thanks a lot.
This is exactly what I needed, very good video :D
Nice explanation. Thank you so much.
Awesome Video of Async processing :)
thanks a lot
Awesome video sir :) Thanks a lott
Thanks
Narendra! Great Content. Thanks
Very nice 👍
Great content again..wish you a happy new year dear Naren
Thanks for getting a mic 🎙 audio is better
Good one. The usage of real-life examples like the takeaway one was nice.
Good explanation.. thanks
Great Video Narendra!!! I got lots of help from the tutorial.
nice comment
Good content. Thanks, Narendra.
Here are some ideas for other viewers about handling failures. Say a worker reads a message from the queue. The worker starts processing the message, but then the worker crashes. The message is then lost, because it was removed from the queue. One solution is for the message queue to expect confirmation when a message is handled. If a worker takes a message, but doesn't tell the queue "the message is handled" within some time limit, then the message queue software puts the message back in the queue. In this case, it's important for the work to be idempotent or for workers to anticipate race conditions, since a worker might time out some task even if the worker does not crash. If a message is re-queued too many times, then the message is discarded or sent to a "poison queue". Other workers watch the poison queue and decide how to handle the failed tasks.
Nice explanation.....keep it up bro
Great video bro👍
Better sound...(Y) Great Video..!!
I am fan :)
Hey man, I have interviews coming up and these videos are incredibly helpful. Thanks a ton! Will keep a lookout for your content.
Nice work Narendra👌
From where do you get the system designs?
Nice explaination sir
can you use kafka and rabbitmq as a message queue?
What a detailed explanation.Could u please suggest how did you learnt all these? Please suggest some study materials.Thanks
Thank You for the explanation. What happens when the messaging queue crashes?
Generally you would not have just one queue for this kind of stuff.. You will have multiple queues. However, coming back to your question, queues are inherently fault tolerant, meaning they log everything and if they fail they create the last state. So if you happened to send a message to queue, and it crashes after accepting the message, in that case, when it is comes back online, it will recreate its last state from the logs and pass the message to the threads. Hope this helps
@@20frieza Thank You..That was very well explained.. :)
Great! How would we handle the downtime of queue(RabbitMQ, Kafka etc)?
kafka is durable, we can configure rabbitmq to be the same. So whenever it's back up it'll republish the messages
in scalable Point of View can i say this works like a pipeline manner??
Can you please make a video on fb system design?
Great video but web app looked like more of parallel processing rather than asynchronous processing.
we are not giving response to the client after video processing but immediately so that client is not waiting for app server
Awesome, I have few question. Here (ruclips.net/video/BFcNDPt6SlE/видео.html) you told that we need to pass video reference not the video and send the video in queue is not good practice. Will you clear me how we could send the video reference in the queue? If we save the video another place that also need some time to store the video and how we could optimize that also?
the client needs to upload the video to some data storage like S3
Jesus Christ... How many things do u know! Narendra :D Thank you
Ha ha, I learn and present. I don't know all
@@TechDummiesNarendraL r u interested in doing some algorithm videos? (add more ads plz, that's the least we can do :))
HI Naren,
I need a small help with the system design question i'm trying to solve. It would be helpful if we can have a char or if you share your email. Looking forward. Thank you
Tyala mahit
AWESOME