Django Celery Advanced Concepts | How does Celery Work? | Pool, Concurrency, Autoscale |Scaling Apps
HTML-код
- Опубликовано: 5 фев 2025
- 🎥 In this VIDEO of this series, I have gone a little advanced to take celery to the next level so as to use celery effectively in our projects. I am guaranteeing you that after this video, you will know everything about celery and this will definitely help you to be superior to others and earn a high income.
What we will cover?
✅ How does celery work? Complete Explanation with deep analysis
✅ Why use celery, Django celery beat?
✅ Understanding Celery Worker, Worker Processes
✅ Exploring celery features like Pool, Concurrency, Autoscale
✅ Different celery pools - prefork, threads, gevent, eventlet, solo
✅ Analysing when to use threads, process or green threads(Gevent, Eventlet)
✅ Best Practices
Don't miss the ending part, those who were facing difficulty in understanding because of the high level of the videos - 59:15
🤩 After this video, you will have a better understanding of how processes, OS, CPU work and how to use them in different real-time situations.
🚀 This series is specifically created to cover everything about celery. Celery is very important and you must know how to use it effectively to create robust applications in order to get a high-income job or become a successful freelancer.
👉 This series is not just going to be another simple Django celery integration Tutorial. In this series, we will try to understand each and every concept from the depth and apply it practically.
How this series will help you 🎉-
1️⃣ Using Celery effectively to build robust applications
2️⃣ How to build Logic while development
3️⃣ How to scale large applications
4️⃣ Different use cases of celery
5️⃣ In-depth knowledge of celery features and how celery actually works
🚀 Check out this article on Celery: www.priyanshuo...
👉 Ask your queries in the comment section box
Links -
💻 Discord: / discord
📸 Instagram: / priyanshu._gupta
✈ Telegram: t.me/priyanshu...
If you like the video do give it a thumbs up 👍 and share it with your friends and share your valuable feedback in the comment section
👉 Do subscribe to my channel "Priyanshu Gupta" for more skill development and point to point videos
🤷♂️ Have more queries?
Ping me on Instagram - / priyanshu._gupta
🤑 The Ultimate Revolutionary Course - • The Revolutionary Course
Link to download Process Explorer - docs.microsoft...
✨Keywords
Celery
Django Celery
Django Celery Beat
Django Celery Results
Django Celery Periodic Tasks
Redis
Rabbitmq
Pool
Autoscale
Concurrency
Worker
Worker Process
Child Process
Prefork
Solo
Gevent
Eventlet
Green Threads
Multithreading
Multiprocessing
django celery,django celery beat,celery with Django, celery django example, celery django production, integrate celery with django, celery django redis, python django celery, python django celery redis, django celery redis, celery pool eventlet, celery pool thread, celery gevent vs eventlet, celery gevent vs prefork, celery gevent example, celery thread pool, celery concurrency, celery autoscale vs concurrency, scaling celery workers,celery advanced tutorial, pool celery
#celery #django
Man, What an underrated channel
Thanks bro 🤝
I’m really grateful to you. As an experienced backend developer, I’ve also been in situations where I struggled to understand these topics. But when I started watching your videos, they made a huge impact on my life. Your practical knowledge and explanations are outstanding.
Thank you so much, and all the best, brother! 🤝
Content reaching to right audience!! Thank you so much. I hope can be more helpful.
Always maintain quality work and solve production and top-level problems. thanks dude
🔴 Don't miss the ending part of this video, I have an important message for you - 59:15
✅ Do watch this video after you have watched this if you want to know when to use celery and when to use multithreading, multiprocessing etc. - ruclips.net/video/y_FkspGs6ro/видео.html
👉 In case anyone wants to connect with me 1:1, feel free to schedule it on my calendar - topmate.io/priyanshugupta
🗒Article: www.priyanshuofcl.com/blogs/celery
Don't forget to like 👍 the video and subscribe 🔴 to my channel if you really want to level up your skills to have an edge over others.
Where were you, this channel is so underrated.
oh men thanks for the tutorial, i am not english spekaer but i undertand all of this tutorial! thank you for sharing
great videos on celery, I was afraid first to implement it in my project since it's a big change but now am comfortable.
keep going.
Thats great 😊
Great tutorial. You really are a good teacher: you understand that people need to really understand how tools work to use it confidently and genuinely. I am starting a job (new role) that requires to master celery quickly and your videos make the difference. The documentation is confusing and the majority of tutorials are only practical "getting started" stuff. On the other-side conferences lecturers talk to people who already understand them.
I would be interested interested in more details that are covered in the user guide.
Yes, you got me right. Thanks by the way.
All the stuff is very good and the way you explained is brilliant 👍 thank you . When I searched for the celery, got your videos and I strated working on it . Its going very smoothly 😊 thank you for sharing your knowledge 👏
@@PriyanshuGuptaOfficial Hi, how could i use greenlet.settrace inside celery task so that i could know whenever control switches between greenlets?
This tutorial has broadened my perspective. Thank you for taking the time to explain.
Glad it was helpful!
Thanks for this series, I learned a lot from it. And this is the most detailed explain about Celery that I have ever seen.
My pleasure 😇
No one in RUclips has explained Celery in depth till now.
Awesome content
this a great tutorial, first time landing on your channel and i can say your explanation on this concept is the best so far to what i have come across. i keep learning from your channel
Hope you learn more from the channel and see something new
@@PriyanshuGuptaOfficial how do i use greenlet.settrace from within the celery task to monitoe whenever greenlets yield control to other greenlets . I need to log this event for a given celery worker of gevent pool with concurreny 3
thanks man, nice refresher for deeper concepts and not just a basic hello world setup
Glad you like it. Thanks for commenting.
best video
You are awesome for making this series of videos! Thank you!
what a mind blowing video bro? keep uploading this types of content.
Sure bro
brother , your doing super duper job please continue the good work
Thank you. Continuing!!
Thanks Broda you actually did a right thing making this playlist i was fed up searching for the right resources about celery this tutorial covered nicely and one more suggestion this being totally theorotical u can push this video to part 1 and keep the rest of the videos after this coz most of the other tutorials have covered the other videos but this particular video is the highlight of this playlist anyways Kudos to you :)
Thanks a million to brother, I really appreciate the value of knowledge you share and the nonstop hardwork you did to create these videos for us. Thank u so much.
Thank you for your positive feedback! I'm glad you liked my work.
My pleasure😊@@PriyanshuGuptaOfficial
This is amazing. I just landed on your channel and I am glad I did.
All the stuff is very good And the way you Explained is also brillant 👏When I searched for the celery and I got your series And started working on it.Its going very smoothly 😊thank you for sharing your knowledge 👍
Thats great 😊. So happy to read this
Nice video 👍
Great sir i got it ur concept priyanshu sir
Excelent🙂
Thanks for the explaining !! Very good video, now I understand a bit more Celery!
Thank you 😊
@@PriyanshuGuptaOfficial come back bro where u gone ???
Thanks a lot brother ❤
Great tutorial for celery. Detail explanation, seamless throughout the tutorial. Keep up the good work.
Thanks bro
great tutorial as always😀
Great work
brilliant ❤️
Very useful
Detailed explanation
Great work
Keep posting
Excellent presentation!!! Thank you!!!
Best Video
Awesome!
I did complete your 700 Likes😇😇
Now i can keep celery in my resume
Yes definitely 💯
Great tutorial❤❤
Wow amazing content man instantly subscribed
Very well explained. Everthing at one place for django background tasking.
Thank You Sir
Superb video. Thankyou so much
good video brother
Thanks brother
This is a great playlist !!
Thanks for the video
It helped me a lot
Welcome ✌️
Thanks for the motivation brother :)
Great Tutorial!
Greetings from Brazil! Thanks for sharing such knowledge!
Welcome 🤞
Keep making such content, very much useful for making highly scalable apps. Not only celery, this concept itself is much needed irrespective of framework
Yes indeed!
Thank you because you are shearing this knowledge. Greetings from Serbia.
My pleasure!
I really liked your explanation bro excellent, and you really nailed it man.
Thank you so much
Great Content Bhai! Thank you, keep making such content.
Wow Amazed !! thanks for the help ! really helps a lot
Crystal clear explanation
Thanks buddy!!
awesome lecture...
thanks a lot bro...upload more videos like this
Definitely 💯
Great, thanks for the video.
Why this video have less like... This is very amazing and helpful video... 🙏 Thanks buddy
My pleasure 😇
Superb work🙌🙌
Thank you
great explanation
Thanks a tonnn brother. Im stunned by the clarity of your knowledge.
Thanks
Subbed, excellent content.
Thanks bro
Can you please make me understand how can I use compute of ec2 for celery ? how scaling will happen over there ?
So what you can do is first decide the pool of celery according to your tasks(cpu intensive/io operations). Try to increase the workers and use the full capacity of your ec2. You can also make dedicated queues for different task types and respectively make the separate celery workers(listening to dedicated queue). This will reduce the load on one queue. Then you can deploy rabbitmq on a separate server. Then you can deploy the celery workers on a new ec2 machine in order to scale horizontally. In order to manage services over different machines easily, you can use ec2 auto scaling feature, or you can use docker swarm/kubernetes.
@@PriyanshuGuptaOfficial Thank you so much for quick reply. Much appreciated !!!
Thanks a lot for your hard work.
Thank u so much!
Awesome Explanation Bro :)
Amazing video!
Great tutorial 👍
It's Amazing!
thanks.
Good course
Thanks Priyanka ji..you are too nice and smart
you are the best
Thanks a lot bro!
Love You Brother -------
Hellow -----
Request to you that please make a video how to use django custom admin panel instead django default admin templare....
Wait For Your video Notificaiton
Hellow -----
My self Abdullah from bangladesh .
I Watch your video regularly and now i complete django playlist.
i learn everything from your youtube ---
thank you ---
I have a request to you that please make a video how to use django custom admin panel instead django default admin templare....
please
please
please --------- i wait for your notification
very informative! thank you...
My pleasure
Priyanshu! Great video. I'd like to translate these series to spanish but I don't find the option of adding transcript. Let me know if you can add it!
Great video, Thanks for sharing.
You can also add managing queue, adding ofair and also one ques is it best to add asyncio inside celery task?
Thanks. Use asyncio inside your celery task if that single task has to peform multiple I/O operations.
i find your channel is like i find diamond 💎 😀
So happy to hear that 😊. Hope you get the valuable content.
Can you make video on flask redis and celery to cache any task query from database
perfect !
Thanks for the video, if possible could you show how to use celery with genericApi views and class based views?🙏
If you want to call celery task from view, its very easy. Just call task_name.delay()
thank you.
explanation focus on concept not on writing code.
good explanation.
if you split this video in several small videos as playlist about celery it will be good.
I suggess to comment PAAS like Heroku and serveless like AWS support for celery in deployed django project
Please make a video regarding Eventlet & gevent please
brother windows par concurrency and autoscale karne par celery task receive karta hai but execute nahi karta. i want to autosclae my celery workers according to processes load on windows. should i create multiple pool=solo but it will run fixed amount workers even when there is not need. so what should i do tasks are processing type
This was very helpful. Can you please also upload it on green threads?
Can we pause and resume celery task?
Yes, checkout 4th part of this series
thankyou for your video
i have some problem with celery
my work station has 64 core (128 threads)
251GB RAM
i want to parallel run my process
running my single one task require around 30 minutes and internally one task use internally 10 processor (threads) and around 10 GB RAM required
i hae used concurrency , autoscale also
when i run manually 20 ~ 25 tasks are optimal at a time
if i set concurrency 20 then the celery is waiting all 20 tasks to complete
not jumping to 21 if first 1 , 2 task completed
do you have any idea ?
can you write me a celery options so i can try ?
thank you
from S.Korea
Hi
Priyanshu Gupta
46:27 - regard the gevent and eventet can you plese create video ? also about stateful tasks
thanks
can you share your ppt of celery ?
This was very helpful.Can you please also upload on green threads?
Will try my best
bro how much time it will take for sending 10,000 to clients using celery concept.. i mean how many minutes it will take?
It should take around 0.5 seconds to send each mail. So you can estimate accordingly. Plus you can optimize it further by sending in batches and using pool of workers(here thread or gevent makes more sense as its an io operation)
@@PriyanshuGuptaOfficial thank ui
@@PriyanshuGuptaOfficial now celery threading will be support in windows machine or it will support only solo?
@@PriyanshuGuptaOfficial bro pool of workers (thread) this thread will support on windows 11?
Sir ek video django testing using faker in depth
One video on django ORM deep knowledge
Then start a series in API with project or project kisi ka clone ya phir esa kuch jisme doubt clear ho jaye ki api mai validation ko kese lagde jese python mai lgate the
Hi Bro can you provide some hands on coding for celery
If possible please make a video with code part explanation
See other parts of this series
@@PriyanshuGuptaOfficial ohk thanks
how can i deploy my celery web application for free ? currently im using pythonanywhere
In my project celery not receive task and not performed task in background , please reply on my comment
This can happen due to various reasons. Please message on discord server or telegram with more details, so that I can understand your problem properly.
@@PriyanshuGuptaOfficial thanks for reply, celery is working now in my project
Please provide ppt, it will be very helpful
WHat is the need of solo? Like why do we even need this?
You can use it if you want to perform a single task dedicatedly without any context switching
it feels like, you have to work more on delivering.
really good content, but your voice tone makes it difficult to follow. You could try toning it down maybe.. just a suggestion :P
Thanks for the tip!