Couple of very high level takeaways I could get from this podcast: 1. Keep all features behind a feature flag.This helps when you need to turn something off that's breaking production without doing a code release. 2. Wherever possible have a stand by setup like a static api that can keep your most business critical services up to ensure no customer/business impact in case a disaster strikes your system. 3. Run time auto scaling of DB capacity doesn't necessarily work for use cases like live streaming. You need to scale up before traffic surges. 4. Careful planning and back of the envelop calculation for all system components that are likely to get impacted by traffic surges. Like DBs, Kafka, CDN etc.
@@harshitsaini7879 It is for this reason I say NFR are no more Non Functional, they are Functional Requirments from Day One for SaaS and Cloud Solutions.
The way Prachi broke down and explained the intricacies of scaling and load handling for high-traffic applications like JIO Cinema is truly commendable.
Rule 1. If it works, leave it Rule 2. Preplan things Rule 3. Always assume that things will fail 100% and have backup ready Rule 4. Feature flags are superb In short, system design requires presence of mind and future planning (generally taking peak cases)
Feature flag (keeping in config service, based on geography), Charles testing tool, Exponential backoff for API retries, Request flow ( client -> Multi CDN -> LB -> Origin server -> DB), DB autoscaling is not recommended (takes lot of time 45 mins from her example for scaling 5 instances) (prescale is recommended after some calculations), Increasing TTL of caches when there's a hockey spike (can or cannot help), API personalization (serving static response when having this hockey spikes), Always Prepare for plan B,C,D, Multi CDN Optimizer (decides where to send the traffic, used for live streamings), Cache Offload < 90%, Designing cache policy in CDN is important (offloading cache monitoring), Kafka, DNS issue mitigation at backend,
Key notes: Feature flags, Circuit breaker pattern, APM, Kubernetes, Client Caching, Multi cdn optimiser, Cache offload, Kafka distributed system, Ads timing is manual, Ad personalisation is on the server side.
Not sure how many non tech people will be watching it, but If you're watching this, take a moment to appreciate the people who are disrupting their own schedules and family time to ensure you enjoy a seamless IPL match. Their dedication makes this experience possible and tech a good place to be in.
During the whole podcast I had this question ringing how jio cinema use ads and at the end you discussed it, I will be exploring the whole architecture again and in a much more detailed fashion. It just rose my curiosity to another level.
Might be a possibility that jio decided to have at least one female director, so hired her on low salary. Just an assumption. Beacuse there are many ofher IITians out there why they hired from normal college?
Can we take a moment to appreciate the engineers involved in this, they stretch their time , compromise connections with their families for us to have a seamless experience, it’s a long work for 2 months and hope they deserve necessary appreciation and bonuses for their time. Yes , tech is just one aspect, but I hope asking questions like how the team is able to manage the work load is also very important.
The ad insertion part was quite interesting for me, I did not realize that even the ads that we see during the games could be personalized and inserted based on different strategies. SCTY-35 markers was something completely new to me. The guest really had an end to end grasp of the whole solutions and was able to present a very coherent picture. Loved the podcast :)
Hey @arpit, can you get someone from IRCTC technical team who can get us understanding on Tatkal mechanism. I believe what Amazon, hotstar, jiocinema do in their seasons(be it ipl, billion day sale etc) irctc handles far more requests on daily basis during Tatkal.
Excellent content! As a backend engineer, this is exactly the kind of content I love to watch. I hope you continue to bring such insightful guests on the show.
Great insights, mostly when you see these kind of things people throw around standard fancy terminologies, but here its just from pure experience, deciding on base64, multiplication factors, one small change * the number of API requests etc... Enjoyed watching this.
I recall Hotstar tech video about the realtime scaling during IPL and they were discussing same strategy specially for moment when there is "DHONI" in the chapter 😃😃😃
Thanks Arpit and Prachi. Interesting questions covering end to end flow and very well put in-depth answers. Looking forward to the next episode! One question on personalized Ads insertion: * Why can't we pre-determine personalized Ads for a user in advance and play them during the break? I think bucketing a user based on certain user parameters is a good work around for now. pros: * More personalization (More revenue) * Not much time is spent on choosing an Ad: Since we would already have a list of Ads to pick from, there will not be much time waste to select an Ad during break. cons: * Waste of compute/memory resources to personalize Ad for a user who never watches the stream till they get to Ad break. Some more follow up questions: * How we can know more about user? This brings more number of Advertisers on the platform. * Can we add Ad bidding in this high concurrency streaming situation. Following is another interesting question for me to dive deep: * How can we show a particular Ad to exactly 'X' number of people? i.e. No over delivering or under delivering.
O my god 😂 I also have studied in the same class in the same batch at the same college where she pursued her bachelor's in engineering. Look at me, where do I stand today and look at her , oops , I am really out of words to appreciate her for this height of success she has reached. Wow so cool happy for being her classmate once upon a time. At the same time I really feel ashamed.
@@anuragsengupta2880 Respected Sir, Please can you provide your pincode so that I can send you "Burnol". That'll help you with your Jalan problem. Thanks 🤝
Man!! What an amazing podcast ❤ no useless talks, every point was clear though I’m a data engineer, not everything topics discussed we work on. Please do more such podcasts 🎉
00:01 JioCinema's Prachi Sharma explains the complex tech and prep behind live streaming IPL to 20 million devices 02:16 Pre-match and post-match processes for handling live streaming infra 07:33 Performing comprehensive audits for system stability and performance. 09:53 Using feature flags and simulations to ensure smooth live streaming of IPL matches 14:36 JioCinema uses an in-house conf service for feature flags. 17:16 JioCinema uses a multi-CDN infrastructure for live streaming IPL to 20 million devices 21:38 Managing varying traffic loads and mitigating system failures 23:53 Use of panic mode and planning for unexpected surges 28:05 Benefits of using Multi CDN 30:15 Images overload caused by star feature 34:30 JioCinema plans and handles use cases for live streaming IPL to 20M devices 36:47 Debugging DNS failures for JioCinema 41:13 JioCinema uses different methods for ad insertion in live streaming. 43:54 JioCinema live streams IPL to 20 million concurrent devices
Gem of a podcast! Loved it! Just a small suggestion, please ping for more real time examples when the speaker is briefing about particular part of application or explaining any backup procedures, giving out more use cases just helps in connecting better, even for newbie like me!! Overall was a interesting talk!
Great podcast, never a dull day when you are streaming at Bharat scale ❤️ More power to you, Prachi 🚀 thanks Arpit for bringing such insightful content 👍🏻
Thanks Arpit Sir, for doing system design PODCASTS! with actual engineers. I mean this is such a good concept, I hadn't thought was possible, and you are making it happen. The guest was really good, got to learn a lot about system design just by listening to this podcast. Good quality content 👍 keep making for us.
This is a great podcast. I am a senior backend engineer in the UK and I found this to be very useful. Thanks for the content. PLease make more videos :)
What a session🎉, lot more details in a less time, we wish more these kind of sessions across all other company cloud operations, thanks Team, great job.
It is so beautiful that the streaming platform does have many bottlenecks and they have many ways to mitigate according to the failures they faced or they may face. Thanks for enlightenment Arpit❤
Loved the term "Brace for impact" made me smile, kudos to such highly available architecture. Way Prachi explained things in detail hats off!! Thanks @arpit for bringing this podcast!! Loved it♥
Hi all techies, I am stuck trying to figure out the maximum capacity of my system. Specifically, I am running Nginx on a system with: 4 CPUs 8 GB RAM 1 Gbps network I want to know how many requests it can handle. Additionally, I want to understand the capacity for the backend Node.js application and the database. Is there any formula or type of calculation to determine this?
It totally depends on what you are doing in every request, be it talking to database, or manipulating string, or encryption, or making network calls. Even within each of this, it depends on how time consuming each task/operation is. There is no magic formula that can help you determine the #req your machine can handle. So the best way to do this is by load testing your setup and then extrapolate.
Btw the match (yesterday) between RCB and CSK had 50 crore concurrent viewer. That's 500 million concurrent viewer. Some crazy engineering work at Jio Cinema! Kudos to them 🚀
@@rohitlakhotia3357 bhai hotstar vale concurrent dikhate the. Hotstar pe maine dekha tha maine about 1.5 crore highest. Jio vale dikhate hai ki kitne logo ne stream suru hone se leke abtak us match vale tab ko open kya.
Brilliant discussion. It would be worth having a follow up on the video production side of the infrastructure and how those live feeds are ingested by Jio and then streamed out to the clients. How are some of the design choices made to support the real time requirements
Great content @Arpit Bhayani ! Would be fantastic to also delve into the tech stack, including databases, caches, cloud services, and programming languages, and how these support their use cases. Excited to hear more on this!
There are certain things some businesses cannot talk about and these details are sensitive for them and they cannot talk about it. This is why we never delved into that topic.
Amazing content!! Can you make a video on how to calculate no of servers required with given rps and latency? How do hardware limitations change the calculation? How sync and async calls affect the numbers?
Couple of very high level takeaways I could get from this podcast:
1. Keep all features behind a feature flag.This helps when you need to turn something off that's breaking production without doing a code release.
2. Wherever possible have a stand by setup like a static api that can keep your most business critical services up to ensure no customer/business impact in case a disaster strikes your system.
3. Run time auto scaling of DB capacity doesn't necessarily work for use cases like live streaming. You need to scale up before traffic surges.
4. Careful planning and back of the envelop calculation for all system components that are likely to get impacted by traffic surges. Like DBs, Kafka, CDN etc.
Always have a plan b,c,d... in case something went wrong.
any auto scaling almost never works for any kind of sale type event.
@@harshitsaini7879 It is for this reason I say NFR are no more Non Functional, they are Functional Requirments from Day One for SaaS and Cloud Solutions.
not 20 million, it is 200 million
During the fire, disable resources for P2 & P1 for the p0 remain up
The only IPL thing I'm interested in
The real IPL
Haven't watched a single match this year 😅
Me 2
Same here, I guess they use lamda function with kubernetes
GOAT
The way Prachi broke down and explained the intricacies of scaling and load handling for high-traffic applications like JIO Cinema is truly commendable.
No she didn't.
Rule 1. If it works, leave it
Rule 2. Preplan things
Rule 3. Always assume that things will fail 100% and have backup ready
Rule 4. Feature flags are superb
In short, system design requires presence of mind and future planning (generally taking peak cases)
Feature flag (keeping in config service, based on geography),
Charles testing tool,
Exponential backoff for API retries,
Request flow ( client -> Multi CDN -> LB -> Origin server -> DB),
DB autoscaling is not recommended (takes lot of time 45 mins from her example for scaling 5 instances) (prescale is recommended after some calculations),
Increasing TTL of caches when there's a hockey spike (can or cannot help),
API personalization (serving static response when having this hockey spikes),
Always Prepare for plan B,C,D,
Multi CDN Optimizer (decides where to send the traffic, used for live streamings),
Cache Offload < 90%,
Designing cache policy in CDN is important (offloading cache monitoring),
Kafka,
DNS issue mitigation at backend,
thankss
Thanks mate
Key notes: Feature flags, Circuit breaker pattern, APM, Kubernetes, Client Caching, Multi cdn optimiser, Cache offload, Kafka distributed system, Ads timing is manual, Ad personalisation is on the server side.
The engineering behind the amazing stuff we get to experience in life is nothing less than magic! Thanks for making these podcasts happen Arpit
Not sure how many non tech people will be watching it, but If you're watching this, take a moment to appreciate the people who are disrupting their own schedules and family time to ensure you enjoy a seamless IPL match. Their dedication makes this experience possible and tech a good place to be in.
Thanks Arpit for bringing such content. I look for these podcasts with seniors engineers. Cheers 🎉
During the whole podcast I had this question ringing how jio cinema use ads and at the end you discussed it, I will be exploring the whole architecture again and in a much more detailed fashion. It just rose my curiosity to another level.
crazy how at very young age she is a director of engineering of company like jio/viacom
and she is from normal college
Might be a possibility that jio decided to have at least one female director, so hired her on low salary. Just an assumption. Beacuse there are many ofher IITians out there why they hired from normal college?
She had prior experience with such engineering design in hotstar. So here is the niche that takes priority over the age factor.
@@ancient.sanataniirrespective of that, she was superb. Incredible communication and she must be a very valued person ! 👏🏼👏🏼
Can we take a moment to appreciate the engineers involved in this, they stretch their time , compromise connections with their families for us to have a seamless experience, it’s a long work for 2 months and hope they deserve necessary appreciation and bonuses for their time. Yes , tech is just one aspect, but I hope asking questions like how the team is able to manage the work load is also very important.
The ad insertion part was quite interesting for me, I did not realize that even the ads that we see during the games could be personalized and inserted based on different strategies. SCTY-35 markers was something completely new to me. The guest really had an end to end grasp of the whole solutions and was able to present a very coherent picture. Loved the podcast :)
Hey @arpit, can you get someone from IRCTC technical team who can get us understanding on Tatkal mechanism. I believe what Amazon, hotstar, jiocinema do in their seasons(be it ipl, billion day sale etc) irctc handles far more requests on daily basis during Tatkal.
+1
😂😂😂 +999999
🤣🤣🤣🤣🤣
boss 5 crore or 10 crore people use the app or more.. during match, how much do you think IRCTC on Tatkal time?
+1
Excellent content! As a backend engineer, this is exactly the kind of content I love to watch. I hope you continue to bring such insightful guests on the show.
Watched this during my weekend run!
What an amazing conversation- the passion that you both have for engineering is so inspiring. 🙏🏻
Great insights, mostly when you see these kind of things people throw around standard fancy terminologies, but here its just from pure experience, deciding on base64, multiplication factors, one small change * the number of API requests etc...
Enjoyed watching this.
Great to see someone really know what they are talking about.
I recall Hotstar tech video about the realtime scaling during IPL and they were discussing same strategy specially for moment when there is "DHONI" in the chapter 😃😃😃
Thanks Arpit and Prachi.
Interesting questions covering end to end flow and very well put in-depth answers.
Looking forward to the next episode!
One question on personalized Ads insertion:
* Why can't we pre-determine personalized Ads for a user in advance and play them during the break? I think bucketing a user based on certain user parameters is a good work around for now.
pros:
* More personalization (More revenue)
* Not much time is spent on choosing an Ad: Since we would already have a list of Ads to pick from, there will not be much time waste to select an Ad during break.
cons:
* Waste of compute/memory resources to personalize Ad for a user who never watches the stream till they get to Ad break.
Some more follow up questions:
* How we can know more about user? This brings more number of Advertisers on the platform.
* Can we add Ad bidding in this high concurrency streaming situation.
Following is another interesting question for me to dive deep:
* How can we show a particular Ad to exactly 'X' number of people? i.e. No over delivering or under delivering.
Feels like review version of OG old Hotstar scaling to 25 million talk by Gaurav Kamboj
Thanks for bringing this for us Arpit!
Prachi is an ex-Hotstar engineer 😅
Ya so many of these things seemed similar to that hotstar video, its like a lot of the same legacy but under a different company
@@RwikKumarDutta Most of the devs in JioCinema are from Hotstar...
Absolutely exactly the same.
O my god 😂 I also have studied in the same class in the same batch at the same college where she pursued her bachelor's in engineering. Look at me, where do I stand today and look at her , oops , I am really out of words to appreciate her for this height of success she has reached. Wow so cool happy for being her classmate once upon a time. At the same time I really feel ashamed.
After watching this podcast, I am fan of Prachi Sharma. She explained so well. Superb.
"ISP" : That guess by Arpit at 38:18 gave me goosebumps 🫡
Thank you for this one!..please bring gaurav kamboj as well
Awesome, having worked in Jio as a backend engineer, I really enjoyed watching this!
Hey what was the questions asked for fresher role ?
Wonderful video Arpit. Prachi was amazing in her explanations. Great asset to JioCinema.
Brilliant discussion - "Dhoni ke side effects on Content streaming architecture" would have been perfect title. Thanks for bringing this.
Don’t make everything about that middle aged credit hungry manipulator.
@@anuragsengupta2880 Respected Sir, Please can you provide your pincode so that I can send you "Burnol". That'll help you with your Jalan problem.
Thanks 🤝
Man!! What an amazing podcast ❤ no useless talks, every point was clear though I’m a data engineer, not everything topics discussed we work on. Please do more such podcasts 🎉
Really valuable for those who do not know how much work it takes to run the streaming platform.🔥
Kudos to you, Arpith, seriously no fluff, pure engineering ❤
Real time Problems and Solutions they come up with... These are really useful podcasts. Thanks to her.
00:01 JioCinema's Prachi Sharma explains the complex tech and prep behind live streaming IPL to 20 million devices
02:16 Pre-match and post-match processes for handling live streaming infra
07:33 Performing comprehensive audits for system stability and performance.
09:53 Using feature flags and simulations to ensure smooth live streaming of IPL matches
14:36 JioCinema uses an in-house conf service for feature flags.
17:16 JioCinema uses a multi-CDN infrastructure for live streaming IPL to 20 million devices
21:38 Managing varying traffic loads and mitigating system failures
23:53 Use of panic mode and planning for unexpected surges
28:05 Benefits of using Multi CDN
30:15 Images overload caused by star feature
34:30 JioCinema plans and handles use cases for live streaming IPL to 20M devices
36:47 Debugging DNS failures for JioCinema
41:13 JioCinema uses different methods for ad insertion in live streaming.
43:54 JioCinema live streams IPL to 20 million concurrent devices
It's really wow, the way Prachi explained things, i am curious to join Jio Cinema now 🤣, anyway the have done a fantastic job handling IPL Traffic 👏
You've won a subscriber!
Thanks Arpit for bringing in a knowlegable person and asking the right sort of questions!
Thank you Prachi! You have a very sound knowledge and a great way of explaining all of it in simple words.
Lot of useful interesting information in this clip. Prachi mentioned a lot of minute technical details.
I Like the concept of hockey spike when dhoni is on field
Intresting 21:46
Gem of a podcast! Loved it! Just a small suggestion, please ping for more real time examples when the speaker is briefing about particular part of application or explaining any backup procedures, giving out more use cases just helps in connecting better, even for newbie like me!! Overall was a interesting talk!
Thank you to both of you. Lots of health and happiness to you and your family.
It was just awesome.
This reminds me of my days in Myntra during EORS and BFF I know scale is very less but such events requires lot of planning and enthusiasm
by far the best one with so much problem statements and explained in a nice and proficient manner
If anyone says Chatgpt can take software engineer's job. Just send them this video!
Great podcast, never a dull day when you are streaming at Bharat scale ❤️ More power to you, Prachi 🚀 thanks Arpit for bringing such insightful content 👍🏻
Very nice interview. Great insight into what it takes to stream such popular events.
Amazing podcast....the engineering behind all this is mind blowing
Excellent discussion. Very apt questions and deep routed knowledge answers.
This is an amazing system design I watched full length.
Amazing
Loved this discussion. Very Helpful ❤
Amazing Podcast, Such Brilliant minds working in harmony no less than magic!!
Thanks Arpit Sir, for doing system design PODCASTS! with actual engineers. I mean this is such a good concept, I hadn't thought was possible, and you are making it happen.
The guest was really good, got to learn a lot about system design just by listening to this podcast.
Good quality content 👍 keep making for us.
glad to hear that chaos engineering is being used by large infrastructures to ensure resiliency of the systems
This is a great podcast. I am a senior backend engineer in the UK and I found this to be very useful. Thanks for the content. PLease make more videos :)
wow! In 45 minutes lot of things got to be understood, thanks for this session.
What a session🎉, lot more details in a less time, we wish more these kind of sessions across all other company cloud operations, thanks Team, great job.
really a great conversation between u guys ,learned a lot and still there are many things i dont know,but sounded interesting to me.
It is so beautiful that the streaming platform does have many bottlenecks and they have many ways to mitigate according to the failures they faced or they may face.
Thanks for enlightenment Arpit❤
Loved the term "Brace for impact" made me smile, kudos to such highly available architecture. Way Prachi explained things in detail hats off!! Thanks @arpit for bringing this podcast!! Loved it♥
Exponential Retries, Really good point
Amazing session Arpit, it was a dream to know such system from a person who knows so ins and out of it. Thanks for being curious Engineer 😊
Thanks Arpit , this is gold tier!! Pls do more such videos.
Also realized that one can learn system design faster by analyzing how big companies do it like RUclips, Jio cinema, discord, Instagram.
Hello Prachi, your cutting edge tech has improved a lot but App on Android TV is struggling.
Very great podcast...Please make a part 2 on the ADs part talked about at the end
Thanks for bringing up such raw and real engineering convo! More interesting than IPL
loved this podcast very much. The standards and tricks mentioned are very nice
Thanks Arpit . Such a informative video i found first time. Keep bringing this type of content
Amazing video Arpit
Always been a big fan of your videos
Very informative podcast. Thanks for bringing it up. Kudos to you guys
It's actually higher than 20 Million. I was watching that time it was 218 Million and in some match it reached 500 Million Concurrent Viewer 😮
KiHi, it was a nice conversation with perfect blend of tech terms. Thank you Arpit Bhayani and Prachi Singh.
"Will this survive Dhoni's spike?" - this is something every software engineer would love to discuss in an arch review! 😀
Hi all techies,
I am stuck trying to figure out the maximum capacity of my system. Specifically, I am running Nginx on a system with:
4 CPUs
8 GB RAM
1 Gbps network
I want to know how many requests it can handle. Additionally, I want to understand the capacity for the backend Node.js application and the database. Is there any formula or type of calculation to determine this?
It totally depends on what you are doing in every request, be it talking to database, or manipulating string, or encryption, or making network calls. Even within each of this, it depends on how time consuming each task/operation is.
There is no magic formula that can help you determine the #req your machine can handle. So the best way to do this is by load testing your setup and then extrapolate.
Thanks Arpit for bringing such content. looking forward for such podcasts with seniors engineers.
Thanks Prachi for letting us know the engineering behind the massive level of scaled system…
Really liked how Arpit instantly isolated the issue and asked if it with the ISP.
Prachi explained it really well, great questions
As a non cricket enthusiast i love about this video related to IPL.
Didn't know Dhoni would have such an impact on their infra 😂
Its so interesting to watch these amazing engineering working at such a high scale
Amazing podcast. It was great to get an idea of how things go on behind the scenes. Thank you Arpit for such insightful podcasts
Btw the match (yesterday) between RCB and CSK had 50 crore concurrent viewer.
That's 500 million concurrent viewer. Some crazy engineering work at Jio Cinema! Kudos to them 🚀
Are jio pe total viewers dikhate hai jinhone match ka thora sa bhi part dekha. Vo ek sath 5 crore nhi hoga
@@chetankumar9463 it say na ki X total viewers? Should be concurrent but does seem a little weird considering that's half India population
@@rohitlakhotia3357 bhai hotstar vale concurrent dikhate the. Hotstar pe maine dekha tha maine about 1.5 crore highest. Jio vale dikhate hai ki kitne logo ne stream suru hone se leke abtak us match vale tab ko open kya.
Muskil se dhoni vale match me 1 crore jata hoga.
Brilliant discussion. It would be worth having a follow up on the video production side of the infrastructure and how those live feeds are ingested by Jio and then streamed out to the clients. How are some of the design choices made to support the real time requirements
Great content @Arpit Bhayani ! Would be fantastic to also delve into the tech stack, including databases, caches, cloud services, and programming languages, and how these support their use cases. Excited to hear more on this!
There are certain things some businesses cannot talk about and these details are sensitive for them and they cannot talk about it.
This is why we never delved into that topic.
Why i wasn't aware of this channel before, amazing content!
Thanks for this amazing podcast , this is the best content and best insights
Loved this podcast. Please bring more like this.
Amazing content!! Can you make a video on how to calculate no of servers required with given rps and latency? How do hardware limitations change the calculation? How sync and async calls affect the numbers?
Commenting after the RCB vs CSK mach concluded, and I am curious to know the numbers behind this game now! Great job Jio Cinema!
Loved this! Some interesting stories were shared. Keep them coming Arpit☺️
Great podcast, Arpit and Prachi! Thank you so much.
Very insightful man. Thank you for sharing 🙏
Wow this was a masterpiece & it inspires me to work harder for the potential which lies in this field ❤
IPL sponsor should change from tata to aws😅
Lot of good insights from the podcast. Thanks Prachi & Arpit. :)
Amazing video Arpit! Thanks.
Jio Cinema has really improved this season compared to last year. Rarely any lag...
Thanks a lot Arpit for sharing this amazing podcast! Keep posting such great content for all of us please :)
She experience the exact scene in jio , on youtube there is also a video by hotstar engineer on ipl times , they faces same situation
she just replicating hotstar here, she worked at hotstar for 4 ipl out of 6
This is awesome discussion, thank you Arpit.
Hotstar did this way back when it was not cool.