I just love the indian yt scene for programming rn, both the youtubers (piyush, hrkirat) are trying one up each others project and we are profiting from both of em.
Here is my updated version of the production architecture: Step1: The Client will upload the video to a s3 bucket using a pre signed url Step2: Add trigger to the input s3 bucket for a Lambda function Step3: This Lambda function will run the ECS containers to process and upload the video to the output bucket.
Would like you to make part 2 of this , this was highly helpful but would like to see end to end ...getting stuck at a lot of things... Will also be helpful to know all the technologies you told in the assignment together
I thought about the same project few weeks ago but i had no i idea how would i deploy it My implementation was limited to localhost but after seeing your video, i got a good idea of the entire architecture. You're amazing bhaiya and 1000 times better than other didi bhaiya influencers
Initially, I thought the video.mp4 would be converted into a codec format like HLS on the fly and streamed in segments. Now I understand it better, thanks to your video
Not in codec but by default its using hls on the fly think so this was for understanding hls and how it works because hls is by default by all the browsers check it by using both index.m3u8 and video.mp4(here 3mb of video is loaded after some time another 3mb)
You can use rapid apis Judge0 for output of the code (not scalable) for scalability you can use docker spin up the container and install complier in the container which could run the code there and show the output to the user🤔
@@patelaryan0914 But how to handle live contest which can have 1 lakh+ user on a time and they will submit code. And also we have to show live ranking for that. This is a system design interview question which is asked in many companies.
Thankyou so much Piyush Bhaiya i was looking HLS video transcoding after Hitesh Sir dropped out his video on video streaming. Thankyou for such amazing video.
Please Please continue AWS series. 🙏🙏🙏 Ese kese banega aapka humara Bond. Naa hi aap video banate hai aur Naa hi reply dete hai kisi bhi comment ka. Bus apni marzi ke content banate hai...pehle comments to padh lijiye humko kisme dikkat aa rahi hai 😢
Very nice video. You can use ffmpeg tool available for macos which provide more features and speed for transcoding video and also there is also node package for it.
the docker container is downloading whole video before encoding it which wil take so much time instead better approch would be to setup a kafka stream which stream the video from s3 and setup a flink which will encode it and send it to respective stream.
Hey piyush again great video, also i would like to request you a comprehensive video on zookeeper, watchers, service registry, and its other features like watchers and stuff, it would be really helpful to build big scale backends
I have already made an lms system which has same functionality when you uploaded teachyst transcoding architecture almost 6 months ago, but still would like to see full implementation by you
Hi Piyush, Had a doubt. We are making the output bucket as public and then streaming it via videojs. What if the video is restricted to few people only. For example if it is a paid content we shouldn't allow everyone to just stream it. In that case should we update the CORS to only allow request from our origin.
Hey Piyush, You said there will be different index.m3u8 files for different formats like 360, 480,.. But after running the docker commands we ended up with only single index.m3u8 file. So, how different formats comes into the picture in index.m3u8 ??
I have completed the task which you have given. It has frontend, backend, sqs consumer and video processer which runs on ECS. I am not able to share the github link here.
As far as i know you said ffmpeg will run for 4 times so can you please explain the server where this docker image is running is that server is capable of running ffmpeg 4 time for a file which is of 1 gb , and if yes then what will be the configuration of machine which we have to opt for this HLS conversion process,please reply
Omg, Piyush! The container kept troughing 503 Service Unavailable and Repeated Startup and Shutdown because it needed more CPU core and memory for Transcoding, which was expensive and not permitted in my account. Can we use a VM instance or some particular service for video Transcoding right out of the box from the cloud? Please reply.
@piyushgargdev I'm using gcp cuz I have some credits in that , can you guide me through a lil if have time. Btw I'm thrilled 😁 you replied. Am actually trying to transcoded large video/MP4 format with multiple audio streams. Pick from one bucket, process and upload to a another from where the clients picks up. I would be flattered if you can shower some of your experience on me. 🙌🙌♥️
I just love the indian yt scene for programming rn, both the youtubers (piyush, hrkirat) are trying one up each others project and we are profiting from both of em.
Add Chai aur Code (Hitesh OG)
Harkirat is scammer
@@brutal4815 I agree. The guy is after your wallet. Thats his only interest.
Also codedamn
@@brutal4815 correct
Roz aise topic jiska mujhe door door tak koi Idea nahi
Thanks piyush bahi
The event trigger mechanism for new uploads, was really impressive 👌
Here is my updated version of the production architecture:
Step1: The Client will upload the video to a s3 bucket using a pre signed url
Step2: Add trigger to the input s3 bucket for a Lambda function
Step3: This Lambda function will run the ECS containers to process and upload the video to the output bucket.
Would like you to make part 2 of this , this was highly helpful but would like to see end to end ...getting stuck at a lot of things... Will also be helpful to know all the technologies you told in the assignment together
Harkirat and Piyush on their way to build to create their own universe 🔥
I have tried my self this . Learned a lot . But would love to see the end to end so that we can improve
I thought about the same project few weeks ago but i had no i idea how would i deploy it
My implementation was limited to localhost but after seeing your video, i got a good idea of the entire architecture.
You're amazing bhaiya and 1000 times better than other didi bhaiya influencers
Initially, I thought the video.mp4 would be converted into a codec format like HLS on the fly and streamed in segments. Now I understand it better, thanks to your video
Not in codec but by default its using hls on the fly think so this was for understanding hls and how it works because hls is by default by all the browsers check it by using both index.m3u8 and video.mp4(here 3mb of video is loaded after some time another 3mb)
Fantastic overview of Adaptive Bitrate Streaming! Keep up the great work!
love the way you explained HLS, need part 2 🙏
Please bring part 2 of it, Loved this one ❤
We definitely need such more advanced projects tutorials and ideas ❤ Thanks!
Thank you sir you have solved a lot of money for me by making this tutorial now I can create my own streaming platform
Making videos out of the box and explaining in simple lang.
Thanku
Plz. Sir part 2 chahiye implement krke jyada clarity ayegi plz.. tab jakke hum aaage kich khud bna skte hai ❤❤
Please make a video on online code compiler like leetcode with system design. And also add how to handle online contest on it.
You can use rapid apis Judge0 for output of the code (not scalable) for scalability you can use docker spin up the container and install complier in the container which could run the code there and show the output to the user🤔
@@patelaryan0914 But how to handle live contest which can have 1 lakh+ user on a time and they will submit code. And also we have to show live ranking for that.
This is a system design interview question which is asked in many companies.
@@vishalpanchal2343 my solution is not scalable if you want to try by your self than you can for that definitely we need a video from piyush..
Great level of information you are providing bro✨🥂
I am a thesis based Master Student, and working on the same topic with MPEG DASH adaptive streaming for CDN optimization
Brother very few Creator Make this type of contents. Give some time, If I couldn't do it, Definitely You will make it. I want Part 2
most unique channel for the programmers
Thankyou so much Piyush Bhaiya i was looking HLS video transcoding after Hitesh Sir dropped out his video on video streaming.
Thankyou for such amazing video.
Amazing Content. Why didn't I find you earlier? Thank you so much brother for this insightful knowledge.
I want part 2 please.
It would be great learning that how you approach the coding part for this also
Thanks yaar. You are a blessing to us. Happy to see creators like you
We need part 2 🔥🔥
this is so intrestesting , thank you so much for the quality content, not many people providing it 🍀
Please also talk about the costs of running these systems. A little brief will do as well.
Wow! ekdum useful and bhayankar app hai.
we want part 2 ⚡
Please Please continue AWS series. 🙏🙏🙏
Ese kese banega aapka humara Bond. Naa hi aap video banate hai aur Naa hi reply dete hai kisi bhi comment ka. Bus apni marzi ke content banate hai...pehle comments to padh lijiye humko kisme dikkat aa rahi hai 😢
AWS must continue...
Bring also part - 2 Piyush Bhai 💕💕
Sure. waiting for part 2.
I will definitely try to code a video transcoder 🙌
Piyush + Harkirat >>>>>>>>>>>> whole youtube
Absolutely ❤❤
💯💯
Chai with code as well
Bhai bura na manna harkirat bhai ki video mujhe km samjh aati h. Piyush bhai ki teaching style 🔥
Very nice video. You can use ffmpeg tool available for macos which provide more features and speed for transcoding video and also there is also node package for it.
What a coincidence, I just started researching HSL.
please give me 2 part. and please continue aws series.
the docker container is downloading whole video before encoding it which wil take so much time instead better approch would be to setup a kafka stream which stream the video from s3 and setup a flink which will encode it and send it to respective stream.
Thnx for this, I learnt a lot of things in this video
Make a part 2 for fully understand HLS Streaming
Hey piyush Try discussing what if we use lambda functions instead of docker containers
Love everytime i see your new project
thank a lot for the video today i got what hls streaming is
Very informative video Piyush sir, love from Pakistan 😊
amazing video, waiting for part II
Hey piyush again great video, also i would like to request you a comprehensive video on zookeeper, watchers, service registry, and its other features like watchers and stuff, it would be really helpful to build big scale backends
part 2 is needed
Wonderful Piyush Sir
sir,is it okey to use pre-signed urls instead of sqs??
What is need of queue system and how to use sqs in this case
Please reply...
Hi, thanks for the tutorial. I’ve a question. 31:04 why don’t js consumer directly listens for events in s3 bucket, why a queue is needed there?
Hi piyush I want second part of video transcoding from scratch also do include front end where user can upload video from front end
Absolutely loved it ❤, Let me complete the task 🏃
Please iska part 2 leke ao mere bhai❤🎉
I have already made an lms system which has same functionality when you uploaded teachyst transcoding architecture almost 6 months ago, but still would like to see full implementation by you
Really appreciate these videos. keep it up.
Aws series please continue❤ brother✨
Hi Piyush,
Had a doubt. We are making the output bucket as public and then streaming it via videojs. What if the video is restricted to few people only. For example if it is a paid content we shouldn't allow everyone to just stream it. In that case should we update the CORS to only allow request from our origin.
Loved the Video will try to do Myself and also want a Part 2 for this, Awesome Content !
Live streaming ke uper bhi kuch banaiye bhaiya
Hey Piyush,
You said there will be different index.m3u8 files for different formats like 360, 480,..
But after running the docker commands we ended up with only single index.m3u8 file.
So, how different formats comes into the picture in index.m3u8 ??
Part 2 🔥
Make part 2!!!❤❤
part 2 bhai must
maine ye chiz stripchat ke live streams ke workding ko smjne waqt sikha tha
bhai seriously stripchat ? kuch aur to nahi kar raha tha na🙄🙄
coming after Netflix crash issue on Tyson Vs Jake boxing match
Great video! I learned a lot about streaming, and I will try to implement this in Azure using different Azure service.👍
Can you bring the tutorial on the Kubernetes ? ( it would be a great relief )
I have completed the task which you have given. It has frontend, backend, sqs consumer and video processer which runs on ECS. I am not able to share the github link here.
Great video as always, need part 2
Hi piuysh pls make video on node js microservice with docker and kafka it is not available anywhere thanks
Sir make video on design of recommendation service of social media apps or shopping website using kafka
🚀 please make another part on it❤
Thanks for the crisp information ✨️💯
also a request you to upload the system design from the eraser as a ss in the repo
We want part 2
Can use bullMQ + Redis instead of SNS
PLEASE MAKE THE FULL WORKFLOW
bhaiya please ek devops series laiye
WE WANT PART 2
ek number thumbnail\
Bro, are you for real!!!! I just watched hitesh's video streaming and was looking for other resources. You are fucking goat man!!!
Part 2 lao 🙂
Sir sqs se message polling karne par ek message 2 to 3 times same aa jata hai jisse container three times spinn ho jata hai how to solve???
we want part 2. please....
Hi Piyush Lets code it
but here ia an issue what if I have uploaded videos in chunks and stored it on s3 buckets.Then what should be the best approach to generates HLS?
While configuring s3 notification there is option for multipart upload. Once the upload is completed then notification service will trigger.
Bhaisab 🔥
please make a video on live streaming like a youtube live
Great 👍 but do part 2
Great video ✅
Please part 2 sir
is there any way to implement it without docker
As far as i know you said ffmpeg will run for 4 times so can you please explain the server where this docker image is running is that server is capable of running ffmpeg 4 time for a file which is of 1 gb , and if yes then what will be the configuration of machine which we have to opt for this HLS conversion process,please reply
You can use elemental media convert
Hey I request you to make one video on kubernetes
Please make part 2
Sir, please bring the part 2
thank you bhai
Omg, Piyush! The container kept troughing 503 Service Unavailable and Repeated Startup and Shutdown because it needed more CPU core and memory for Transcoding, which was expensive and not permitted in my account. Can we use a VM instance or some particular service for video Transcoding right out of the box from the cloud?
Please reply.
For sure, you can have an Ec2 machine with docker-ode to manage containers.
@piyushgargdev I'm using gcp cuz I have some credits in that , can you guide me through a lil if have time. Btw I'm thrilled 😁 you replied. Am actually trying to transcoded large video/MP4 format with multiple audio streams. Pick from one bucket, process and upload to a another from where the clients picks up. I would be flattered if you can shower some of your experience on me. 🙌🙌♥️
Very informative !!!!