Hey guys, sorry for the poor sound quality. Please turn on the subtitles for a clear understanding. Also, if you are preparing for a system design interview, try get.interviewready.io 😁🔥
In last video during the interview i noticed you were not that comfortable about cdn I might be wrong But then Here you are, developed proper understanding on it. And sharing that knowledge. Great
mic up brother! would make all the difference and don't worry about zooming in and out so much. On a pos note, content and material was good and clear explanation. Thank you!
Great video. First time I watched about 3 years ago, I didn't understand anything but now, this is pretty obvious to me. Your content needs a lower barrier of entry in my opinion but otherwise it's a great resource to prepare for technical interviews.
I thought it was designing a CDN like Akamai, it is simply use case and explanation of why you need a CDN. Kindly change the title to be more appropriately. Perhaps : What is a CDN or What does CDN do.
@@gkcs grt explanation . but audio quality :(.. try this Rode NTG3 Condenser Shotgun Microphone .. bit expensive but worth the price for next 6-8 years
Hi Gaurav.. a few questions crept up.. We use CDN mostly not to hit the actual servers for frequently accessed static content.. Can you give more information on 1. what content will not be served by CDN 2. Does CDN need a load balancer 3. For the content which cannot be served by the CDN, is there a way to reduce the time to hit the actual server located at a different geography. Is setting up another sever in the geography going to help? 4. One naive question.. Are data center and server (application deployed) different? Setting up data center means only data realated stuff and not the application servers?
Hi Gaurav thanks for the great content. One correction - at around 10:16 the video says that 'the most popular CDN is S3' which I think is not true. S3 is a storage service and not a CDN per se.
i think he mentioned the cloudfront as a CDN service , so S3 stored and cloudfront serve the data to Edge Network[Nearby user geo location], and @Gaurav great explanation, even for security engineer it help me to understand the system very well, keep the good work
Right he was confused or may be in hurry said so... but s3 is just a storage in a region but cloud front is cdn which may get things from s3 and actually makes the access faster.
Great content. I really do appreciate you not using the word particular in every sentence like other indian youtubers. I get tired of hearing it so often.
At the end you mixed the concept of s3 and cloud front (cdn) which sounded same but they are different actually. May be you wanted to wrap up the video soon but anyways overall explanation was good.
What if the data gets updated but the cached data in the CDN has not expired ? Do we update data in the CDN too (incase of Push CDN) ? Do we let the data be inconsistent (incase of PULL CDN) till the TTL expires and the data is re-fetched ?
What is the sequence of call in this case ?? Client -> Application Server (in other region) -> CDN (local cache) or client directly connects with CDN and Application server is completely skipped ??
When Client requests for an static resource, let’s say requesting an image, clients are configured to hit a url with resource Id and the url is nothing but the CDN. So the client makes request to CDN, if found in cache (cache hit) then serves back the image, if not (cache miss) then CDN hits your service and get the resource, store it in cache and return the image to the client. And there should be optimal TTL for every resource you cache else either there will too many cache misses or caching for too long in both case you end up paying more to CDN 😊
Awesome video Gaurav. Right now I am working on a design where I have to host static files on CDN which stands behind the ALB. The only problem is adding security headers to it. AWS provides Lambda@edge to add Headers or quite recently AWS came up with CF functions. Maybe you could add that to the video. Also, maybe various people have already written, S3 being a storing mechanism and CF being the CDN AWS provides.
@Gaurav Sen - Thanks for explanation and I liked it.I do have a very naive question. The information you provided in this video is knowledge. So why do you title it as "system design". What aspect of the system is being designed in the video?
Thanks Gary :D I titled it system design since it is an important part of most systems, and some systems especially depend on CDNs to scale their solution.
@@gkcs -Thanks. Probably you may refer such videos as 'System design knowledge base'. Anyways, I should not be too picky on the title of the video and appreciate your effort on putting the video together. -:) I have another or new video request. Video can first start with back of the envelope calculations. In a system consisting of clients, servers, data base, networks of various types, what are the bandwidths of each components in the system that one can assume in real life? How would you find the bottleneck in the system, if # of servers, type of database, storage, network speeds are provided? Lastly, would you mind touching upon the subject what is the current industry bottleneck in cloud storage?
Nice content, there is a concept called PNI , please share the consequences if local Cache Utilization is 100%, then will the user request go via PNI(Via internet)
Thank you so much for the awesome video. I've been having difficulty distinguishing CDN and caching. 1. is CDN always placed right after the client? 2. If I place a cache before the DB, can we also make it a CDN? Thank you!!!
A Cache is a utility for faster lookup of frequently accessed data which is either pre-computed or queried to avoid multiple high-latency calls to a database/remote server. 1. A CDN is a cache very close to the user. It isn't right after client, but by design, a CDN is much closer to client than the server 2. A cache before a DB helps reduce a query time by 10-100x. Because it is behind server and not closer the the client, it cannot be categorised under CDN
How the user is gonna hit a specific cache (you mention the location in the request) but which tool is responsible to determine that? I guess something like geoDNS is gonna help
Great explanation Gaurav. What kind of web servers the cdns use to serve the content? Also the caches (edge locations) , are those coupled with web servers to serve the content that is cached?
No, for a user, URL remains same for a specific resource. But the routing behind the screens will take to the nearest CDN server. But again it's not guaranteed that it will always take to the nearest server, it depends on lot of factors like load, outage, routing/network configuration in-progress etc.
Hi @Gaurav. I'm a big fan of you. I have seen most of your videos. I recently had a interview and this question was asked to me. How would you design a system in which you need to allocate a 900TB integer matrix (that is already ordered on the rows and columns), so you can search for a number in that matrix the most efficient way. Any suggestions ?
@@gkcs Thank you for your response!!!! What strategy would you use to split the matrix? The numbers in the matrix are not unique. What data structure would you use to save the data?Those were some follow up questions.
What is the extra purpose of content delivery network if famous cloud providers have regions and we can replicate data in those regions to serve customer better. Also is S3 a CDN or CloudFront a CDN?
Looking at the CDN outage recently causing problems. Why didn’t companies implement something like if cdn request fails then directly hit the server for static files? Slow will be better than nothing right? Or am I missing something here?
Its better to just fail static content rather than failing the whole system, Lets suppose you had everything on the server and the server provider had an outage.. your static and dynamic content both would be out.. rather than just Static content ( Like images etc. )
Depends on the type of failure. Client doesn't know if the request are serving from cache or server. If the cache went down i think standard CDN provider must have a fallback option to serving from server or nearest cache server. If the DNS itself fails they don't have another option. This is my view only but if someone else have other insight - please add a point or correct me if i'm wrong.
@@anuj9538 With the latest fastly failure that took place, i realised maybe some big brains like Twitter Github etc. Weren't using the fallback option because their static contents were completely down for a while.
Can you please explain what is happened when we upload an image to Facebook and my friends react to it? Does this image upload to direct DB or via CDN? If it is upload directly when it is a callback to CDN?
bro I got job offers from Accenture as infrastructure and security engineer role and programer analyst role from cognizant(Genc) which one should I choose for the better carrier?
Hey guys, sorry for the poor sound quality. Please turn on the subtitles for a clear understanding.
Also, if you are preparing for a system design interview, try get.interviewready.io 😁🔥
Thank you for the advise :)
When the content is good - sound and video qualities can be very conveniently ignored!
Thanks for the video!
I thought my speakers are gone 😪...jokes apart it was just excellent content
Great explanation Gaurav, thanks! 💯
Thank you 😁
Sound quality should be improve
Thank you for sharing! Greetings from Brazil!
I love when you make it a story, it becomes much clearer and understandable, thank you gaurav, as always :)
Thank you so much for this 🙌🏾
You're welcome!
Very good explanation
That moment when a person who's teaching smiles
Thanks for the content!
grt explanation . but audio quality :(.. try this Rode NTG3 Condenser Shotgun Microphone ..
Paxos/Raft new learning. Your videos always amaze and I get to know new terminology every single time.
Thank you so much Gaurav
In last video during the interview i noticed you were not that comfortable about cdn I might be wrong
But then
Here you are, developed proper understanding on it. And sharing that knowledge.
Great
I noticed that too. I’m glad he took the initiative to learn CDN’s and share his understanding with us. 👍🏽
mic up brother! would make all the difference and don't worry about zooming in and out so much. On a pos note, content and material was good and clear explanation. Thank you!
Excellent video
Great explanation .. thanks
your videos are great! thanks! a little tip: maybe a microphone would help with the sound. keep them coming!
Great video. First time I watched about 3 years ago, I didn't understand anything but now, this is pretty obvious to me. Your content needs a lower barrier of entry in my opinion but otherwise it's a great resource to prepare for technical interviews.
Its not system design , you are telling me the concept.
great explanation, thank you
Great work, Gaurav! thank you so much 🤩
Thanks man, i dont have experience with most of things you say rn, vut i definitely know I will need it.
Asynchronous Dynamic Reverse Proxy Index AI based upon allocation requests with redundancy exponential difference reduction.
Thanks for the diversity t-shirt. It drew me in, and I’m happy to have found your channel as a result.
Very clearly explained! Thank you for the video - it helped me a lot! :)
I thought it was designing a CDN like Akamai, it is simply use case and explanation of why you need a CDN. Kindly change the title to be more appropriately. Perhaps : What is a CDN or What does CDN do.
Hope you take care of the acoustic next time....
Wonderful video, It would be more appreciatable if clarity of sound would be better. Thanks for sharing such knowledgeful videos.
Yes the mic had some issues this time. I'll make sure this doesn't happen again.
@@gkcs grt explanation . but audio quality :(.. try this Rode NTG3 Condenser Shotgun Microphone .. bit expensive but worth the price for next 6-8 years
@@gkcs probably 1 mic is a single point of failure, you can distribute this as well 😂
@@KAUSHALDOKANIA Hahaha! Damn :p
Hi Gaurav.. a few questions crept up.. We use CDN mostly not to hit the actual servers for frequently accessed static content.. Can you give more information on
1. what content will not be served by CDN
2. Does CDN need a load balancer
3. For the content which cannot be served by the CDN, is there a way to reduce the time to hit the actual server located at a different geography. Is setting up another sever in the geography going to help?
4. One naive question.. Are data center and server (application deployed) different? Setting up data center means only data realated stuff and not the application servers?
thanks you
Hi Gaurav thanks for the great content. One correction - at around 10:16 the video says that 'the most popular CDN is S3' which I think is not true. S3 is a storage service and not a CDN per se.
You’re right, all the content is explained well and then he mentions S3 as an example for CDN 🤦🏻♂️
i think he mentioned the cloudfront as a CDN service , so S3 stored and cloudfront serve the data to Edge Network[Nearby user geo location], and @Gaurav great explanation, even for security engineer it help me to understand the system very well, keep the good work
Right he was confused or may be in hurry said so... but s3 is just a storage in a region but cloud front is cdn which may get things from s3 and actually makes the access faster.
He mentioned cloudfront which is a AWS service that provides these geographical caching benefits
Awesome Explanation!
use kubernet instead of CDN?
its a design interview, but **spoiler alert** its also paid ad placement for akamai
Akamai didn't pay me. But if they'd like to, I wouldn't mind :p
the sound is bad :/
Please use mic , lot of echoes
Me and boys on the ways to use excel sheet for inter service communication
Great content. I really do appreciate you not using the word particular in every sentence like other indian youtubers. I get tired of hearing it so often.
I would really appreciate if you can you a good microphone. The content is amazing 😀
Great.
Great video.... Kindly take care of the audio, it ruins the experience of such great explanation.
Thanks for the video. This seem to be a complicated explanation of CDNs, I feel.
Hey Guarav, Nice video on CDN. Just want to point out S3 is not really a CDN it's an object storage. Btw AWS cloudfront is a CDN service
Thanks. We use them accordingly at our startup InterviewReady 😁
cdn are replicated servers. are they anyhow related to caching?
At the end you mixed the concept of s3 and cloud front (cdn) which sounded same but they are different actually. May be you wanted to wrap up the video soon but anyways overall explanation was good.
Wonderful Explanation, the audio quality can be improved!
Could not proceed ... Because of poor Audio quality
Nice content. Please make your studio echo proof by attaching those "panels" to the walls
Please use a better quality of mic, the explanation is very good but the audio quality is not so good.
It'd be great if you explain concept of sso on whiteboard
Isn’t Cloudfront more relevant as a CDN than S3? S3 may be the origin for Cloudfront. ??
Could you please do videos on networking fundamentals like vpc , elastic ips etc
What if the data gets updated but the cached data in the CDN has not expired ? Do we update data in the CDN too (incase of Push CDN) ? Do we let the data be inconsistent (incase of PULL CDN) till the TTL expires and the data is re-fetched ?
What is the sequence of call in this case ?? Client -> Application Server (in other region) -> CDN (local cache) or client directly connects with CDN and Application server is completely skipped ??
When Client requests for an static resource, let’s say requesting an image, clients are configured to hit a url with resource Id and the url is nothing but the CDN. So the client makes request to CDN, if found in cache (cache hit) then serves back the image, if not (cache miss) then CDN hits your service and get the resource, store it in cache and return the image to the client. And there should be optimal TTL for every resource you cache else either there will too many cache misses or caching for too long in both case you end up paying more to CDN 😊
@@codr0514 Thanks... Thanks a lot.
Can you please work on audio clarity.The voice is echoing.
Awesome video Gaurav. Right now I am working on a design where I have to host static files on CDN which stands behind the ALB. The only problem is adding security headers to it. AWS provides Lambda@edge to add Headers or quite recently AWS came up with CF functions. Maybe you could add that to the video. Also, maybe various people have already written, S3 being a storing mechanism and CF being the CDN AWS provides.
Please work on your audio quality. Great Content
Good content, but audio is not clear
The audio quality is quite poor Gaurav bhai
please i have a question. i am an isp , can i own a cdn for me to save banwidth .please answer
Have a look at my netflix video here: ruclips.net/video/x9Hrn0oNmJM/видео.html
Please use a better quality microphone. The voice was echoing a lot.
nicely done but poor sound quality.
Noticed the wedding ring in the last video but thought better of commenting there. However, congratulations.
Thank you 😁
@Gaurav Sen - Thanks for explanation and I liked it.I do have a very naive question. The information you provided in this video is knowledge. So why do you title it as "system design". What aspect of the system is being designed in the video?
Thanks Gary :D
I titled it system design since it is an important part of most systems, and some systems especially depend on CDNs to scale their solution.
@@gkcs -Thanks. Probably you may refer such videos as 'System design knowledge base'. Anyways, I should not be too picky on the title of the video and appreciate your effort on putting the video together. -:)
I have another or new video request. Video can first start with back of the envelope calculations. In a system consisting of clients, servers, data base, networks of various types, what are the bandwidths of each components in the system that one can assume in real life? How would you find the bottleneck in the system, if # of servers, type of database, storage, network speeds are provided? Lastly, would you mind touching upon the subject what is the current industry bottleneck in cloud storage?
great
Nice content, there is a concept called PNI , please share the consequences if local Cache Utilization is 100%, then will the user request go via PNI(Via internet)
Loved it
bro video was on point, had a hard time understanding because of unclear voice
Thank you so much for the awesome video. I've been having difficulty distinguishing CDN and caching.
1. is CDN always placed right after the client?
2. If I place a cache before the DB, can we also make it a CDN?
Thank you!!!
A Cache is a utility for faster lookup of frequently accessed data which is either pre-computed or queried to avoid multiple high-latency calls to a database/remote server.
1. A CDN is a cache very close to the user. It isn't right after client, but by design, a CDN is much closer to client than the server
2. A cache before a DB helps reduce a query time by 10-100x. Because it is behind server and not closer the the client, it cannot be categorised under CDN
How the user is gonna hit a specific cache (you mention the location in the request) but which tool is responsible to determine that? I guess something like geoDNS is gonna help
Love how the users from netherlands are green ;)
Thank you. Could you elaborate on why we need Distributed Consensus (Paxos/Raft) on our cache CND?
Cloudflares ?
Finally trimmed good haircut!
Thought this is Designing CDN itself but it's about using CDNs....
Does CDNs use object stores for hosting. Can we make use of S3 as it's alternative.
My man! You and your videos are nothing short of phenomenal! Thank you very much.
I just quit watching this video in just a 15 seconds, sound quality must improve, please do re upload the video…
Thanks Gaurav for the video. How is the deployment taken care of ?
Nice t-shirt man
Sound sir, sound...
Great explanation Gaurav. What kind of web servers the cdns use to serve the content? Also the caches (edge locations) , are those coupled with web servers to serve the content that is cached?
I can relate it with CSGO game now. 😅
Can we make our own cdn and use it? Like you explain in the video and what will be the cost if we developed it.
Difficult to say. Why would you want to reinvent the wheel though? Cost savings?
@@gkcs Not exactly cost but want to learn how it work from ground. That's it.
@Gaurav : Once a file, say a video is uploaded to CDN, does it get a different URL for each geographic location ?
No, for a user, URL remains same for a specific resource. But the routing behind the screens will take to the nearest CDN server. But again it's not guaranteed that it will always take to the nearest server, it depends on lot of factors like load, outage, routing/network configuration in-progress etc.
Just an observation: If your T-shirt designer would have put "Respect" before "Integrity" then they wouldn't have to add extra "Pride" at the end.
Hi @Gaurav. I'm a big fan of you. I have seen most of your videos. I recently had a interview and this question was asked to me. How would you design a system in which you need to allocate a 900TB integer matrix (that is already ordered on the rows and columns), so you can search for a number in that matrix the most efficient way. Any suggestions ?
Split the matrix into 900 sorted chunks. Now use a reverse proxy to route requests based on the input number (binary search)
@@gkcs Thank you for your response!!!! What strategy would you use to split the matrix? The numbers in the matrix are not unique. What data structure would you use to save the data?Those were some follow up questions.
@@manuelvaldez448 i would split the matrix into 30*30 chunks to make a higher level matrix.
Hey dude, thanks for video, although definitely not the best one of your, you can do better - more depth would be nice
You opened the black box buddy.....
What is the extra purpose of content delivery network if famous cloud providers have regions and we can replicate data in those regions to serve customer better. Also is S3 a CDN or CloudFront a CDN?
S3 is the storage mechanism. Cloudfront is the CDN.
great hairstyle :)
so is s3 installed on each of the nodes on the CDN?
its the bad sound and the strong accent combo that makes this a bad video
good content but t bad audio quality. please buy good mike
Love your shirt!
Hi can you provide the simulators for CDN? What are the different simulators can be used for the simulation of CDN?
Use a staging environment, or stub it through the programming environment.
Looking at the CDN outage recently causing problems. Why didn’t companies implement something like if cdn request fails then directly hit the server for static files? Slow will be better than nothing right? Or am I missing something here?
Its better to just fail static content rather than failing the whole system,
Lets suppose you had everything on the server and the server provider had an outage.. your static and dynamic content both would be out.. rather than just Static content ( Like images etc. )
Depends on the type of failure. Client doesn't know if the request are serving from cache or server. If the cache went down i think standard CDN provider must have a fallback option to serving from server or nearest cache server. If the DNS itself fails they don't have another option.
This is my view only but if someone else have other insight - please add a point or correct me if i'm wrong.
@@anuj9538 With the latest fastly failure that took place, i realised maybe some big brains like Twitter Github etc. Weren't using the fallback option because their static contents were completely down for a while.
It wasn't a CDN Outage first of all, it was a DNS outage for specific set of customers which got in triggered due to a config push
@@umanga312 dns outage was a recent one, there was some cdn outage because of a misconfig before that I suppose.
Can you please explain what is happened when we upload an image to Facebook and my friends react to it?
Does this image upload to direct DB or via CDN? If it is upload directly when it is a callback to CDN?
Voice is bit unclear!!
Yes the mic was off 🤦♂️😅
bro I got job offers from Accenture as infrastructure and security engineer role and programer analyst role from cognizant(Genc) which one should I choose for the better carrier?
Bro! I think you should join the congnizant role. It's more future proof.
Good content. Poor Audio.