Just found your channel and I'm really enjoying these bite sized videos for certain topics. It certainly helps a new developer get in touched with industry key terms and concepts.
@@BeABetterDev If i have a web application installed on 1 server, what is the method to make the app more scalable. and the load balancer router used only if we have 1 server running the application ?
@@BeABetterDev I have a question please. I have a system design interview coming up so how do I know which load balancer to recommend? I know nginx is popular. I know a bit about aws but I also know the company uses GCP so is it best to go with that? Thanks for your help :)
Hi Elena, I haven't worked with NGINX too much but here's my take. NGINX load balancing requires server setup, configuration, updates, and monitoring that needs to be done to ensure the LB is constantly up to date and working correctly. Additionally there's lots of manual configuration that needs to be done via config files that affect how load is distributed across the machines in the cluster. This may be overwhelming for some to understand the implications of the settings and how to optimize for their use case. AWS / GCP both have Load Balancer products that generally take care of the installation/updates for you. You don't have to worry about provisioning the server as this is all done for you and AWS ensures a healthy load balancing instance(s) remains up at all times. The nice thing is that configuration is pretty self explanatory and can be set up through the AWS / GCP console. One thing I really like about AWS is that it supports two popular types of load balancers, Application and Network. Application load balancer operates on the L7 level of the OSI model - this allows you to route requests to different targets based on the content within HTTP request. AWS also supports a network load balancer which operates on the L4 level which is generally operating on the packet level. L4 is much more efficient than L7 and allows you to pump many many more requests through it without compromising it - so its great for scale. There's a great article on the difference between L4 and L7 here: www.nginx.com/resources/glossary/layer-4-load-balancing/ All in all, what LB product you go with generally depends on questions you want to ask yourself like "how much configuration do I want to do? what scale are we expecting? do we want to route requests based on request content or not?" The answers to these questions should determine the path you take. Hope this answer was helpful. I'll keep an eye on this comment stream if you have any followup questions. Alternatively you can email me at daniel@beabetterdev.com and I'll do my best to get back to you. Good luck on your interview!
@@BeABetterDev Thank you so so so so much Daniel for this detailed answer, you are awesome! I found it very helpful indeed and made me realise which LB I want to go with. I am super happy I found your channel and will definitely recommend to others.
Thanks so much Elena! Glad I could help and welcome to the channel! Thanks so much for sharing with others - really appreciate you getting the word out! Daniel
how does the load balancer Offload traffic or redirect traffic OFF itself? meaning.. if a request comes into the load balancer, then how does it keep all that traffic from flowing through the load balancer itself? It has to route traffic away from itself. So, how does that work if the load balancer is the main point of entry to the network?
When people talk about load balancing, my doubt is always, "What will you do with the state"? How can I "scale" by adding different machines if I need one source of truth for my state? On applications with databases, we can easily replicate many things to serve more traffic. Still, in the end, we will have one super central machine with the DB or multiple DBs, which makes me think of a big issue due to replication. Thanks for the video!
Ok new question: So load balancing and horizontal scaling have allowed you to handle increasing traffic. But! If all these requests then hit a separate database server, you still have overload issues no? And if you were to split it up (not sure how (I'm a newbie)), would you not have data integrity issues??
Hey Alexandru, Great question! You will definitely get increased traffic on your database. To mitigate this, you can use a cloud based database provider that handles autoscaling as well (DynamoDB is a good choice!). Cheers
Dumb question, but, what if load balancer is overloaded? What makes one load balancer better in terms of handling millions of users compared to two web servers? Are these machines normally resource-intensive?
Hi Churka, Load balancers are usually dedicated pieces of hardware that are optimized to simply handle large volumes of traffic concurrently. I've seen many load balancers capable of handling thousands of requests concurrently and routing them successfully to appropriate clients. As an application grows, it may be appropriate to 'partition' the load balancers. Its almost like a piece of lego that can be continuously broken apart into separate chunks. Each chunk has the identical amount of scaling capability of the parent. Hope this helps!
Thanks for the tutorial clearly it helps massively, can we configure 2 load balancers instead of one(so called Active-Passive) in case of one becomes unresponsive? Note: I'll go and hit the subscribe button
Hi Tcherno! This is definitely possible to set up an active-passive configuration. Of course this depends on the tech stack youre using. If youre using something like AWS load balancers, this is handled automatically for you. But if you're rolling your own for instance, nginx application, it would require more work on your part. Hope this helps and thanks for watching!
Nice explanation. I have a small question. Is it not that load balancer itself will become bottle neck as all requests will come to it and it has to forward requests? How is this handled? Is return traffic handled from server to client through load balancer or it goes directly between server and client? If it goes through load balancer is it not a bottle neck?
Hi Mallesh, This is a great question. The load balancers are typically highly specialized hardware that can handle many many concurrent requests/connections at the same time. Additionally, they have built in health checks to ensure both the LB and downstream targets are always available. In the event of a failure, most cloud providers will automatically provision a new LB for you with no action from your side. Hope this helps, Daniel
Thanks for such informative video. Can one load balancer handle millions of requests? How Facebook, Amazon etc handles the user traffic? Do they have single load balancer? If I am in India and Load balancer is in USA so does it mean my request will go USA Load balancer and then to actual server for processing? Kindly explain.
Hi Paresh, Load balancers can definitely handle millions of requests. This is what they were designed to do. Websites like Facebook, Amazon, etc do have load balancers on the backend, but make heavy use of caching to reduce the load on their backend services unless it is absolutely necessary. Hope this makes sense. Daniel
Liked! Subscribed! Hey got question here, how do we point public IP and private for load balancers? I understood private in vm itself and public definitely via firewall, but how about for LoadBalance + IPSec config? Not sure if you understand what I meant.
: hi.. i have a question can you apply load balancing in a scenario when 1 machine/ISP connection has the highest priority while the others would be just backups? (using this setup for home-based ISP connection btw) great video!
Hi there! Yes this is certainly possible, but I believe this would need to occur at the DNS level. What you are effectively asking for is a concept called DNS Failover. You can learn more about it here: docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-configuring.html
![How load balancers work - System Design Interview knowledge [Beyond the interview]](/img/1.gif)
I think "Security" deserves to be on the list too, since only the load balancer is exposed to the public net.
Just found your channel and I'm really enjoying these bite sized videos for certain topics. It certainly helps a new developer get in touched with industry key terms and concepts.
Welcome aboard!
@@BeABetterDev If i have a web application installed on 1 server, what is the method to make the app more scalable.
and the load balancer router used only if we have 1 server running the application ?
Wow thank you, you're super great at explaining technical concepts.
Precisely what i needed to understand this...
Glad I could help!
Great Explanation :D
Question : Doesn't LB again makes a Single point failure problem??
This is amazing and helped me fully understand what I was puzzled in.
Glad I could help Azim!
Great explanation! Thank you.
Glad it was helpful!
Fantastic explanation.
Glad it was helpful!
wow really great, detail explained and more improvement
DUDE. Great video. Thank you.
You're very welcome, Paul!
You explain things very well
Thanks Neil!
Great explanation. I'll be setting up a GEO based load balancer :)
Thanks Slim! Good luck on your project :)
Fantastic video! Thanks a lot.
You're very welcome Elena!
@@BeABetterDev I have a question please. I have a system design interview coming up so how do I know which load balancer to recommend? I know nginx is popular. I know a bit about aws but I also know the company uses GCP so is it best to go with that? Thanks for your help :)
Hi Elena, I haven't worked with NGINX too much but here's my take.
NGINX load balancing requires server setup, configuration, updates, and monitoring that needs to be done to ensure the LB is constantly up to date and working correctly. Additionally there's lots of manual configuration that needs to be done via config files that affect how load is distributed across the machines in the cluster. This may be overwhelming for some to understand the implications of the settings and how to optimize for their use case.
AWS / GCP both have Load Balancer products that generally take care of the installation/updates for you. You don't have to worry about provisioning the server as this is all done for you and AWS ensures a healthy load balancing instance(s) remains up at all times. The nice thing is that configuration is pretty self explanatory and can be set up through the AWS / GCP console.
One thing I really like about AWS is that it supports two popular types of load balancers, Application and Network. Application load balancer operates on the L7 level of the OSI model - this allows you to route requests to different targets based on the content within HTTP request. AWS also supports a network load balancer which operates on the L4 level which is generally operating on the packet level. L4 is much more efficient than L7 and allows you to pump many many more requests through it without compromising it - so its great for scale. There's a great article on the difference between L4 and L7 here: www.nginx.com/resources/glossary/layer-4-load-balancing/
All in all, what LB product you go with generally depends on questions you want to ask yourself like "how much configuration do I want to do? what scale are we expecting? do we want to route requests based on request content or not?" The answers to these questions should determine the path you take.
Hope this answer was helpful. I'll keep an eye on this comment stream if you have any followup questions. Alternatively you can email me at daniel@beabetterdev.com and I'll do my best to get back to you.
Good luck on your interview!
@@BeABetterDev Thank you so so so so much Daniel for this detailed answer, you are awesome! I found it very helpful indeed and made me realise which LB I want to go with. I am super happy I found your channel and will definitely recommend to others.
Thanks so much Elena! Glad I could help and welcome to the channel! Thanks so much for sharing with others - really appreciate you getting the word out!
Daniel
please provide more detailed video with various type of load balancer
Thank you!
You're welcome!
how does the load balancer Offload traffic or redirect traffic OFF itself? meaning.. if a request comes into the load balancer, then how does it keep all that traffic from flowing through the load balancer itself? It has to route traffic away from itself. So, how does that work if the load balancer is the main point of entry to the network?
Love the video! Thanks for the great content :)
When people talk about load balancing, my doubt is always, "What will you do with the state"? How can I "scale" by adding different machines if I need one source of truth for my state? On applications with databases, we can easily replicate many things to serve more traffic. Still, in the end, we will have one super central machine with the DB or multiple DBs, which makes me think of a big issue due to replication. Thanks for the video!
Use database for states. Use specialized servers to handle db and don't do it on your own
Ok new question: So load balancing and horizontal scaling have allowed you to handle increasing traffic. But! If all these requests then hit a separate database server, you still have overload issues no? And if you were to split it up (not sure how (I'm a newbie)), would you not have data integrity issues??
Hey Alexandru,
Great question! You will definitely get increased traffic on your database. To mitigate this, you can use a cloud based database provider that handles autoscaling as well (DynamoDB is a good choice!).
Cheers
Nice....so on point
Thank you so much!
Dumb question, but, what if load balancer is overloaded? What makes one load balancer better in terms of handling millions of users compared to two web servers? Are these machines normally resource-intensive?
Hi Churka,
Load balancers are usually dedicated pieces of hardware that are optimized to simply handle large volumes of traffic concurrently. I've seen many load balancers capable of handling thousands of requests concurrently and routing them successfully to appropriate clients.
As an application grows, it may be appropriate to 'partition' the load balancers. Its almost like a piece of lego that can be continuously broken apart into separate chunks. Each chunk has the identical amount of scaling capability of the parent.
Hope this helps!
Thanks for the tutorial clearly it helps massively, can we configure 2 load balancers instead of one(so called Active-Passive) in case of one becomes unresponsive?
Note: I'll go and hit the subscribe button
Hi Tcherno! This is definitely possible to set up an active-passive configuration. Of course this depends on the tech stack youre using. If youre using something like AWS load balancers, this is handled automatically for you. But if you're rolling your own for instance, nginx application, it would require more work on your part.
Hope this helps and thanks for watching!
interesting topic, thank you!
What is the different between a load balancer and a router? Both are used to handle traffic.
Great information!
Thank you..
Glad it was helpful!
Isin't consistent hashing one of the ways to route the traffic?
Nice explanation. I have a small question. Is it not that load balancer itself will become bottle neck as all requests will come to it and it has to forward requests? How is this handled?
Is return traffic handled from server to client through load balancer or it goes directly between server and client? If it goes through load balancer is it not a bottle neck?
Hi Mallesh,
This is a great question. The load balancers are typically highly specialized hardware that can handle many many concurrent requests/connections at the same time. Additionally, they have built in health checks to ensure both the LB and downstream targets are always available. In the event of a failure, most cloud providers will automatically provision a new LB for you with no action from your side.
Hope this helps,
Daniel
Thanks for such informative video. Can one load balancer handle millions of requests? How Facebook, Amazon etc handles the user traffic? Do they have single load balancer? If I am in India and Load balancer is in USA so does it mean my request will go USA Load balancer and then to actual server for processing? Kindly explain.
Hi Paresh,
Load balancers can definitely handle millions of requests. This is what they were designed to do. Websites like Facebook, Amazon, etc do have load balancers on the backend, but make heavy use of caching to reduce the load on their backend services unless it is absolutely necessary. Hope this makes sense.
Daniel
Liked! Subscribed! Hey got question here, how do we point public IP and private for load balancers? I understood private in vm itself and public definitely via firewall, but how about for LoadBalance + IPSec config? Not sure if you understand what I meant.
: hi.. i have a question
can you apply load balancing in a scenario when 1 machine/ISP connection has the highest priority while the others would be just backups? (using this setup for home-based ISP connection btw)
great video!
Hi there! Yes this is certainly possible, but I believe this would need to occur at the DNS level. What you are effectively asking for is a concept called DNS Failover. You can learn more about it here: docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-configuring.html
Who balances Load Balancer and DNS server?
"Wow my writing is terrible, I apologise" 😅😂
Way to dumb it down so amateurs can understand the concept!
Thanks Joani! Glad you enjoyed.