When using dynamic load balancing, especially with a feedback loop (e.g. dynamic weights updated based on some server-side metric), you probably want some static guard rails to keep imbalance under control in case something unexpected happens (e.g. a sudden hot spot). Also, it's worth looking for the possibility of positive feedback loops, for instance if certain kinds of errors happening on one particular server (e.g. configuration problem) make processing very fast, and thus make this server even more attractive to the load balancing system. Error metrics are easily overlooked when computing weights for load balancing.
good point mentioning that certain error scenario will free up the resources when used some circuit breaking mechanism. Load balancing should make sure to not overlook this property.
Summary:- Static - doesn't consider server real time connections ,performance and metrics Dynamic -> Consider performance and metrics Static :- 1) Round Robin -> Sends request in the round robin fashion 2) Sticky Round Robin -> Sends subsequent request of the same user to the same server. All related data on the same server. 3) Weighted Round Robin -> Assigns weights to the servers. Sends more requests to the server with more weight 4) IP/Hash -> Calculates the hash and sends request.(can generate evenly distributed load based on the hash function). Dynamic:- 1) Least Connections -> Sends requests to the server with least connections. 2) Least time -> Sends requests to the server with current low latency and faster response time.
🎯 Key Takeaways for quick navigation: 00:07 🌐 *Overview of Load Balancing Algorithms* - Crucial for large-scale web apps. - Two categories: static and dynamic. - Goal: Grasp core load balancing for better app architecture. 01:07 🔄 *Static Load Balancing Algorithms* - Distribute requests without real-time server consideration. - Examples: Round Robin, Sticky Round Robin, Weighted Round Robin. - Trade-off: Simplicity vs. Adaptability. 02:37 🔍 *Hash-Based Algorithms* - Hash functions map requests to servers. - Challenge: Optimal hash function choice. - Advantage: Even distribution with a wise function. 03:11 🔄 *Dynamic Load Balancing Algorithms* - Adapt in real-time based on server conditions. - Examples: Least Connections, Least Response Time. - Trade-off: Adaptability vs. Overhead. 04:13 ⚖️ *Trade-offs between Static and Dynamic Algorithms* - Consider trade-offs in load balancing selection. - Static for stateless apps, dynamic for complex ones. 04:44 💬 *Viewer Engagement and Conclusion* - Encourage sharing load balancing experiences. - Highlight simplified "static" and "dynamic" differentiation. - Promote system design newsletter. Made with HARPA AI
When I first heard of load balancing I imagined the round robin algorithm like distributing cards to all the players in the game. I didn't even think there are algorithms on distributing the load
ChatGPT responses with a bit different algorithms, less Round Robins and more Least Connections: 1. Round Robin 2. Least Connections 3. Weighted Round Robin 4. Weighted Least Connections 5. IP Hash 6. Least Response Time Bard responses with a bit different algorithms, Geolocation instead of IP Hash: 1. Round robin. 2. Weighted round robin. 3. Least connections. 4. Least response time. 5. Sticky sessions. 6. Geolocation. :D
One thing i would like to correct Sticky session algorithms can be based on cookies, IP or URL in order for load balancer to identity which requests are part of same session between client and the target server selected initially by load balancer. Also another algorithm to add to Dynamic algorithms is health check which test periodically on server health probes to see if its live and ready
Is Load Balancing a good defence against a DDOS? • If it's a high volume lightweight attack like a Smurfed ping won't the load balancer need to process each packet anyway, even if very briefly? • Under what circumstances will trivial processing overwhelm the load balancer and is this different between algorithms? • I'm assuming Sticky Round Robin or IP/URL can just drop all the packets from a user or IP respectively to dodge a denial of service attack. Is this the case? • How are TCP/IP layers split between the load balancer and the destination machine? Particularly, if an attacker sends massage parts at a rate just under the connection timeout will the load balancer get overwhelmed by excessive open connections or will the destination machine (slow loris attack)?
your load balancer reverse proxy is simply a way to distribute requests to different services. For example if you do not have rate limiting in place to stop one client from requesting to much to the load balancer it didn't really do all that much to defend against the ddos other than effectively distributing the load to be able to handle more requests.
@@KingMikolaj I spent a while thinking of that comment 3 months ago. I'm glad someone finally responded to it. It feels like you are saying a load balancer is completely useless in mitigating a DDOS attack. You seem to be saying that a load balancer can only be used to distribute requests among services. I have never owned or implemented a load balancer but I feel this is not the case. Why would we need to distribute requests if not to keep a server from getting overloaded?
@@PatrickStaightI understand it this way: to some extent a load balancer of course may help servers not to be overloaded, evenly distributing ddos (i.e. very frequent) requests among the servers. but at some point servers will not be able to process such a big number of incoming requests - again, they are evenly loaded with the help of our load balancer. in order system not to fail, we need to implement something else - not only a load balancer. we need somehow to detect that someone sends too many requests and stop processing or limit them
@@vishalchoubey5606 random is different from consistent hashing.Consistent hashing will always send specific request to specific backends. I don't think this is good in case of load balancer. Let suppose we have problematic "page", it will always be send to one and same backed. Let also suppose we have viral "page", it will always be send to one and same backed. Then the backend will die.
Agree, DNS uses this to distribute the traffic among the load balancers, this is before the actual API servers. Multiple load balancers are deployed for fault tolerance. But this may not be the choice at the load balancer as it can cause API server overload.
can u explain about concurrency and redundancy problems when using multiple servers? for example should we have a separate copy of the application on each server or there are other ways for doing it? another consideration is concurrent access to shared resources... how preventing conflicts and data corruption is implemented? probably another subcategory is distributing the database itself... i think these can be candidates for separate videos indeed ;)
Hi, your videos are too low in volumn I need to increase it too full for clear but when switch tab next YT video it'll be too loud. So it for all the end users or only for my system. But your videos are very informative thanks for that.
Hey, I'm new to System design, Can we add multiple load balancers in a complex system? if yes then how to map requests between them? Do we need another load balancer to manage those two?
ByteByteGo - please please share the tools and softwares you use to create these wonderful videos. It will be extremely helpful to learn them and use it for work and share knowledge in general. Thanks in advance!
Surely there has to be hybrid solutions to this problem. Some of these details in the "cons" category for these different methods feel like they could be easily overcome. #3 for example - why would an admin be required to actively monitor these values and adjust them? Surely that task could be automated with code.
The lowest latency method shouldn't require *that much* overhead, right? If you have enough traffic that you're tracking multiple outstanding requests per server, then you can order each server by whichever is longest: the latency of the last fulfilled request or the elapsed time since the oldest outstanding request. That way, you (practically) don't have any overhead measuring latency. If you don't have enough traffic to accumulate multiple requests per server, then load-balancing shouldn't be a huge problem. You can add more complex logic to minimize hot-spots and detect server failures, but the gist is still the same.
I think dynamic latency algorithm is complex because you should take in account each specific request.. example, a get request that fetchs a resource from a remote server should not have the same weight as a post request that create a resource with the use of a remote external api..
I've seen a lot of load balancing configurations in my time, but I don't think I have ever seen anyone use URL hashing. Wonder what scenario that is good for.
url hashing lends towards session affinity, or if one of your serving servers is in a different security tier/domain than the others. It lets you serve that secure content from a one or a subset of your fleet. It also shows up in microservices here and there.
What load balancing method is suitable for Backends such as Nodejs used for Live Streaming Applications? *So the Backend is a live streaming application
If the load balancer is a single server with single IP address, how can it possibly handle so much traffic? I know it just forwards request but the IO overhead can add up, right?
Deberias replicar el servidor con alguna tecnologia como docker para tener mas y mas sevidores desplegados horizontalmente en contenedores cuando se alcance el limite asignado para cada uno. Si solo hacer depender a tu app de 1 servidor y este no esta manejado por alguna tecnologia de reemplazo de contenedores como kubernetes es muy posible que tu servidor falle cuando tu app vaya escalando :(
Hello everyone, I am working on load balancing algorithm in CloudSim 3.0.3 and main problem is deadline of task. If I have MIPS that can not execute this task before deadline, what I need to do? Can anyone help me?
The narrative here is powerful and impactful. A similar book I read was transformative in its reach. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
When using dynamic load balancing, especially with a feedback loop (e.g. dynamic weights updated based on some server-side metric), you probably want some static guard rails to keep imbalance under control in case something unexpected happens (e.g. a sudden hot spot).
Also, it's worth looking for the possibility of positive feedback loops, for instance if certain kinds of errors happening on one particular server (e.g. configuration problem) make processing very fast, and thus make this server even more attractive to the load balancing system.
Error metrics are easily overlooked when computing weights for load balancing.
Good point. But i think a server with multiple errors should be monitored and fixed thus, it should not keep running with errors for a while..
good point mentioning that certain error scenario will free up the resources when used some circuit breaking mechanism. Load balancing should make sure to not overlook this property.
Summary:-
Static - doesn't consider server real time connections ,performance and metrics
Dynamic -> Consider performance and metrics
Static :-
1) Round Robin -> Sends request in the round robin fashion
2) Sticky Round Robin -> Sends subsequent request of the same user to the same server. All related data on the same server.
3) Weighted Round Robin -> Assigns weights to the servers. Sends more requests to the server with more weight
4) IP/Hash -> Calculates the hash and sends request.(can generate evenly distributed load based on the hash function).
Dynamic:-
1) Least Connections -> Sends requests to the server with least connections.
2) Least time -> Sends requests to the server with current low latency and faster response time.
🎯 Key Takeaways for quick navigation:
00:07 🌐 *Overview of Load Balancing Algorithms*
- Crucial for large-scale web apps.
- Two categories: static and dynamic.
- Goal: Grasp core load balancing for better app architecture.
01:07 🔄 *Static Load Balancing Algorithms*
- Distribute requests without real-time server consideration.
- Examples: Round Robin, Sticky Round Robin, Weighted Round Robin.
- Trade-off: Simplicity vs. Adaptability.
02:37 🔍 *Hash-Based Algorithms*
- Hash functions map requests to servers.
- Challenge: Optimal hash function choice.
- Advantage: Even distribution with a wise function.
03:11 🔄 *Dynamic Load Balancing Algorithms*
- Adapt in real-time based on server conditions.
- Examples: Least Connections, Least Response Time.
- Trade-off: Adaptability vs. Overhead.
04:13 ⚖️ *Trade-offs between Static and Dynamic Algorithms*
- Consider trade-offs in load balancing selection.
- Static for stateless apps, dynamic for complex ones.
04:44 💬 *Viewer Engagement and Conclusion*
- Encourage sharing load balancing experiences.
- Highlight simplified "static" and "dynamic" differentiation.
- Promote system design newsletter.
Made with HARPA AI
When I first heard of load balancing I imagined the round robin algorithm like distributing cards to all the players in the game. I didn't even think there are algorithms on distributing the load
I am a product manager in payments domain, this video is very Useful and easily covered my my need of the hour
ChatGPT responses with a bit different algorithms, less Round Robins and more Least Connections:
1. Round Robin
2. Least Connections
3. Weighted Round Robin
4. Weighted Least Connections
5. IP Hash
6. Least Response Time
Bard responses with a bit different algorithms, Geolocation instead of IP Hash:
1. Round robin.
2. Weighted round robin.
3. Least connections.
4. Least response time.
5. Sticky sessions.
6. Geolocation.
:D
I;ve just finished to "warm up" load balancing section before intervview and at the end get your vid!:)
You make it so simple to understand
You make amazing content. Keep it up!
best channel for fresher developer !
One thing i would like to correct Sticky session algorithms can be based on cookies, IP or URL in order for load balancer to identity which requests are part of same session between client and the target server selected initially by load balancer.
Also another algorithm to add to Dynamic algorithms is health check which test periodically on server health probes to see if its live and ready
Is Load Balancing a good defence against a DDOS?
• If it's a high volume lightweight attack like a Smurfed ping won't the load balancer need to process each packet anyway, even if very briefly?
• Under what circumstances will trivial processing overwhelm the load balancer and is this different between algorithms?
• I'm assuming Sticky Round Robin or IP/URL can just drop all the packets from a user or IP respectively to dodge a denial of service attack. Is this the case?
• How are TCP/IP layers split between the load balancer and the destination machine? Particularly, if an attacker sends massage parts at a rate just under the connection timeout will the load balancer get overwhelmed by excessive open connections or will the destination machine (slow loris attack)?
your load balancer reverse proxy is simply a way to distribute requests to different services. For example if you do not have rate limiting in place to stop one client from requesting to much to the load balancer it didn't really do all that much to defend against the ddos other than effectively distributing the load to be able to handle more requests.
@@KingMikolaj I spent a while thinking of that comment 3 months ago. I'm glad someone finally responded to it.
It feels like you are saying a load balancer is completely useless in mitigating a DDOS attack.
You seem to be saying that a load balancer can only be used to distribute requests among services.
I have never owned or implemented a load balancer but I feel this is not the case.
Why would we need to distribute requests if not to keep a server from getting overloaded?
@@PatrickStaightI understand it this way: to some extent a load balancer of course may help servers not to be overloaded, evenly distributing ddos (i.e. very frequent) requests among the servers. but at some point servers will not be able to process such a big number of incoming requests - again, they are evenly loaded with the help of our load balancer.
in order system not to fail, we need to implement something else - not only a load balancer. we need somehow to detect that someone sends too many requests and stop processing or limit
them
you miss random load balancing. it is similar to round robin, except uses random function e.g. something like rand(0, 256) % 4.
DNS supports that.
Rather use consistent hashing :`}
@@vishalchoubey5606 random is different from consistent hashing.Consistent hashing will always send specific request to specific backends. I don't think this is good in case of load balancer.
Let suppose we have problematic "page", it will always be send to one and same backed.
Let also suppose we have viral "page", it will always be send to one and same backed. Then the backend will die.
Agree, DNS uses this to distribute the traffic among the load balancers, this is before the actual API servers. Multiple load balancers are deployed for fault tolerance.
But this may not be the choice at the load balancer as it can cause API server overload.
This channel is a gem 💎
can u explain about concurrency and redundancy problems when using multiple servers?
for example should we have a separate copy of the application on each server or there are other ways for doing it?
another consideration is concurrent access to shared resources... how preventing conflicts and data corruption is implemented?
probably another subcategory is distributing the database itself...
i think these can be candidates for separate videos indeed ;)
Thanks I was looking for this video.
Hi! You've excellent stuff on your channel. Thanks a tonne for sharing 🙏✌🏻
great content thank you for all information.
Hi, your videos are too low in volumn I need to increase it too full for clear but when switch tab next YT video it'll be too loud. So it for all the end users or only for my system. But your videos are very informative thanks for that.
When you see a video that says "Every Developer Should Know" and you look like John Snow, know nothing 😅
thank you very much !!!
How do you scale the load balancer? How do you prevent the load balancer from being a single point of failure?
You load balance the load balancer! Via DNS-based load balancing
you add another one and use Keepalived to link them both
@@sealoftime how to do this?
thanks 🎉❤
Hey, I'm new to System design, Can we add multiple load balancers in a complex system? if yes then how to map requests between them? Do we need another load balancer to manage those two?
Geo DNS may help you
ByteByteGo - please please share the tools and softwares you use to create these wonderful videos. It will be extremely helpful to learn them and use it for work and share knowledge in general. Thanks in advance!
Surely there has to be hybrid solutions to this problem. Some of these details in the "cons" category for these different methods feel like they could be easily overcome. #3 for example - why would an admin be required to actively monitor these values and adjust them? Surely that task could be automated with code.
The lowest latency method shouldn't require *that much* overhead, right?
If you have enough traffic that you're tracking multiple outstanding requests per server, then you can order each server by whichever is longest: the latency of the last fulfilled request or the elapsed time since the oldest outstanding request. That way, you (practically) don't have any overhead measuring latency. If you don't have enough traffic to accumulate multiple requests per server, then load-balancing shouldn't be a huge problem.
You can add more complex logic to minimize hot-spots and detect server failures, but the gist is still the same.
I think dynamic latency algorithm is complex because you should take in account each specific request.. example, a get request that fetchs a resource from a remote server should not have the same weight as a post request that create a resource with the use of a remote external api..
I've seen a lot of load balancing configurations in my time, but I don't think I have ever seen anyone use URL hashing. Wonder what scenario that is good for.
url hashing lends towards session affinity, or if one of your serving servers is in a different security tier/domain than the others. It lets you serve that secure content from a one or a subset of your fleet. It also shows up in microservices here and there.
Consistent hashing is widely used. Isn't it ?
is consistent hashing considered a load balancing algorithm?
It can be used as a part of it.
Is it possible one request share with multiple server using least connection and duplicate occurrence?
where it's mentioned in your book I mena which one?
What load balancing method is suitable for Backends such as Nodejs used for Live Streaming Applications?
*So the Backend is a live streaming application
Looks like there is an error on preview in 2. No Bob's requests on the right side, only on the left.
If you already have latency metrics, wouldn't least time be the "best" choice?
Is it better to avoid load balancers entirely and use message queues instead?
Different purposes. Message queues address communications (typically within) a system. Load balancers address network traffic.
If the load balancer is a single server with single IP address, how can it possibly handle so much traffic? I know it just forwards request but the IO overhead can add up, right?
Deberias replicar el servidor con alguna tecnologia como docker para tener mas y mas sevidores desplegados horizontalmente en contenedores cuando se alcance el limite asignado para cada uno. Si solo hacer depender a tu app de 1 servidor y este no esta manejado por alguna tecnologia de reemplazo de contenedores como kubernetes es muy posible que tu servidor falle cuando tu app vaya escalando :(
Anyone heard Prequal Load Balancer owned by Google? What algorithms it is used?
Do most devs using any cloud hosting even need to concern themselves with load balancing algos?
What about topo based algorithm
Hello everyone, I am working on load balancing algorithm in CloudSim 3.0.3 and main problem is deadline of task. If I have MIPS that can not execute this task before deadline, what I need to do? Can anyone help me?
Insert cookie with least connection
I studied queing theory in college. This is it. M N queues
what about least errors
The narrative here is powerful and impactful. A similar book I read was transformative in its reach. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
Me after watching a 20 minute video on consistent hashing and seeing none of it here :)
Thx ❤