How I scaled a website to 10 million users (web-servers & databases, high load, and performance)
HTML-код
- Опубликовано: 11 сен 2024
- Ex-Google Tech Lead Patrick Shyu talks about scalability, and how he grew a website to handle 10 million users (per month). I cover load balancing, content delivery networks, mysql query optimization, database master/slave replication, horizontal/vertical sharding, and more.
* Note, these experiences were from projects before I began working at Google, so I'm talking about my individual experiences (Google uses a ton more techniques, though the basic concepts are similar). I'm sure I missed some things, so please share in the comments below if you have thoughts on how to scale! I'd love to hear.
Join me in DeFi Pro and make passive income with crypto. defipro.dev/
Join ex-Google/ex-Facebook engineers for my coding interview training: techinterviewp...
💻 100+ Videos of programming interview problems explained: coderpro.com/
📷 Learn how to build a $1,000,000+ business on RUclips: youtubebackstag...
💻 Sign up for my FREE daily coding interview practice: dailyinterviewp...
🛒 All my computer/camera gear: www.amazon.com...
⌨️ My favorite keyboards: iqunix.store/te...
🎉 Party up:
/ techleadhd
/ techleadhd
Disclosure: Some links are affiliate links to products. I may receive a small commission for purchases made through these links.
#techlead
This is the first video from this guy that I watch where he actually sounds approachable and down to earth.
he's been trolling on those other videos where he seems 'unapproachable and cocky', it's just a skit.
This guy is weirdly addictive 😁😁
Lol cok dogru
Yeah he is!
haha! I agree with you!
I know nothing about programming, but there’s just something about a funny Asian guy that is hilarious to me
I know! I've been binging a lot of his videos. They're informative and hilarious at the same time.
This is better than a 5 how-to-scale-up-your-website articles combined. Easy to understand, practical, and to the point!
quality articles are good too .
Not that I'm anywhere close to needing to scale something like this but it's always inspiring to listen to you 👏
Your channel is also very good! +sub
A mistress is always needed.
Majj Majj - you mean concubine.
starcraft brood war...just trust me. techlead knows what im talkin about
I think you need to get in a line. I'm sure there are more people waiting in the queues. LOL
TechLead,
Can you revisit this topic? Maybe you've learned a better way since 2018 for scaling your product?
Totally agree.
Thanks for asking this.
But does he still code nowadays?
@@lychenus no, these days he just puts info into his servers with mind control, because of course he is the Tech Lead (as a millionaire). 🤣
Above concepts like caching , sharding , masters and slaves , CDN etc are related to system design. If you want to learn it in detail watch "Gaurav Sen" yt channel. Do let me know if you know someone
I have no clue about setting up websites, but I learned today how to setup a functioning system with slaves and masters. Thanks TechLead
I'm currently a performance engineer (used to be a software developer, most likely will go back to that soon). I'm currently working with techs such as OpenShift and the way we scale micro-services through OSE is with JMeter, SysDig, and Dynatrace. It's a combination of finding the right cpu/memory combo, along with optimizing the application code/config itself. Let's take a Java micro-service for example; gotta make sure the GC isn't going crazy, make sure you're actually using the right GC algorithm, making sure your multi threading isn't screwing up (thread pools, dead locks, etc.), doing some code review to ensure the algorithms used are as optimized as they can be, etc. Basically, just try to vertically scale the micro-service at first.
Then, it's just a matter of verifying that when you scale horizontally, you actually double your performance and that it scales correctly.
Finally, you can set the auto-scaler depending on your needs.
That's the way my colleagues and I currently work when scaling web apps in a micro-service architecture right now.
Nice.
This is pure gold. You are awesome!
Thalaiva ingayuma?
I have better option when compared to this
Thank you so much for making this. I really appreciate how you value simplicity over computational efficiency. I've always felt I'd rather spend a bit more to keep things simple and straight forward so in times of crisis the solutions are that much simpler and quicker to arrive at. The tips about database master/slave and the computed and cached queries is genius. I always feel a bit anxious when I write joins and am always curious whether or not it would be wiser to do two smaller queries or like you mentioned, have a cron job perform the join when server load is low (or on a slave) and then create a whole new table of the queried result.
Great information! I truly appreciate it.
Gonna memorise this for my interview
Please don't. Do it, and then speak from your experience.
Your description is pretty accurate except for the load balancing part. Nginx or HAProxy are solid choices for load balancing and are desirable in a scalable architecture. I have also been using a similar architecture and have been able to achieve pretty awesome results!
I also feel that the LB part was a bit off too. For LB, what matters the most is the network bandwidth that your data center is providing you with. If you are doing SSL termination, you will need a bit of CPU but that's all. You don't need disk space nor memory.
This is one of TechLead's best videos imo. I've done a bunch of server setups and full stack development and half of the stuff he mentioned never occured to me, like splitting users between databases based on a master user table. That being said I still think a load balancer is a good idea as it ultimately saves you time. Also Firebase or AWS is really a huge time saver, even if it does cost a bit more. For me personally the hours I spend trying to get all the configs and scripts right on a linux server and complaining all the dependencies from source with all the correct configs ends up taking so much extra time that the the price I spend on FIrebase or AWS is worth it. That being said if you want to go the configure it yourself route, this video is a treasure trove of really good ideas.
This is the content I was waiting for.....interested in learning system design as I am already 2 years into software development...
Thanks TechLead!!!
No, his name is The TechLead
U've just narrated the entire system design of some web-app i know.. THIS IS THE TechLead!
6:16 issue with connection ,9:21 master-slave db setup , 10:41 db sharding
Thanks 🙏🏽💎
To provide a more pragmatic comment...
For scaling a one man web application to 10 million users I would recommend cloud managed services as far as possible since you won't have the man power to keep your finger on all of the moving parts of your service. Using managed cloud services you don't have to manage servers and can save costs as well depending on how you configure your services.
For static content I would use Cloudfront to serve static files hosted in S3 (they work well together since they both from AWS so you can do e.g. origin access identities to limit S3 access to only Cloudfront so that nobody can access your static content directly.
Dynamic content can be managed with the app making requests to api gateway (serverless rest api managed service) with integration requests to either lambda (function as a service)+dynamodb (managed nosql db) if you want to go full serverless and not manage a single server. If you do want to manage your backend servers, api gateway can also integrate with an ALB (managed layer 7 load balancer) and have an autoscaling group that you can configure to automatically scale-in or scale-out your ec2 instances based on health checks done by the ALB. No need to script and stress. The ALB supports sticky sessions if you want to maintain a session to a backend instance, although this isn't necessary if you're using a stateless front-end app with token-based authentication (cognito to the rescue, another managed service).
As far as I know, there's nothing wrong with table joins inside a RDBMS as long as you're doing your database schema correctly, like doing correct data normalisation, correct indexing and you write your query joins to use those indexes. As far as I know, internal RDBMS caching and query engines will outperform anything you can write in a front-end or client side when it comes to joining data, especially if you prepare your statements, and even if you don't, modern RDBMS have features so advanced and so ingenious, you won't be able to outperform, unless you write your databases wrong and you don't take advantage of your advanced RDMBS.
Often at high scale you will denormalize data and join it on a web tier. Sometimes it is cheaper and simpler to scale the web tier vs. the database tier. I'm talking about when you have 100s of front end web-servers and 10s of database servers though. The database server will often be more efficient at the joins though (depending on which DB you use).
doesn't rdb cause not available issue where nosql is available all the time since nosql does not require indexing? so for some real time app, rdb may not be suitable.
@@GrindThisGame highly dependent on the requirements, I've worked with high traffic ecommerce site where that worked, then worked with travel site that tried to do something similar but the data model had so many dependencies that simple needs turned into heavy queries returning unnecessary data, a lot of glue logic to pull the required data together, bunch of bottle necks in querying the multiple APIs and lack of cascade rules lead to bunch of abandoned data and dangling IDs pointing to nowhere. all of that nightmare could've been avoided with foreign keys and one simple query. people should be really suspicious of any off the shelf magic bullets, for any significant challenge the solution architecture needs to match the problem.
@@GrindThisGame Agree. I load some queries to web tier with denormalize and caching techniques especially for heavy join tables or high volume queries. MSSQL index is helpful most of the time but MySql index may not be helpful all the time especially for complex join tables.
This guy has experience. That's for sure.
This sounds so much like a high level system design interview.
My 2 cents : You can use dynamic content on the CDN too. Most CDN's with their enormous infrastructure have direct routing to the origin servers. They bypass the traditional internet and the requests directly land on your server via the CDN. Plus you can use a global traffic management SAAS applications which most CDN provide to load balance based on geo.
wow , im a beginner jsut putting my first site out in a few weeks....this is a great summary of information..thank you
I wish he make more videos like these where he is actually sharing his knowledge, instead of those cocky videos he make nowadays
Nah, he doesn't need to do stuff like that because he is the Tech Lead (millionaire). 🤣 But seriously check out Hussein Nasser's channel as he goes into detail about all this stuff.
Can you make a video on can flipping optimizations?
am I the only one who uses this channel for drink reviews?
Possibly, if you can prove you do.
I suspect he has secret drink sponsors
I just start my first job after watching you for 2 years
To be honest , I am little lost but I am working hard to keep up
Dude I just found your channel and this is so helpful as a young app developer. Thank you!
This was the very first video of yours I ever watched. I never heard of you before, but I was on a kiteboarding trip driving home alone, and thumb searched for scaling AWS. In the first minute, I'm like - this guy sounds like he doesn't know anything lol... I proceeded to listen to your channel over mobile data connection for the next 6 hours of my trip. Wow awesome channel. Wish I had discovered it before!!! Just came out of a 10 year tech job.. happy to experience the down time and explore passions... like kiteboarding and salsa! Your channel is greatly inspiring.. sharing. And thanks!
I understand everything you say because I worked in a really good startup. Nice talk.
Reducing load on dB server using caching layers like memcache, redis, cassandra. Using symbolic links to remote mount file systems. Database sharding. These are treasures of info.
Incredible description, you explained everything just enough to understand the primary roles of each layer without getting too technical. These vids are great, keep it up!
So refreshing a tech lead video without sarcasm or bravado. More like this please.
12-factor app patterns are a great place to start if you plan to scale up an app
Our application is a billing solution for internet service providers.we are using kubernetes to scale operations on top of contabo.We use ceph for clustering block storage,percona for database group cluster and stored procedures for long running database queries.
Non sticky user sessions should be considered from beginning. Sticky sessions won't help in scaling and will definitely cost you in form of the best available infrastructure.
Yup!
from the outset i tell my engineers "Each time the web server sees the user request, its like the first request they ever make!" with the exception of client persistance like cookies.
How does that work with sessions? Lets say an ecommerce website where you store session data across requests
@@JamesSmith-cm7sg Store sessions in a RAM-based database like Redis?
These points come from his experience not from a book. That is the value of this video.
Session persistence is handled via sticky sessions in load balanced environments (from what I've learned), cross load balanced environment the user will usually be re-directed to the original machine the session was created on, FYI I am using Azure at the moment to manage all these resources. You can scale up a database server, or scale out for read operations via replication. But overall I would agree about trying to reduce the amount of database interaction, caching data that doesn't change often, as database IO is an expensive IO operation (and usually the bottle neck of most large web infrastructures), but overall trying to develop in a very efficient manner while knowing the various limitations of the tech stack is important.
Cookies and sessions arent really a problem if you use a stateless authentification with an RSA signed JWT instead. Just one thing to keep in mind if you want a develop a scalable app.
Something I do to avoid heavy database calculations and lag time is have Oracle jobs run and create materialized views based on other table data, and it pretty much just became a look up from there. Typically had some beast servers for my databases though.
wow, I have always conflicted view on concurrent write atomicity issue whenever I try the master-slave approach, but it's clearer now. Thanks for the upload, its really helpful.
Had to watch at 0.5x speed just to verify that your can actually flipped... So smooth lol
Shit, you made me watch it in 0.5x also luuuul.
I can't believe that I understood all of it...
I was worried that I have learnt nothing by debugging the server logs,and create random webApps...looks like I have learnt something 😀
This is the difference when you hear from a person who has actually done the stuff than those who just learn it online!
probably the most interesting video you put out. thank you patrick!
This guy gave a really clear explanation of this, thanks TechLead lol
Thanks Patrick for inspiring me alot I can now proudly say I have transformed alot from security to a junior database developer am now learning angularjs courtesy of you .
not a programmer, but entrepreneur, its good to know this with the outsourced help i require. now i can speak their language.
Wish I had discovered this channel way back.
Very glad I've been learning Amazon Web Services over the last 7+ months. I once had a medium scale project on dedicated servers a few years ago and that was a small nightmare to manage from time to time. AWS has solved almost all problems regarding scaling for me, highly suggest people learn it (or a similar cloud service).
Maybe for another video it would be good to talk about monitoring the stack/infrastructure (maybe about failover too?) and the difference between named & concurrent user loads? :-)
When I watch your videos I feel like Tech Lead and then I remember there can only be one TechLead, sigh
Elixir got a mention! Woot! Yeh, you know your favourite technology isn't popular when even a passing mention is a cause for excitement.
its very popular
2:50 It's called "engine x" and it's a web-server, popular for its ability to handle thousands of HTTP keep-alive connections using very little RAM.
Ngnix :)
We asked, he delivered! Thank you techlead!
Man, your videos lately are Killing it.
It is good to know fundamentals, but for the general purpose, just host is somewhere at cloud. AWS, Azure or Firebase will all take care it for you. You just need to use slide bars to scale it or let it auto scale. And cost wise you just pay as you go, much less than dedicated servers. And, you do not locked up in their system. They just run VMs with same code you write.
Yeah but he's old school so everything he talks about is a few years out of date..
12:00 he knows
hbb that's useless if you don't know how to take advantage of things like lambda, read replicas,efs,etc.
For short term, using Azure or Amazon would be great if you needed scale. But it is far more expensive than setting up servers yourself using Linode or your own hardware. It is very rare that you'll need the kind of scale that he talks about in the video. If you use Azure etc you still have to learn very specific things just on how to use their systems, - time that you could have spent setting up your own stuff and actually learning more about hosting that is more useful than learning to use azure/amazon.
Also, an important point he is not talking about is the use/purpose of your applications. Not all have busy database access all the time or big images to load. This is extremely important to consider when deciding how to scale. You need to consider how your app is used!! and even better; think about this when you make it ;)
It think his point is, when you start your website and put it on AWS or somehwhere else its fine, but once your website gets traction and going, then you kind of are stuck with the service you choose. because setting up the website on a dedicated server will require some downtime and a lot of additional work to get going instead of keep developing new features for your website.
Master-slva setup on MySQL is paramount for backups ... there's not other way to make real-time backups other then using master-slave server ... and also redundancy ...
About the keeping the DB query simple ..yup, that's KEY ! DB's are not made to make calculations ...
I was thinking the same, it s kind of defacto according to my prof at uni.
Wow this drink review went off on a wild technical tangent.
Nice tips!
I also try to minimize the amount of data that I am storing.
Data that is not stored cannot be hacked (and costs less to maintain).
That master slave idea is so good.
Liked your video :) You are funny as all get out! The databases we used usually would do the job - but then again - I worked at SAP - so - they tend to have a dispatcher that would send the processing (when the user logged in) to an Application Server (which pretty much acts as a type of Middle ware). Whenever systems became slow, one would just add another Application Server. Queries on the DB were simple (although, you would get the Query From Hell every so often). As you have said, if you have a lot of Memory in use, the DB will pretty much sit in it. I remember working on one project where the company had 30,000 users/employees - but - it was determined that there would be 2000 Simultaneous users for the system (so, scaling efforts took place with that number - 2000 - in mind).
One way they controlled the performance was to associate different applications with certain Application Servers. This way, the local data associated with that particular app could "cache" on that one App Server (ex; you would associate Human Resources on one App Server, Material Management on another App Server, etc.)
Maybe you can look at Oracle for your data partitioning needs. They had table partitioning which would allow one to partition large data over several tables (and not have all of it located in just one table). Something like UserId would be the Key.
The only time more than one DB would be used at a time was for backup purposes (Pepsi did this). The DB being used was Oracle - so - Offline Redo Logs would be copied (scp) to the machine that had the backup DB. The idea was that if the main DB got trashed, you could immediately switch over to the backup DB, turn it on and keep on going.
13 minutes of video for a year's worth of work :)
u also didn't mention about dns and direct connects. they play a huge part in reducing network latency, and api failovers. direct connect allows your edge dc server (where ur ui lives) to connect to your core dc servers (api process ,DB etc) in two hops. dns can auto route traffic to another dc if ur default dc goes down.
Wow, I was searching something that might help or give advice to me regarding on how to deploy a large scale system correctly and your video actually gave more than enough ideas on how to set it up properly.
Switched to use Google App Engine because my one server setup was no longer coping with the load. Yes, had to rewrite the DB layer, but that was a lot easier than doing all the stuff you talked about.
very nice! you have used the principles of microservices before they have been standardised
Cookies are stored on the client and they are per domain. Sessions can be stored in a distributed memory fashion (using Memcached) or on a database.
This has just given me so many insights of how to scale our product at work.
Hello TechLead, I'm watching your channel since long time, and this one is pure gold! Thank you for sharing your experience.
On my current project (as a team lead myself) I've decided to go with AWS Mobile hub, lambdas and DynamoDB.
DynamoDB does not feel right for my project, though, but I'm looking for I'm using netlify.com as hosting. I'm deploying front-end React.js App statically generated html/js/css build with `react-static` from production and staging branches on push from git.
For AWS lambda functions I'm using separate branch, and deploy functions to AWS with Claudia.js. Authentication goes smooth through Cognito - JWT. I've started the project before GraphQL was implemented in Mobile hub, and now I have 3 environments: development and staging on one aws account, and production on second aws account, and it is 16 tables for each environment.
The architecture issue I'm experiencing is tough. For each page reload I'm requesting user authentication and only after authentication, I'm requesting ~700 products for a shop and storing it to Redux with Immutable.js for pagination and fuzzy search. In same request I'm receiving 5+ full tables.
I feel like it is pretty long to request each item of a table iteratively, but another option called `Scan` is very fast (and costly $$$)
Can you advise, would a switch to GraphQL solve my issues? The thing that worries me is that browser performance could drop down if I will use GraphQL right in components, oposite to storing data in Redux with Immutable.js.
After I added indexes and a CDN my social network app got about 20x faster
Faster is not always scalable.
@@shanaya_saanvi But that is why we scale. So that our system will be able to handle heavier load and serve more users, so that it will not slow down. Not Slow Down means Go Faster.
Hi The Tech Lead, thank you for being the tech lead today for TechLead
I think you should create a separate playlist for reviewing those drinks. 😂
One of the best video I have seen. Very helpful
WOW! So you actually made the world a better place! That's AWESOME!
Most things you mentioned is exactly what I did a long time ago, when my local social network was growing very fast and had to serve my service to an incredible amount of users. I guess I would have told the same things in this video, good job :)
Love this video. TL describes a lot here
Good info! Definitely will use these tips in the future! You should do an episode on the Q&A from your subscribers.
It is okay to hit the database. In my understanding, "hitting database" mean that your query is not indexed and it has to go walk through rows and rows of data. If you have proper indices, it is simply just a lookup and you don't need in-memory caching server. Save you the headache of invalidate cache and scaling the cache servers. Database index IS caching.
Nginx is pronounced "Engine X"
does it really matter?
@@AloeusCapitalManagem yes, bc if you're teaching someone they should know how to look it up.
Enjinks is pronounced as enjinex
For a tech lead who can't pronounce nginx as commonly pronounced `Engine X`, that's kinda red flag to me.
You’re really good at organizing complex ideas into feedforward thoughts. Thank you. I love how you went deep into scalability. Can you do something similar explaining the difference between scalability and performance using in memory engine spotlights like tableau and qlik for business analysis? It’d also be cool to hear your thoughts on buzz word city which is happening around statistics. Can you be the mayor of Data science for a bit while also being the tech lead, tech lead.
About to deploy my first application. Thanks for this video!
You are an inspiration dude.
Can you do a physical series on what you just said. Set up on physical machines and the software to setup. I am trying to create a rack server setup dyi. 1 server for hosting file which the users interact with, which is connect to a 2nd server that holds the database , 3rd server for video files and 4th server for images file.
Ey, the endpoint configuration thing is a great idea to keep in mind!
Very nice... can you pls elaborate on cdn links, symbolic links for file systems...video is very informative
Simple and beautiful explanation. The only thing I found missing was the discussion on the upload speed/bandwidth of the server environment and what effect it can have on the user experience.
straight to the point, nice, thx!
Sounds like AWS Cloudfront, DynamoDB global tables, EC2 instances with EFS and ALB with auto scaling groups would work great for you.
Old school scaling. This is how many of us did it in the early 2000's. In my experience this starts to get extremely painful past single digit millions of active users per day. Definitely more cost effective than AWS or GCP if you know what you're doing, and good enough for most medium sized businesses.
In the days of the cloud and microservices people have mostly forgotten about this option. When compared to something like microservices, there's very few changes you have to make to your code to make it scalable in this way.
Asker: Tell me about yourself.
Me: My name is Techlead and I am Techlead.
Asker: (sudden death)
Instead of master slave, data shredding, I'd use Cassandra, I was surprised to hear you using Cassandra for cache , usually it's used for big data across different servers and data centers!
good advice - the webserver connections staying active due to inefficient database queries, ; also the timeouts can be an issue depending on how they are dropped and queued up creating a backlog that never gets cleared.
I am building a website. Thanks for the upload.
I am facing a similar situation and found your talk. You nicely vocalize some ideas which I was hatching for some time. Keep the server architecture simple, so that you can easily duplicate them for horizontal scaling. In very much agreement.
TECHLEAD THE TECHLEAD
Saurabh Chaudhary - yes, but who is leading the leaders ?
Awakened Sleeper techleadlead
Astonishing explanation.
In my experience, I have face a problem where we were running out of synchronization of data from database update to other microservices.
There were IBM queues used to deal with synchronization, however they were getting old and slow also the support was not so good. So, I hav decided to replacing IBM quese with kafka pub/sub.
This is so cool, but I'm glad that nowadays deploying an app is way easier with tools like Firebase.
Firebase is cool but from my understanding it is mostly for mobile and a lot of functionality is deprecated on web applications. (i haven't used it)
recently started watching your series -- great stuff, thanks. It is possible to initialize mysql slaves without shutdown of master, if master is small enough to make an impact-free mysqldump backup with single-transaction and master-data options. Log position and filename is then printed at top of dump file and one can load backup on new slave and configure replication to start at that log position on the master.
depends how long mysqldump takes, but it could take hours... if your db is small enough, then yeah that'd work. it's fine to also just take the site offline on a sunday night probably too while you do this maintenance...
thanks!
Your explanation helped me understand my assignment. Thank you so much for your sharing.
Can you do a video talking about how you and Management Information Systems people interact like in day to day office interactions what role does one play in Google or the companies that you have worked for? I like these kinds of videos by the way you're the best!!! Fan from the Philippines
awesome! im walking this path and my mindset is basically the same as yours
consistency is not aways necessary
db queries must be optimized/background scripts must be set in place
share the load the more you can