$400,000,000 Saved - NO MORE AWS
HTML-код
- Опубликовано: 30 сен 2024
- Recorded live on twitch, GET IN
/ theprimeagen
Original: tech.ahrefs.co...
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Im the devops manager at a ~5B company. I have a huge 'on prem' colo deployment of systems. I can confirm that AWS or other hyperscalar clouds run me about 10-12x the cost. Now you have to deal with having data center ops and paying those salaries etc. But even with all that its way more expensive.
All calculations I have done on cloud costs for customers is around the same. It's 10x ish.
I've worked for one of the largest Telcos in Australia. It is cheaper to run internally, but the problem is always the internal politics of who controls what. It would literally take months to get certain changes done that would be a sinch with AWS. This is one of the reasons why departments go to the cloud because they are sick of the politics.
@@dexterplameras3249 This is actually super true.
Inflation will not help at all in these calculations, we all know that by now.
You are doing something very wrong.
Here's a profitable company, managing their own servers for several years now, telling you that they would pay 11x for worse service in AWS, and would thus no longer be profitable.
Chat: But what if a hard drive fails? And what about labor costs?
trust the chat to actually know stuff
you can outsource just the storage to the cloud (aurora, bigquery etc).
@@raptyaxa5771 this is why you use a 5 year ammortization schedule and you sign contracts with Dell and a Storage vendor (NetApp, Pure, etc.). I would never have hardware without service agreements, what happens if you 1) dont have the expertise available or 2) you have a hardware failure. Service contracts are a must if you're running an enterprise size business. Before movign to the cloud we were paying about 300k to account for growth / obsoletion
Not to mention being a hostage and not being able to get your data back ever again.
@@Darth_Bateman didnt watch the video, but wait, is having your own server a thing again? Not the worse idea, but if your shit goes viral it'll go offline since it doesn't have a cluster to scale to..
One of the things not mentioned here is that there are other resources on AWS that cost money as well. EC2 and EBS is not the only thing. You probably will have some kind of firewall, virtual networks, backups, log ingestion, etc.
gateways prices are huge too
@@OtakuArtful Yup, those WAFs are a major cost point.
Nat, vpc, cloudwatch, waf it's not even close. Between public cloud and self host, always self host if you have the ability.
Develop single code, but ready for both cloud and your DC. Deploy in parallel both in the cloud and on your hardware. When your hardware cannot handle it(or DC burns to the ground), redirect traffic to cloud and it will autoscale. If big traffic becomes a new standard then increase hardware in your DC.
When you have too much money, you build your own cloud :)
just build your own cloud 5head
Build your own space company, change from cloud to space 5Head
I mean you could also build your own cloud with your computer and cloudflare tunnels.
IaaS (ah yes... The cloud)
DC = data center
back when netflix got popular it forced a lot of ISPs around the world to upgrade their stuff as big chunk of their customers started maxing out their series of pipes at the same time. never been their customer but I appreciate the positive growth pressure
And now those same ISPs are pushing for charging companies like Netflix for "traffic", nevermind who is actually requesting (customers).
I remember my ISP is blocking netflix for this reason lol
I don’t know how they did it but in my country where monthly usage is capped at a few hundred gigabytes (quota), only Netflix does not count into your monthly usage (offered by select ISPs) which basically makes it better than any other service
Just so you know he's a Netfix Engineer
series of TUBES
Cloud makes sense for businesses that require elastic availability, 100% on prem might not make sense then. Say ticketmaster has a consistent load of 50000 users per day, but now Taylor Swift is selling a new tour, it would make sense for them to scale into the cloud for that period to offset the abnormal load.
Cloud by default will always be more expensive, but amazing to scale only when needed.
only VPS in a cloud environment by default will always be more expensive, the fact I see only people that can't build a solution without a service VPS like, is the main problem...website of Taylor Swift to sell tickets can run and scale on AWS for few bucks per month, VPS not needed at all
If you take a 3 year reserved instance plan payed upfront it reduces the cost by 62%. Still massively cheaper with your own hardware, and once your paying for cloud upfront that obviates the entire advantage of cloud which is that you have month to month flexibility.
The conclusion is: Start in the iAss, when you grow you will know where to put the infrastructure in ;)
you mean IaaS?
Nah, prety clear, iAss
I am what about you guys? 😂
Instructions not clear. I ended up in prison.
In singapore right? right?
@@chudchadanstud roflmao
So what you didn't calculate in is:
* Pay for system admin team to manage all of this for 30 months
* Pay for developer time for all of the tools and things you will need to develop now that you don't have them ready
* No multi-az since you are only renting one space
* Equipment breakdowns due to fauilt / overuse etc.
* Backup strategies you will need to develop. You ARE planning on backups right?
* Since you are renting, I am guessing you are renting the space and the connection so who is doing the infrastructure costs of the building like data center security, building upkeep, upgrades, etc?
* If you ever need to go global what will do to keep the response times low?
I am not saying AWS is cheap. What I am saying is that you have a LOT more to consider than just rent and electricity.
(disclaimer, I'm a dev at ahrefs)
1. the devops team is more or less the same size when using aws
2. This one is partially problematic sometimes, but for our problems it's more the other way around in general, AWS is lacking some tools. Nevertheless it's indeed can be a real limitation. And here usually the cost isn't so much the salary of the devs but the time required to build a solution.
3. yep that's correct, if you need multi AZ the computation would be different, but the computation for AWS would also be different, as you would need to duplicate the storage to different zones. So the AWS cost would also drastically increase.
4. equipment breakdown is part of the price of the hardware
5. backups are mentioned in the article
6. The people the rent is going to (this question is exactly the same for AWS, who is paying to keep the AWS building in working order?)
7. we are global, but this isn't really a relevant question here, see point 3
So overall there's not much additional costs so long as we don't depend on a product that is only available on AWS.
There is no big difference when comparing team sizes. You still need specialised people managing AWS.
Still cheaper
Also If you have any compliance reqs cloud is much easier. And all of the bespoke shit you write is more technical debt. Then five years go by and you have to re train ops people on your bespoke shit. Not fun.
@@louisroche9574 "1. the devops team is more or less the same size when using aws"
So the devops team size didn't change taking on more critical responsibilities , but did everyones pay go up to match.
Sounds like a big mgmt win.
I wonder why in the recent years the tech industry is making dumb choice after dumb choice, it's like engineering has gone completely out of the window.
so much abstraction
I've been conditioned to think that buying vs renting/leasing is more cost effective over the long term. Think real estate and autos. I remember when AWS cloud was taking off and having this twinge of misgivings about the costs. Plus the fact that you are beholden to the owners of the cloud. I agree with everyone who says there needs to be a balance. The ability to be able to spin up hundred/thousands of servers at a moment's notice is a great convenience but once your infrastructure is more mature, the cost savings are too great for standalone.
That's always was the supposed argument of all arguments for cloud - "you only pay per hour for what you use, so you can't multiply that cost by whole month". It always seemed silly to me because, yeah, sure, if your server ran for just a couple of hours every day it might make sense. But in reality, most of the time that's not what happens. You need to have some instances that just run all the time, even simply because the demand won't drop below some level. And it's not like you can just nuke all your databases or delete all your data from the cloud every day because it's "not in use". Ahref's case is especially striking, because the cloud cost overhead is so high that even if 90% of their servers were never needed, they still are better off not paying for AWS.
i love the datacenter being an api.
If you have a large enough scale for it to be worth it, open stack or canonical MAAS can give you API driven self hosted hardware :)
@@KadenCartwrightI’ve only worked for companies with
For anything I've ever stood up, it's always been substantially cheaper to do it on our own hardware in a co-lo than putting it in any cloud, especially if we want anywhere near the performance we have today on our own hardware. None of this is a surprise to me.
With a beginning of russian war against Ukraine, out major banks moved to cloud in EU from on-premise in UA. One of them (monobank) moved to AWS and they also got significant increase of bill for infrastructure. This was mentioned by their CEO Oleg Gorokhovsky. He also was questioning himself why orgs tend to use clouds.
They should try Hetzner
I'm not a cloud expert, but I am a little surprised that competition between cloud providers hasn't seemed to reduce costs much over the last five years or more. Probably one aspect of it is that the differences in usage specifics between cloud providers means that once a typical company has started investing in one cloud provider, there's an aspect of lock-in at play that prevents easy transitioning from one to the next.
This is due to venture capital. You're not dealing with profitable companies competing with each other. You're dealing with massive, MASSIVE amounts of venture debt from companies that have no road to profitability in other areas of their businesses. And since it's rather expensive to get started, it's very difficult to bootstrap a cloud company. If you don't have the venture capital, you can't afford the servers. If you get the venture capital, you'll never be profitable enough to lower your prices.
(I'm working on a cloud platform with some buddies and it will always be both competitive and profitable as long as we never pay ourselves salaries - it would require several *thousand* servers before the cumulative profit-per-server could pay us competitive wages)
I'm a cloud expert instead: VPS are expensive, the key to lower the costs in a cloud environment like AWS is to use their native cloud services, avoiding any service VPS like: EC2, ECS, EKS, etc. this guy was unable to do, posting that just showed his lack of knowledge, TCO can be easily lower than on prem, but you can't improvise, you have to be prepared
Competition reducing prices is economic propaganda
To be fair, if your idea of using AWS is just paying for dedicated EC2 instances, you’re doing it wrong. You need to architect your applications intelligently to keep your costs down.
And get 100% locked in to AWS?
Yep, lambdas, kube and etc, but al that depends on the load you have and the codebase.
RUST BTW
@@lhxperimental Not necessarily. There are many AWS and Serverless providers. There’s tons of ways to roll databases. There’s many ways to do compute and single sign on and such.
Not everything has to be in AWS and really you can architect your applications to appropriately fit cloud services in some places and monoliths on dedicated hardware for others.
@@lhxperimentalthats why you have a exit startegy from the cloud, or use terraform if you're sticking to cloud stuff to move as easily as possible.
You actually choose whether you get locked in AWS.
AWS is locked in WITH ME >:)
@@lhxperimentaldon’t use your phone send us letters with a parrot so google doesn’t lock you in with RUclips. You obviously didn’t touch a server in the past 10 years if you think a company that large can rely on ec2 instances
What I love about cloud is that it lets me, an individual, make stuff and deploy it for cheap-to-free without having to worry about maintaining additional hardware. It's kinda the same deal when it comes to installing internal applications at smaller companies (or smaller teams/orgs within companies). When scale comes into play, the waters get muddied. There's a ton of tradeoffs to consider, and how much money you pour into either on-prem or cloud infrastructure is going to determine how big those tradeoffs are and in what direction. Not every company will be better off one way or the other, it's about analyzing your individual situation and picking the option that's best for you.
I think one thing we have to keep in mind when looking at these example companies who're doing these migrations is, that they're pretty special cases - be it massive scale web scraping (ahrefs) or video streaming (amazon prime video). __This is not what most of us are doing/serving__. Most devs here, me included, probably do standard web development - maybe with some SSR. But this will not lead to comparable load profiles in respect to user count. Most of Ahrefs capacities most likely go to 24/7 max load scraping, not to serving customer requests.
So that is even with reserved instances.
1 TB of RAM sounds ridiculous to me anyway. And I am an actual Java developer, so quite used to stuff eating up ram like no other.
And I've done self hosting in colo in the early 2000s. For some reason management never seemed account for replacing cost every 3/4/5 years and trying to wring lots of 'free' time.
A lot of ML stuff can use a lot of ram. BTW a lot of RAM is also very useful for storage caching. Idk how necessary it is when you can have a 16x NVMe RAID 10 tho.
The amount of vector logic in AI/ML nowadays will make you crazy. Literally Python code jerking off of each other and produce pretty image or cosplaying as a human will cost you more than just 1TB of RAM.
Everyone defending AWS in here pretending like you also don't have to hire a TEAM of platform engineers to keep your AWS infrastructure going. 🤣 I don't believe there is ANY DIFFERENCE in labor costs between managing a large cloud and some on prem infrastructure.
it blows my mind that they just brushed over capacity planning and autoscaling as if they need to pay for Ec2 instances that handle their full capacity 100% of the time
That is very true actually, but i assume their load is reasonably high based on their on-prem capacity already.
AWS is expensive if you don't know how to use it correctly
@@matthiasg4843and nobody knows even Amazon
Great point but to be fair since it was 10x more expensive you'd need to run at
Fair point but when you're dealing with very large instances like they would be you can't rely on AWS having enough capacity when and where you want them.
The TCO in AWS could be easily less than on prem, obviously, only if people know how to use a cloud environment, in this case the #1 of cloud providers: AWS.
If you use services VPS like or worse Kubernetes, inside a public cloud environment, you are NOT using the cloud, you just made a migration "lift and shift" of your infrastructure, the fastest and the expensive way to migrate, this is not the way a Cloud Architect approach a migration, this is the way as an elder System Administrator think.
If he does't want to learn how AWS work and how to optimize the infrastructure cost, is better that he stay far away from any public cloud, instead of complain of something he doesn't know how to use at all is nonsense, I clearly see an unskilled (in cloud environment) guy that complain and cry like a 5yo kid.
I think the key advantage of IaaS is not flexibility in capabilities you invest in, but simply the much, much shorter timeframe of investment. You can double your infrastructure today, and cut it down to a quater tomorrow. If you did that with hardware, the decision you made today would mean paying extra 200% capabilities for the next 60 months, on top of the fact you’d be stuck with the original excessive 100% you would have even without todays fuck-up.
The fact you couldn’t even do such a choice because the timeframe from decision to actual capability is so much longer. If you buy “off the shelf”, you might have a bare supervisor running in less than a month - provided you only bought few machines, your network configuration didn’t need reworking, your UPS had a vacant spot, the server had memory pre-installed and the storage was delivered before the server or in the same shipment.
it’s like buying a car versus renting it. You can think about renting a car for ten years, and conclude that it’s ridiculously expensive. But at the flip side, you can’t buy a car for a weekend - that’s not how buying a car works to begin with.
Good luck trying to get over ~10,000 instances in a single region spun up within a small time frame. Even on a corporate account, you need to plead a case for a need for that many instances which goes through a ticketing process that can take days to weeks which is still dependent on if the resources are even available in the region at the time, which in the case they're not your request is simply denied.
Then take Netflix who was able to host the entirety of the US eastern client base off a single rack and is able to spin up hundreds of thousands of instances whenever they want at a fixed rate.
@@callowaysutton
About 10k instances, it's actually pretty edge case, probably that number even more than any cloud provider have in a day worldwide. And it's actually hard to find a task for this 10k instances, so yeah they want to know will they be paid for this, and for which task it needed, because it can create a shortage for other clients.
About netflix served from single rack, depends on which timeframe you refer, because earlier they used Akamai, and now they use their OpenConnect rack for ISPs. So it's almost never was a single rack.
Per each 1gbit they can serve approximately 400 active streams(average 1080p stream ~512KB/s with some fluctuations), if we assume that some pause video + they ok with some small almost unnoticable delays, then it can be approx. 1000 users of decent content provisioning. From requirements of OpenConnect they can connect it max to single 100gbit, So 1 rack can serve max 100k users with ideal conditions of course, and it's only content serve without site logic/analytics/etc..
UPD: even less than 100k, I forgot to include IN traffic, because it's one same NIC for IN/OUT.
Is it real that Netflix has a single rack for an entire region?
@@Sergey92zp Here's the thing though, I could spin up 10,000 instances very easily on any medium sized $100-500k privately hosted cluster and not even be close to reaching the total capacity, so it being an edge case isn't and shouldn't be used as an excuse for an IaaS type of commodity. For reference, spinning up 10,000 of AWS' smallest EC2 instance in US East costs a whopping 100k+/year, and for that you get 10k shared 2 core, 512MB of RAM instances... or about 20,000 vCPUs and 512GB of RAM. Let's break this down further; it's a common industry practice to split each physical CPU thread 32 ways to each vCPU so that gives us 625 actual threads. The newest Epyc servers come in configurations of 512 threads per server (you could even have a 4 way multinode in 2U for a total of 2,048 threads in 2-4U worth of space) and 512GB of RAM is frankly nothing considering these types of servers go into the terabytes of RAM. If we discuss space and networking, a single 42-47U rack with good 100Gbps can be had for ~6000/m now, or ~72k/y, and considering AWS doesn't ever let a single customer go above 100Gbps per zone it's a good frame of reference. If we were to amortize over a 3 year period that gives us a 150k budget to compete with AWS and with that kind of money, you could easily afford multiple tens of terabytes of RAM, thousands of cores and much faster disks.
For what you're saying about Netflix, you're just completely off. I'd recommend looking at their many presentations about their infrastructure and how they've saved money by using their own hardware/network.
iAss
Idk, the difference between the two scenarios is so insane that it could mean that allocating your own hardware will always be cheaper than any approach on AWS
The upfront costs are very high though so like stated it's not feasible until you have enough cashflow that it doesn't hurt the company to make those upfront investments. Pay as you go is pretty much always more expensive in the end but you don't have to have the money now. Like buying a house for example.
This is why aws is Amazon's money printing venture
Not an apples to apples comparison by any means. AWS is about services and resilience not servers and power costs. If you don't need the services or the resilience AWS provides (think auto-scaling multi-region services in minutes) then there's no point even thinking going with Cloud.
If you want a reliable, scaleable service with backups, I think this is the sort of thing you think about when your AWS costs go above the cost of, say, 4 skilled full-time engineers. If you're spending more than a million, you can bother thinking about it.
You don’t need 4 full-time engineers to keep up few servers. I’d argue cloud is more labour intensive, because poor resource usage costs so much.
On the other hand if you have ridiculously oversized on-prem setup, it can survive multiple disk-failures, hw-failure and still work. Yes you want to fix it ASAP, but the staff you need to do so is not notably expensive compared to managing cloud.
You will lose few nines of reliability compared to amazon. But just because amazon has five nines doesn’t mean your product on AWS has. In either case the likeliest failure point is the same: your own stupid ass or asses
Even if you use AWS; you still have to have to spend time and energy handling backups. I'm not sure why people use "backups" and "reliable" as key benefits for Cloud. Cloud isn't inherently more reliable than computers; since that's what it's based on and it's not abstracted *that* much.
Plus you will have less access to the underline system if you need it. It’s all about balance and ownership etc
Did Parler have reliable, scaleable service with backups. when they got cut off, they could not use their own database because they trusted.
@@w300x how would you build multi region redundant backups on prem?
For those wondering about Reserved Instances: they used 3-year Reserved Instances to calculate the price.
Bottom-line, and how I personally look at this: have an off-cloud strategy. The cloud is great at a very early and/or explosive period (either customer growth, or new infrastructure), but you should plan to staff up and have a cloud exit strategy while the cloud is still advantageous.
But when you need to deploy globally for example because streaming, on premise doesn't look that good
@@neociber24 Exactly! I actually pushed to move our workloads from on-prem to cloud last year. We have all our workloads on the cloud, and the reason was two-fold: speed up moving to a new paradigm of development and deployment more quickly (aka implement DevOps, as a strategy), and for geographical access (we have customers in 3 different continents). But we're already working on our new on-premises infrastructure, first to move our development and testing workloads, but ultimately, to become another DC in our global infrastructure. End game is having our pipelines generate cost analysis so we can choose where to deploy different workloads, aware of the benefits (and costs) of each option.
Hybrid
I disagree. You're planning to hire an entire staff to deal with ordering hardware, configuring VM management, mega-licensing costs from someone like VMWare... All of these things that have *nothing* to do with your business and you have to hire a whole team to do it. Even if you end up saving some money (factoring in staff costs makes me question that you actually would), the management overhead there is huge, NO THANKS! I'll gladly outsource it to cloud services
@@themartdog You're assuming that's not a consideration. You're also assuming we don't have the trained staff to do so. Both those considerations are not true, but YMMV. I was just sharing my experience, not inferring knowledge of everyone's infrastructure decisions! ;)
PS: you're also assuming that every company's needs fit into something that is "workable" in a cloud setting. For instance, egress and overall bandwidth costs are not a consideration for most people. In our case, egress alone was 30% of our bill, and we already found better solutions, both more cost-effective as well as more manageable for our teams. o/
@ThePrimeTime I am surprise that they didn't take into account that when you plan to have a server on AWS for 1 or 3 years there is a HUGE discount for reserved instances which lowers the price -70% down... even more if you use a mix of reserved instances and spot instances (which are -90% discounted). They can save money going to the cloud and serverless (dynamodb) + lambda + s3 can also be considered for super fast retrieval and serve of data instead of using EBS volumes.
Maybe they had some issues with AWS and pepper sprayed this article to them 🤣😂
They did factor longevity. per the article, they used the price of a 3yr reservation.
As someone that is supposedly a software architect, but always ends up having to do Ops on the side, this ignores the biggest cost point in all this: IT and developer time.
We're not just talking about making sure the power stays on. We're talking about automatic backups, configured with a few clicks, autoscaling easily, and starting up and tearing down developer instances in seconds. Suddenly, your IT, or worse, your developers have to do all that.
If you're already big enough to afford 40 million in servers, chances are you have a big enough IT department already. My personal experience though is that the IT department is where every single CFO tries to save money, and it fucks EVERYONE over.
With things like OpenStack, OpenShift, OpenNebula, VMWare, Canonical MAAS, heck even Proxmox that is quite literally not an issue. Almost all platforms have a way of autoscaling, backups, high availability, hardware issue alerts and more features that used to need a lot of specialized knowledge. Now, a single person, or a small three to four person team, could potentially manage the actual hardware of a rack or whole DC location and it would still not even make a dent in the difference between collocated and cloud costs assuming salaries in the low 6 figures.
In Proxmox for example, you could take a physical server down for maintenance and have it live migrate to another node with no noticeable affect to the client. All platforms have at least this feature and more; you could even migrate across two different DCs/locations if you really wanted to, which you can't do in AWS, GCP or Azure yet.
Canonical MAAS literally manages the hardware for you and with its hooks you could potentially even have it automatically auto order parts that failed tests, safely turn off the server, turn on the iBMC LED (to show remote hands which server is having issues), submit a ticket to the DC with the shipping number and have maintenance effectively automated.
The IT landscape today is not the same as 2008 and the large cloud providers are definitely starting to show their age big time
What do you think of companies where they have people dedicated to managing the infrastructure so it doesn't bleed into developer responsibilities?
You need those engineers with AWS or on-prem anyways. So the cost comparison is fair.
@@Victoria-ij3cb I definitely think, if the company is big enough, on-prem is can be nice, but only if your feature set (and therefore requirements) are basically static. Otherwise, what would a single click in AWS to try out (e.g.) K8S turns into MONTHS of waiting, and then getting a faulty, buggy on-prem K8S.
Another BIG disadvantage I've observed in pretty much every bigger company is that IT is understaffed, underfunded, overworked and overregulated. Things that would be a single click on the cloud can take months.
@@invinciblemode Not even remotely true, at least in my experience. For example, my personal, very painful experience with on-prem K8S: We didn't have an expert, so someone just had to google shit and hope something useful happened. Obviously, it took forever.
Then at least 5 different features that are a click away in the cloud, even for the dumbest developer, were missing. Things like a basic load balancer, an API gateway, a replicated database, or a decent storage solution. And again, you'd need experts for all those things.
With a cloud K8S solution, all that just magically happens, and is maintained, automatically.
Even if you only use the cloud for as dumb VMs, you're saving a lot on people that need to setup, install, and maintain your racks, and the software (hypervisor, etc.) running on it.
I guess I just have PTSD, because EVERY SINGLE IT department I've ever come into contact with has been underfunded and overworked, making them decidedly far worse than the cloud, at least from a developer perspective. Maybe in heaven they have a wonderfully running on-prem cloud.
The Cloud just means someone’s server. MS and AWS now retain the same hardware for up to 7 years now. You are being had!
I never noticed the little light up border on the like button that happened when you had your early video breakdown... It only does it during that outburst too
I think they made some mistakes in their calculations. Normally when you host such a system on AWS you don't compare your Servers directly because your instances run in Auto-Scaling Groups and your system scales up and down. This in the calculation means that all the server are 100% loaded at 24 hours of the day. But normally when you have times of a day where no one is on your website the system will scale down and delete your instances. So i'm not sure if that calculation is really fair. And there are some other opinions like Spot Instances. But yes AWS is expensive.
And what another user mentioned is that you need Engineers and Backup strategies and so on. But i think when they scale very intelligent on the load i think AWS would be much cheaper then the calculation mentioned.
they specifically choose to reserve instance, means they literally rent the logical space for the server in the DC. And also, considering the HW specs they buy for the on-prem servers, I don't think spot instances (or non-HA instances) would be adequate for their use case.
0:38 - 0:43. You better press that like button.
High core-count CPUs, 2TB of RAM and 2x 100GBPS bandwidth servers and 80 TB of storaege.
Finally something that can handle "npm install"
Microsoft infrastructure as a service. MIaas 😅
thank you for your videos! I really did find joy in engineering again since discovering your channel🙂
i like how prime randomly gets autism attack's and then just shruggs it off as: 'anyway' and continues on reading 😆
iAss
16x15 is super easy math for 8 bit programmers from 80s.. because 16x16 is 256 of course, so 16x15 is 256 - 16 = 240.
It's apparent Prime is lot younger and haven't been doing 8bit assembly... ts ts... :) ;)
Dont know much about the cloud but the host is entertaining
Honestly me neither, Primeagen is just fun to watch. Then again you probably won't have any real knowledge about the cloud unless you work at a company that uses them or are just into knowing about cloud infrastructure
What is the successful cloud start in my opinion? You need to be a "nomad digital angel"... What do I mean? You need to be able to jump from clouds to clouds easily like an angel. So you need to architect your infrastructure in a way that allows seamless cloud switching without any significant troubles. Use only the services that are based on popular open-source projects. Even if they are a bit harder to use. Every proprietary service will be yet another heavy anchor tied to your leg. So when you start your project cloud helps you to reduce the initial costs. When you are a bit bigger you can switch to the cheaper cloud and hire some infrastructure specialists. When you are big you can build your own cloud and move there. Success...
aka containerization
Agree 100%. And yes, containerization is the key.
Statement that at certain scale having your own infrastructure ia chepaer is not a magic or new one. It is known for decades. The problem is that your own DC will never produce a fraction of easily consumable cloud services than AWS, Azure or Google.
Exactly, when you start is better use a cloud service. When you pass a certain scale of course you can own your own infrastructure to reduce cost.
@@oscarljimenez5717 Yes and no. You have to keep that in mind during development or at least during planning. For example set up your clusters to rely as little as possible on proprietary technology and use your own instead. For example configure your own load balancers inside your k8s clusters so you can just move the cluster to another location easily instead of relying on proprietary stuff (like AWS's and Azure's load balancers).
Moving to the cloud is not just creating ec2 instances lmao, there's a lot of architecture you need to change
for the overwhelming majority of saas businesses it is. You can run a pretty profitable business just with a lambda function and an api gateway.
may offer an alternative term for "unknown known", you probably meant bias :)
They use OCaml and D? Should rewrite it in Rust and save 1/2 of the servers..
they can write it in assembly and save 1/4 the servers... Ocaml and D can be compiled down to machine code.
Let's say you decide to leave the cloud. Do you need to replicate the "services" that you had before. Is there an opensource equivalent of google cloud or aws that you can host yourself? I'm far from being a cloud engineer but I was trying firebase but decided to go with supabase. Is there something similar with aws?
AWS does give you the ability to run some of their services outside AWS, yes. They also have AWS Outposts so you can set up your own AWS "partition" in your own datacenter that can have EC2, S3, etc. all through the normal API.
If you are so dependent on AWS, your tech stack was built with lot of omissions. Pull up your socks and build on stack that you can own
@@lhxperimental That is exactly what I'm asking. How to develop without tying the code to an specific platform. I've heard of openstack and terraform but I know little of them, are they even related to this?
I mean in your example supabase is a "Foss" (I don't think their dashboard is FOSS) "alternative" (doesn't really have the same set of functionalities ) to firebase but their hosting platform isn't open source at all, that's what you actually pay for, the managed hosting they provide.
Or maybe you mean "is it possible to self host dynamo db (or whatever)?" ?
@@heroe1486 Yep, possible to self host
But they’re not considering the cost of personnel for setting up, maintaining and securing the on prim servers right?
ya
Cloud: its cheap to get into when you're small, but once you've ditched your technical expertise, can you rebuild a DC from scratch? Do you need "cloud scale" Do you even need "cloud scaling" or will HAProxy be fine? Get your IT provisioning done by cookie cutter.
That's like saying that you saved 20 Billion by not buying a Nuclear Powered Aircraft carrier. Not buying something isn't a saving.
AWS has discounts up to 70%
Still 2x more expensive for worse hardware.
It's a trap!
@@Ravengenoyup buying dedicated servers is way way cheaper.
@@fritzstauffacher6931 but you have to pay people to run them. those are _not_ cheap. And their processes aren't as mature. Your SRE team isn't born with runbooks. I can stand up an arbitrarily large database cluster with multiregion HA and point in time recovery in like 3 mins clicking around. And it actually works. The hardware isn't the point at all.
still muy caro
Owning your own servers is great. I’m not sure what they are doing, but their cloud build is a straw-man. Firstly they are using servers with peak capabilities. I doubt they are maxing their RAM, CPU, and network AT ALL TIMES. You can save quite a lot by dropping one or more of the specs. They are mainly scraping, not sure if they have variable load, but if you don’t have time critical tasks you can use spot instances, and often save 90%. If you do have static loads, you can use RIs to save a lot of money. The $440M figure is just asinine. Plus they have zero extra staffing costs for managing many of the things AWS manages. I would expect AWS to cost 50% more than they could manage, but with better redundancy, uptime, responsiveness and standardisation. But if their business cases are built like this, they will obviously never consider it. But with numbers this stupid, you’d be a moron to take them at face value.
2:50 How can’t you do 16×15 in your head? It’s 16×16−16. Every programmer should know that 16×16 = 256 and 256−16, well, do I really have to tell you it’s 240?
16x15 gigs = 250 gb = 1/4 petabyte per server. RAID mirroring gives you half of that, so 120GB usable space.
Amazon doesn’t get free computers, only cheap ones. Then you have utilities + profit margin + Bezos yacht fee = the same or higher cost as buying a better server yourself.
Man, I don't wanna be the devil's advocate for AWS, but the author is missing important information.
Those AWS costs are fake. When running on-premises, you need hardware that supports peak time workload. When running on the cloud, you need just enough for the current demand.
On the other side. Premises serves are not run by magic gnomes.
They are run by really skilled and expensive professionals.
Expensive professionals are not accounted on their calculation.
When we use the Cloud. We are not paying only for storage and computer power but especially for professionals who make it run smoothly.
I'm not saying that using the Cloud is always the correct answer, but the reality is far from what was shown there.
Cloud only really works for mid to large businesses, small and enterprise are better off on prem.
Obviously Cloud is more expensive.
buuuut, I guess that if you email AWS sayin that you wanna move your gigawatz of RAM, kazillion of iper pentazillion byte to the Cloud...yeah, maybe you can obtain some discount 😂
They're not wrong, they're also not quite right either.
Doing a direct lift and shift to AWS when your using really powerful kit, will only lead to pain (and extreme cost) as outlined here.
Also you can get NVMe drives on AWS, so using EBS as a comparison is weird. Like when would you compare server + local storage to server + "SAN"
In reality to make the cloud affordable, you need to architect your stuff in such a way as to take advantage of its strengths. Otherwise they will fuck you on cost. N.B. It probably isn't going to be cheaper even still, but it's not just about cost.
Being able to respond to changes rapidly is where you get a lot of value out of AWS and the like.
EBS is insanely slow and expensive, S3 would have been better from personal exp.... I've eliminated that too by storing files as data objects in my database... This helps with overall complexity, the expense is 20-30% larger files so just db mem. Let files not be editable, so users just have to upload replacements and cache hard when something is requested
Look. You can find many ways to pick at this data.
It's entirely possible It's someone cherry picking the information and cost comparisons to make in-house look a lot better.
But by the same token the "cloud is best" mantra is something the cloud computing companies have put around to boost sales.
I would always say companies need to sit down at regular intervals. Cost out all their needs, then decide on how they structure their IT systems accordingly at various points in business growth.
For example, a small start-up may be better off with services in the cloud, as they can get started quicker and don't have to have much in-house expertise and infrastructure.
Similarly, extremely large multinational companies may benefit from cloud services, as it saves them spending massive amounts on purchasing and maintaining hardware and software infrastructures.
But it's the middle ground where cloud services are a far murkier proposition.
Even the company saying in-house is better for the admit they use AWS for certain things. Probably because it's easier than developing that stuff in-house.
😂😂😂😂 all true. I am from VMware working as a SRE in SDDC operation. Managing own data center is the cloud choas and very very very complex and costly.
Prime over here making things up on the fly. “Unknown knowns” isn’t a thing. You either know you know something, or you don’t know what you don’t what you don’t know. You can’t “unknow” something you know you know. That would be transitioning from an “unknown unknown” to a “known unknown.”
Feel me? Not necessarily making things up, but illogical.
The time when services like Amazon make financial sense is also the time when you can run your web app from a 300 Euro Laptop, connected to another 300 Euro Laptop for backup 🤷♂
Honestly, aws is overpriced af. But the overhead you save for running your cloud architecture is so wild is not even an option for the vast majority of the companies. If you can afford it and more important, MANTAIN IT sure, definetly go for it.
There must be some crossover point between AWS and on-prem in terms of cost vs performance. Clearly these guys need performance beyond that threshold. I'm certain there are plenty of people using AWS that fall below that threshold of performance requirement.
I guess that's why there are people that get paid a bunch to work out the cost effective strategy for business infrastructure like this.
Cloud is good for temporary things. Slight surcharge of traffic during holidays, proof of concept, testing a new product.
If cloud didn't lock you in so much, a hybrid approach would be much more effective. You could rely on "the cloud" for usage spikes, or for scalability. Say you suddenly needed 100 more servers, you could order them, and then use 100 cloud servers until your servers arrive and get set up
That's not how bozzo gets a new yacht
HPE is a good example of the hybrid cloud approach
Nitro Unjs
AWS recently has been pushing for more hybrid technologies. Services they have been promoting lately help facilitate that. Better to capture part of a business than none at all
Spikeloads are literally the #1 best usecase for AWS.
You guys miss the main point. That's why you are not the CTO or CIO. The main point is outsourcing risks. When nothing happens and everything is rosie, it is fine, but anything goes wrong, however much money you help saved, you will be first to go.
Erm you don't put all your storage in EBS. Looks like they don't understand cloud. Also now accounting for commissioning and mantainance of servers.
I'm 41, call me old school but I'm at an expert at supporting on prem customers. I work for 15 large jails across the country and these are County entities who never have any money to do anything so there's no way they would go to the cloud.
Unknown knowns is basically how i live my life
its simple, once your aws costs exceed 4 infra engineers, you need to go colocation with your own systems. by then, if you've optimized all your code for the aws cloud, you're not going to be able to convert and need to go bankrupt.
What the f**k. At the begging of the video Prime is saying hello to youtube and at the same time encouraging to press the like button. My like button for this video glows UP to press it. Not gonna lie, I got a bit spooked
What about all the support costs for all your hardware and how many people it takes to support that hardware??? The people costs would be enormous when you factor in benefits. I don't believe it would be that big of a Delta.
ahrefs isn't even hosting, clearly an article to bait and switch. A bad marketing one.
AWS is certainly the 800 lb gorilla but I would be interested to see similar cost breakdowns for Google Cloud or Azure.
a company like ahrefs should have vertically integrated infrastructure anyway... unless im missing something, totally not the use case for cloud services
as someone who came to development from SEO, I love the way your pronounce A-H-Refs.
Please include the article links in video description.
IaaS is pronounced "yaasssss" as in "yaassss queen slaaayy" or the Greek name Ιάσιος.
15TB NVMe server drivers? Now that's sound like bullshit for me.
Even just our App Insights bill is astronomical. imagine having to pay for logging.
DIY is always going to be cheaper ofcourse but like everything else DIY, can you do it right and can you be arsed to do it lol
Watched on the RUclips app on Android. Saw the like button light up when you said to like the video early in the video.
Plugged nose...change the filters in the house...maybe...shot in the dark.
Hey Prime, have you realized that the like ratio is usually 10%? it's pretty steady across all channels content, a few go above 15% but I haven´t seen a higher ratio, there should be some anomalies anyone could share examples???
The prime values that a cloud provider delivers are :-
1. avoid upfront cost of infra and talent ownership.
2. provide flexibility to scale up / down as necessary.
3. allow distributing workloads across multiple regions / zones.
4. reduces complexity by providing an API over infra and services to allow composing your implementations rather than deal with the underlying implementations.
In case of on-prem deployments, considering that they're multi-regions, it's usually still a single zone per region. You could have hardware level redundancies but how do you auto-swap? How do you deal with ISP failures, weather, sabotage, etc. The problem is most people don't know what worms does the on-prem can has.
If the on-prem infra is the physical filesystem, a cloud provider is the logical filesystem over top of it. Techniques like IaC (Infrastructure as Code) basically allow you to programmatically / declaratively not only spawn infra but also allow you to observe it, scale it up / down and even protect you from major security threats.
Does all of this come at a premium? Obviously, DUH! It's not a best effort basis deployment which most on-prem deployments are. There's usually an SLA and a very high one at that.
Also, almost no one discusses the talent cost, local regulatory costs, technical debt that accrues and slows everything down gradually? And lastly, what do you do when there's a sudden spike, let's say 300% on Black Friday and then you don't receive such traffic again for the whole year. What do you do? Over provision 5x? What about the unutilised capacity available for the rest of the year?
If a company doesn't need most of the features mentioned above and has extremely predictable and mostly static loads, sure, the premium for the cloud provider might not make sense. The most probable reason why Netflix sticks to AWS is for the ability to seriously scale up / down in seconds and for AWS's expertise is maintaining a behemoth such as itself.
Finally, scraping the internet is going to be expensive. Should've thought about that before building a business around it. LOL!
EC2 doesn't provide most of those benefits - it's just a server rental. There are plenty of cheaper places to rent servers.
@@thewhitefalcon8539 who said it needs to be EC2. Though, I'm sure if a company that serves 10 - 15% of the internet, Netflix, is fine with using EC2s then there's some value proposition that really works well.
SLA for EC2 is only 99.9 (3 nines) and 95.0 (1.5 nines) for 10 and 25% discounts respectively (most of AWS is similar, you don't get 5 nines from Amazon)
If you can't accomplish 95.0 (18.25 days of downtime per year, more than 24 hours per month), you're doing something truly wacky
In regards to staffing, someone from ahrefs stopped by the comments (and other people backed them up here), the staff cost for devops is basically the same either way
I don't find the dismissive comment at the end helpful at all. They aren't complaining, they're sharing their information. It seems strange and derisive to punctuate with 'By the way, what did you think was gonna happen, boneheads? LOL' if the information is meant to be conveyed compellingly or convincingly
Exactly, their use case is completely different than EC2’s value proposition.
I’m not saying that doing yourself isn’t possible, I’m saying if your business isn’t around managing infra, you shouldn’t be mostly doing it yourself unless it makes business sense to do so.
Ahrefs business revolves around scraping the internet and greedily too. I’m not privy about their approach to scraping but from what I understand they’re simply putting in compute to get as much content as they can.
Considering the rate at which data is being generated and made available online, this is only going to become more expensive or time consuming.
This is a scale and optimisation problem and throwing compute at it is not going to make it easier.
Sounded like Ahrefs was doing a lift and shift comparison with just infrastructure. It would be interesting to have digged into their architecture to see if a cloud native architecture could have fitted better.
And vendor lock them to AWS for ever? Or spend more than Prime Video which ditched their "cloud native" approach?
@@meletisflevarakis40 I would not read too much in to the Prime Video malarkey, it did make good headlines 😉 For me, main takeaway was it reminded us there are good and bad architectures, and just because it's serverless does not mean it's good. I.e, if you are hammering Event Bridge (as was the case with Prime Video), then you will have a bad (expensive) architecture.
@@kazdaman1 vendor locking is a huge problem that very few seem to even consider nowadays
@@aaronhamburg4428 IT's a "future aaron" problem. You deal with it as you approach the issue. Until then, don't waste your time - focus on product and business development.
@@aaronhamburg4428it’s like people just lose their memories every five years and all knowledge of previous bad behavior from vendors who abuse lock-in tactics is wiped from the face of the earth
Jesus Christ, ads every few minutes on prime's videos now... Not great
KISS applies to devops too. Do you really need to have servers in Singapore if you aren't localizing your app for oceanic countries? IMO scaling only causes scaling problems so my philosophy has been just don't scale and accept the money you have.
Frustrating to see zip_zinger_ in the chat pointing out that "I bet their servers are idle 80% of the time" as if this is somehow a good argument for paying 10x+ times more for AWS and being able to make them out because you can't buy enough hardware. He's basically saying you can buy over 10 times the hardware or less and even then have it be idling comfortably and have a massive burst buffer while more than halving your infra spend. Ridiculous how amazing the mental gymnastics of people experiencing this kind of cognitive dissonance can get. The notion that some random ass backseater on Twitch is going to know enough about this company and their infra to have any arguments against this type of analysis is also ridiculous.
Those beefy idling servers have a plethora of highly paid staff and other systems supporting them so while you're paying the cooling and administrative costs of unused beefiness, the cloud person is paying for exactly what they use. Say you have a huge down turn in business, now you're stuck at your peek cost of beef while the cloud person can just downgrade services. You gotta fire people and sell hardware and weasel your way out of building contracts...
People have completely bought into the cloud proponents' argument and regurgitating the same. They have no experience to actually evaluate the costs.
@@lhxperimental and... I see no rebuttal to that argument here so...
@@kevinb1594 is there not some sort of hybrid solution here? Maybe use the cloud to be elastic but for normal load it's run on your own hardware? I love the phrase "peak cost of beef" 😂
@@kevinb1594 Managing servers is not as hard as the cloud providers like us to believe. It is easy to sell the fear of complexity of on Prem. Every one used to do it till about 15 years ago. Now with Kubernetes, it is easier to do than ever before. Could providers want us to overestimate the TCO. Also there is a range of options here. There is public could on one end (AWS, Azure..) to VM and block storage providers (OVH, Hetzner, Linode) to Colo to bespoke deals with datacenters to everything on-prem on the opposite end. Even if you move one or two steps away from public cloud, you will save a lot of money and still not have to deal with internet connectivity, power backup, datacenter real estate, etc. Public cloud providers would like us to become technical pussies afraid of doing even the smallest thing overselves. Prime example is lambda. They want us to pay for a function call! It is as wasteful as ordering 10 grams packs of iorn ore via FedEx for your steel plant.
Hybrid is the way. Some stuff makes sense in the cloud. Some doesn't.
Why do they compare own servers with AWS on-premise instead of with AWS reserved?
Does ahrefs do anything other than run a shitty search engine bot? I block them whenever I see them scraping a website.
Thanks for pointing out serverless costs ;) ruclips.net/video/XAbX62m4fhI/видео.html
AWS has been caught selling the shovels
Breaking news: Dell acquired by Amazon
iAss