Why I Stopped Self-Hosting
HTML-код
- Опубликовано: 7 авг 2024
- 🚀 neetcode.io/ - A better way to prepare for Coding Interviews
Machine Learning Problems: neetcode.io/practice?subpage=...
🧑💼 LinkedIn: / navdeep-singh-3aaa14161
🐦 Twitter: / neetcode1
0:00 - Why I stopped self-hosting
5:00 - Why another company started self-hosting
9:23 - Why Netflix went all in on AWS
⭐ BLIND-75 PLAYLIST: • Two Sum - Leetcode 1 -...
#neetcode #leetcode #python
This sounds more like managed vs unmanaged services
Haven't ever heard somebody call VM from public provider as self-hosting, since you literally aren't self-hosting.
He’s talking about VPS not VM. I.e. a Virtual Private Server as a fractional part of the CPU, RAM, Drive etc which you need to self set-up to get to the point to host anything. A Virtual Machine on the other hand is essentially “just” software. It can be just an operating system (so you still need to set-up and manage things to be able to host in production) or open the door to incrementally handing off more hosting responsibilities i.e. via containers. It’s a general effort gradient compared to bare metal cost and self management and not is or isn’t. Which is also the reason why this video, without more details on what was actually done, isn’t so useful. I.e. one would nowadays rarely self-host a website. But processing video files can easily make sense for dedicated hardware just doing that.
@@dinoscheidt VPS and VM doesn't make (almost) any difference from operating point of view, if it's hosted by someone else. And certainly makes absolutely no difference for the point of this video.
What I'm questioning, is calling a server, VM or otherwise, hosted by someone else, as "self-hosted". As it literally is not. It's someone else buying the hardware and handling the upkeep. Which is important difference compared to actually self-hosted, where at minimum, you own the hardware, and preferably the physical environment the hardware is ran in.
@@jaketusIn the literal sense of the word, you’re right, but here „self-hosted” usually means rolling out using a VPS as opposed to individual cloud services.
@@regnam503 I'd argue that no, it doesn't usually mean VPS. And especially, when we're talking in the context of it-companies the size of Netflix, self-hosted obviously means actual Netflix owned hardware.
And contradictory to this video, Netflix isn't 100% cloud. They have their own hardware points-of-presence in ISP data centers as cache, so that they don't have to pay the huge data transfer costs of AWS. And that's actual self-hosting.
@@regnam503it literally doesn’t. VPS is not self hosted.
I think an important point is that Netflix isn’t using AWS for a majority of their video content being transferred to users. From older posts they were putting things closer to the user by working with ISPs directly to help out the congestion issues.
That’s what edge computing does
That is called CDN
They may be using S3 as the master storage and then they have servers machines at ISPs all over the place that probably get their files from that central S3 data store. Linus Tech Tips has their own multi-petabyte storage but LTT only has 2PB and Netflix is using 8PB. S3 isn't that absurdly expensive and using it includes all the labor and hardware costs ensuring your data isn't lost or offline. I think LTT required about a million worth of hardware for their 2PB machine and Netflix is getting 8PB reliably for 1 million per year.
It's compute that costs a fortune in the cloud.
I personally think the title should be "Why I stopped self-managing my cloud hosting" since in both of your case it is hosted on the cloud, just managed differently. The points you made I mostly agree with though.
why not use docker containerised deployments with github actions???
still doesn't solve his issue of the time consumption of management. i think the main realization he is having is a business mindset vs a software engineer.
My understanding of the scaling of DHH's company is that it's a B2B company where signing large contracts take time and it's easy to plan for more compute power for new large clients, and small clients joining doesn't make much of a difference from the server's point of view.
It sounds like you probably set up the server with quite a bit of technical debt, which is understandable given the short time it took you to set it up initially (those 8 hours). You could have taken the opportunity to learn how to properly set up the server so that subsequent changes would be less significant. Anyway, what I'm getting at is that this doesn't seem to be a problem with self-hosting per se, but rather with inexperience.
Frequent changes and redeployments are time consuming, and a maintenance burden regardless of how long they take in my opinion. Why would anyone ever want to manually do something that can be handled for them? Why would I ever wanna self-host my database? Personally, I would never choose to do that.
If a problem can be solved by throwing a reasonable amount of money add it, that's always preferred than using dev hours for it imo.
Yeah I agree. If you want to trade money instead of learning how to do something yourself, that is totally fine and normal though. Example being as a game dev, you could learn how to produce your own art or outsource that part and focus/learn on the parts you are already skilled at.
I think it is a matter of prioritization and deciding what you think actually benefits you in the long term.
@@NeetCodeIO I get your point. Not everyone has the time to spend on things they'd rather have someone else do.
However, I think your reasoning also discredits those who are into these. Why go through the trouble of self-hosting? Because it's fun to tackle a challenge, and not everyone has the money to throw at the problem.
If we let ourselves slide down the slippery slope, we end up with: Why even do all these in the first place? Why not be a couch potato? It's certainly easier.
@@NeetCodeIOif you had a data intensive application, self hosting your database might be the only solution otherwise it would be too expensive.
Most polite way to say "skill issues"
Clickbait title, I don't even think you can call this self-hosting since you don't even host the things, they are hosted on cloud although you manage them.
how is this self hosting? he is hosting in cloud, not on premise.
??
Not mutually exclusive.
You host a service yourSELF vs using a managed solution.
Just use docker for development, and deploy in docker to production. No hassle, no problems
Idk about no problems. Docker for local dev can be kinda slow when ur waiting for things to rebuild and such. That’s been my experience anyways
@@multivitamin7 why would you be rebuilding? you only need to rebuild when changing ur Dockerfile/compose and pretty much never else. Look into devcontainers for greater convenience developing in docker
@@multivitamin7 You can have a development image that, for example, in JavaScript that you run with hot reloading (e.g. npm run dev instead of npm run build within the container). Then the cost of using Docker becomes just the initial container startup, which is often under a second.
@@kevgoeswoof I've seen this work pretty well, but it depends some on what your stack is like.
Are you really self-hosting if you're hosting it on the cloud tho?
I'm old enough to remember working at companies that didn't use the cloud. Holy fuck was it annoying waiting to get a literal box installed in your datacenter somewhere in the EU. That alone is worth the extra money.
you could do a hybrid where you develop with flexibility in mind. have cloud ready for peak, the usual bandwidth is handled with self-hosting
What the fuck that this take man... there are plenty of tools to do continous delivery in your VM (i don't know why this guy thinks that a VM in the cloud is not part of the cloud????) that you just didn't use. And the guy of the article said him self many times, that the cloud is good to rent, but not to base your business around it, because it will drain you from your profits (also, I am seeing that you don't know who that guy is or what app he has, that allowed him to do this change, but hey, who am I to tell you anything???). And finally, Netflix is not "full in AWS", like the guy in the article that you skipped through, they use AWS for services that they don't want to replicate.
i wonder how much do you pay for your firebase? is billing okay compared to deploying own backend?
Not everything is about unhealthy hyper-growth. I don't think DHH here was trying to compare themselves to Slack/MS Teams levels of scale + complexity with this move or telling that they can do a better job than those folks by not doing cloud.
Such a great video, pretty much my thoughts as well
Ben Davis spotted
Maybe there is a way to self host with less time commitment? Maybe something like dokku/caprover/coolify would help?
Interesting video! Like many things in engineering the answer of whether or not to self host is dependent on what the end goal is and which trade-offs your willing to make. I think DHHs post and your video are great example of what this thought process looks like in action vs using a technology/stack/approach/whatever because the cool kids (netflix, dhh, google, amazon, etc...) are using/doing it.
I used this video as a monologue in the background. Helped me focus. 😊
Whaaat! You used this guy's rambling techno babble soapbox rant for background sounds? - Nice!👍 I'm going to play it at my job site, all his yapping played incessantly on a repeating loop, mixed with soothing classical piano music, like Chopin's Nocturnes. The people at my job site may not have one clue what he's pontificating about but they're gonna love it anyways, lol! We're in this friendly competition as to who can provide the most "interesting" (if not annoying) background music -Lol!😁👍
Well if your website isn't making any money and you don't have any users it doesn't make sense to pay money for a cloud provider. I was just wasting like $70 a month on azure bills for the cheapest tier DB and Windows server .. i'd like to self host but would need to spend $100 to my internet provider for a static IP and haven't had the time to look into work arounds
I think you misinterpreted term VPS and self-hosting. I have own home-lab with self-hosted software. Probably you are talking about VPS -> cloud. It will take some time till you came back when you learn that all those companies do the same but wraps everything in UI.
This is more like managed vs unmanaged, both are 'in the cloud', one just has all the knobs and tunables done for you. I'm a sysadmin not a dev so I feel exactly the opposite because when stuff breaks on the backend I'd be one of the people troubleshooting and fixing it anyway but I can understand your perspective as well. I've worked with companies that have moved their infrastructure to the cloud and others who after reaching a certain size have either moved stuff on prem or lease colo space and it's all situational. In the end if you can get your work done faster and easier no harm no foul.
I think you make several assumptions, that are not always true:
1. Linear scale for infrastructure maintenance time cost, it's not always the case. I'll argue that in some cases, it'll plateau and can be not that big.
2. That engineers are productive every waking hour. Well, I want to believe that, but practice shows otherwise, so I won't necessary count maintenance time as a loss.
But as always in engineering - it depends, and should be evaluated on a case by case basis.
okay the analogy with ppl building their pcs is a weird one. if you want to get yourself a pc you usually build one yourself as prebuilt options are just meh, unless you have the buck to get a custom one.
its pride and also u get to learn wt ur code is actually running on more than just at the surface level
I swear, you are the Dr. K of Software Engineering, and I am here for it!
You have to realize that US engineers are extremely overpaid. An hour of your time would be much less worth outside the US.
The cost of living is also usually less elsewhere, so the difference is often a wash.
@@wisdomlounge4452 It is, but not to that extent.
"I needed to install those python dépendances, that took a couple hours"
Bruh, you're using someone else's tool and you aren't using their docker image?
I have to self host jenkins at work and god i hate it so much, i wish we just used a normal managed CI instead
Why is there to hate self hosting Jenkins?
@@deeprajofficial1729 to quote their own docs, it's not cloud native, it uses the file system for storing all it's data, a pain in the ass to backup, furthermore, they have this convoluted plug in system that turns a simple update into dependency hell.
only jr dev face this issue. i never spend more than hours setting up my self hosted just for 1 item.
More like sales person trying to make small thing big.
David seems to care a lot about being independent and self-owned, growth is not necessarily required for that. Once you embrace this mindset, his behavior actually makes sense.
phonepe selfhosts most things on a huge insane dc, i work here smh managing is cheaper at the cost of my mental
Good
I thought self hosting was on your own pc, not a rented one
Massively agree on this video. Product ITSELF is the most important part. Cost savings comes next.
I love this channel but at just 1:08 mark, the entirety of this video loses the point for me. We other's have mentioned you stopped self managing your 'cloud 'resources.
Skill issues all around. And you weren't even self-hosting to begin with.
VPS ain't self-hosting.
U r speaking 1.5x 😂
i think it is important for you to disclose whether you are being paid by Google talking about this topic, since Google have Cloud division.
$0
skill issue
Today's daily challenge please...
You really need a bit more humility when you present such arguments. Self hosting isn't that bad of an idea.
When this blows up please like this comment 🎉
P.S:awesome stuff your doing neetcodeIO