Level1 Presents: THE FORBIDDEN ROUTER PART III - THE FINAL CHAPTER
HTML-код
- Опубликовано: 11 сен 2024
- The Forbidden Router is complete, but what vessel could possibly contain it?
Follow The Series!:
+ Level1Mini Series: The Forbidden Router Trilogy • Level1 Mini Series: Th...
**********************************
Check us out online at the following places:
linktr.ee/leve...
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-------------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons...
3:36 - "How much NVMe storage does your router have?"
"What do you mean..."
....sufficient (?)
😂 Red Shirt Jeff would probably say 'never enough'
"Yes" :)
Love Level1Techs. Even if I can't afford some of the more expensive goodies there is always good knowledge to be had that is usable on even my budget "home-lab" setup. Thank you again Wendell and team.
Still cheaper than renting a router from your ISP
Totally overkill and totally bananas. Love it! 😎❤️
People : This is madness.
Wendell : THIS IS L1T!
Or his classic "it's fine".
@@guy_autordie Sorta-kinda-maybe...
Normal people, would just. Buy an off the shelf router.
Lunatics, and/or tech savvies as others call em, would make a pc, the router. I freaking love this.
I'm beginning to think Wendell has an NVMe addiction...
New veiwer been hooked for a month, I love the whole cast everyone of you!
I'm not making a router like this, but I have just built an Epyc 7452, SM H11SSL-i, 256GB 3200, 4x PM963, 8x 4tb HDD, and soon a couple of video cards in a rack mounted SM tower (SC743) for cloud gaming :) It's a do everything box.
I do want to also build a low power box that can run essential services and run dual virtualised routers in failover mode. I have a test R720 SFF (it doesn't matter if I break it), but that sucks a bit more power and is a bit louder than I'd like.
Very nice system, definitely more than your average router :)
Finally! Any kind of information on occulink! Wendel I would love if you could spend even 5 minutes sharing your knowledge on adapter cards, cables, and accessories. I took a look at the IcyDock nvme products and just could not find any information on how to use them.
definetly!!!! the only reason i have not purposed the 8xNVme drive cage is i have no idea how to connect that to PCIe adapter card etc aka how do i connect nvme pcie to a sas card or what the hell i need to do
Here I am feeling clever about shoving two 2.5 sata ssds in the optical drive slot of an optiplex sff! Interesting point on the sata ssd passthrough for Pfsense. That's an interesting way to approach redundancy in the system and I might do the same in my build. Great video!
Nice! Should do a video on home GPU infrastructure… multi-user gaming, emulation, transcoding, and machine learning in one box.
Keep up the xcpng coverage. It works really well, once you configure it properly. Configuring it properly is not as straightforward as it cold be, however.
You can tell Wendell's got a few ssd's lying around :-P
Been drooling over one of those motherboards for a while now, still too rich for my blood, and would have a hard time figuring out what to even do with it, but still beautiful piece of hardware!
Surprised there wasn't a rosewill 4u case involved, those Sliger case's do look really good though, especially the 4u ones.
How come you have a CNC machine in the building?
Hey Wendell, I'd like to see some more HomeAssistant style videos if that's something that you use in your home to automate things. I've been getting really into it lately and I've seen some trends that might make it compelling content for Level1Techs 😁 Edit: Especially how powerful ESPHome can be, I've already found some uses in my own situation.
You know that they say, 'when in Epyc Rome'
Definitely. I wish these mobos were under 500, lol! I want a 7402p so bad...but I want to build an 8core Milan routers, tho; I've been thinking about it for 3 or 4mo...these videos read my mind, this looks like fun
@@jeremymoon9088 I'm looking forward to an Epyc chip also, probably with HP FX900 M.2s. There's so much I want to do with it.
All PCIe lanes lead to Rome.
Someone needs to make a short (ish ) rackmount case that supports water-cooling.
Sliger cases will do that fine if you use ie the 120 mm ek square pump res combo
@@Level1Techs 360 rad ?
@13:57, dang I have not seen an Antec 900 case in a long time
I reckon you're right on the money with what people are wanting to do with home computing in 2022.
ICY DOCK really needs some competition, what they're charging for relatively simple products is insane.
Wendell is a treasure trove of tech know-how.
ABV_Normie been real MVP in the forum
How much power from the wall does it pull during idle? I mean like working pfsense and whatever you have there, but just not hitting it with work, throughput, caching etc.
With 500W max I'd not be surprised if it's around 100W doing basically nothing, and that's quite alot.
Would be interesting to see how little idle power draw could a similar concept of a VM-router-cache-small-home-nas-all-in-one-solution pull, using either low power AM4 cpu, or even a used laptop motherboard (like the concept of those little cases for used framework laptop motherboards that you replaced), as in theory such a setup should be capable of really low idle power draw, which would be a really nice bonus along quiet operation (of course normal people don't have access to reasonably priced half-life enterprise gear like those ssd's, but still would be interesting), as in theory, a mobile board without a screen should draw easily below 10Watts from the wall at idle, likely even below 5W, while the ssd's at idle, according to specs of some commercial nvme drives, should pull as little as 0,1W each at idle, or even less.
Also, as a matter of fact I did deploy similar solution in a company (around 50 people), over 2 years ago, with one storage server, one backup server, and 2 xcp-ng servers as nodes in one pool, both connected to the same storage server for provisioning shared filesystems to the network, and on this xcp-ng pool I configured a pfsense router that replaced a 15 year old shit pc with literal spider webs inside, and multiple other vm's for the company needs, that also usually enabled throwing away some "golden" pc's in the server room that were dedicated to either a accounting software, or some special logistics software, or some 15 year old rack machine with an email server on it, and so on. Basically, it's not ideal, but with the backup server it's possible to recover vm's on the second xcnp-ng node (as local disks are still local, as launching them over network storage provided very bad performance), and for any maintenance it's super easy to migrate them to one XP-ng server, and so on. Basically this is a great setup and It costs very reasonable money (although as I remember there was a lot of issues with configuring switching for all the nics in the servers, and clients, and in the end xcp-ng got a small separate switch for all of it's local networks (including storage net), so that's also something to consider.
A full enterprise grade, fully redundant solution would likely cost at least 10x more (as that was done for around 6k $ if i remember correctly (using some cheap new dell workstation pc's for the servers, and getting 8tb of shared storage, 8tb on the backup server, and 2tb for local storage per xcp-ng server, and also some sata ssd's on them for more demanding use cases).
Basically with that setup the company finally entered 21st century, with:
- the infrastructure easily kept up to date,
- easy new vm provisioning,
- and with that also ease of trying out replacement of old software services with new opensource based ones (thanks to the easy access to new vm's),
- very little or even zero downtime for most maintenance tasks on the infrastructure,
- ease to expand the system capacity (by adding servers to xcp-ng pool, or putting a better storage server in there),
- in the end, much easier network configuration, as much more of it is done remotely via the pfsense vm, or the switch console, as "plugging in" new vm's does not require physical presence in the server room.
- much easier UPS infrastructure, as there aren't as many machines in the server room as there used to be.
You can only do a Gamers Nexus upgrade if they decide to remake the cold open from their last server upgrade.
We all know you only travel by telecom rack!
Just watched this entire series, and it's so interesting to me to see that Wendell's done the same thing now that I did a few years ago, except he has the budget to do it real nice :P
I still rackmount a lot of my stuff in order to be able to tuck it away easier. Using commercial 2u and 4u short depth cases allows me to build relatively quiet units. Ive got a 2u with an Athlon 200ge, a 4u with a Xeon 2680 v4, and a 2u with a Ryzen 2400ge. Im about to roll out some homeassistant to remove the devious claws of the megacorporations from my smart home appliances.
MWUHAHAHAHAHAHHA! I enjoyed watching this ALMOST as much as you did making it... with the glee of a six year old amped on sugar :-)
Can the ridiculousness get more ridiculous?
Yes. Yes it can.
And that's why I keep coming back here.
I just bought a few Super micro servers for my work. They were a similar model to that Atom but with an embedded Epyc CPU. Freaking amazing if you can find them. 8 core 16 thread at 2.5ghz, super quiet for a tiny 1u server, about as loud as a desk fan at idle maybe a touch quieter, even under full load they are very tolerable. Deploying these as Untangle boxes. Super excited. Ran a quick Cinebench on them. A touch more powerful than a 7700k, but consumes about half the power.
Aw man... It looks like one of the Blue Man Group guys sneezed on the m.2 drive 😆
Wow those P4500 drives are still pretty expensive for used drives. Used SSD pricing is outrageous. Anything used smaller than 2TB and possibly even 2TB = $50 per TB cap for me. Unless it's full SLC or Optane.
Would you consider doing another one pc many gamers/devs theme? I can only imagine having two of these servers set up with one compute and one with storage using fiber to connect them.
I had to get the 4U version to fit my GPU (RX580......). It's a media center in my livingroom. Nice case. Just a bit pricey, especially when you include 3 noctua fans........ It's nice to have a good looking old school case though. Oh, my motherboard was too old for one of the front USB connectors......
I added a Blueray drive in the front.
My only gripe, was the power supply hole in the back. It was borderline with a corsair RM850x. The plastic connector for the power cable was really close to the metal of the case. It fit though.
For those on something less than a new car budget, instead of adding more NICs, why not just hardware partition the two already onboard into VFs? (SR-IOV)
Theyre intel x550 onboard, so they support all the slick new offload features, so 64 hardware passthrough devices per phys. port, and since they're all on the same chipset, no need for any additional hardware (pcie cards, cables, hugely expensive switches) - the chipset acts as the switch, handling all the "work" that would typically be done by either a switch (if 1 NIC per VM) or cpu cycles by using a bridge.
Saves you the $ on the extra hardware, with the added benefit of freeing up those pcie lanes for more cool crap!
This couldn't have been released at a more opportune time! I'm moving soon and want to build a 25GbE fiber home network, and it will need a forbidden router... I'll re-use my old 5950X for it and this video series has given me lots of good pointers
he said .. " I've got more of amd motherboards and epyc processors" ... that's the music I wana hear to my broke ass ears when I'm drooling over the exact machine I'm dreaming about for more than 2 years :)
Hellz yea! Just checked 2hrs ago and no new videos...Wooooh, and I've been waiting for this one!
("Woooh" n Wendell's voice)
Wendell for president!
Will Wendell go crazy with the forbidden management switch
Nice build!
One day I might be able to move up from 11th Generation Dell and spinning rust!
best case for any home server is a closet in the garage...
I've been virtualizing my firewalls since smoothwall was a thing. I'm not entirely sure why it would be a bad thing, or FORBIDDEN. Just need to make sure you set things up correctly. I even use my Synology now as a hypervisor for my OPNSense installation. Does it work? Works awesome. Does it cause problems. Indeed it does, but it's just quality of life issues I'm having and the performance is adequate for a 1Gb internet connection, but 2.5 would not work very well, as no SR-IOV support on my setup.
Anyways it's only temporary untilI receive my R86S-N.
I think the Forbidden Router might have driven Wendell slightly more insane than he already was.
So much more expensive stuff than my poor little homelab. I'm still on SATA platter drives for my bulk storage arrays and Gigabit ethernet, running on a salvaged Opteron box. I'm looking forward to when I can upgrade to 10 gigabit. :p I'm still using an AMD E350 passive motherboard for my pfSense router, with a DC-DC power supply.
When it’s time to upgrade maybe look into the Aruba s2500. 24-48(poe/nonpoe) ports,4 sfp+ 10 gig ports, about $100 on eBay. IMO it’s a great switch. I don’t NEED 10g ports but the cards I’ve got in my nas and proxmox server were like $20-$25 so I figured why not. Plus it guarantees there’s no network bottlenecks for anything I do.
As much as I love this content I would really like to see a ground up build of an inexpensive PFSense router. I’m building a server right now just for proxmox but I don’t want PFSense on it. I’m going to use an 2400g ryzen and a i350-t4 card to build a PFSense box. It’s hardware I already had and I don’t want to waste. The case choice is what’s driving me insane, I don’t want a huge case for it.
Wendell, what NICs are you using? My biggest struggle with OPNsense/PFsense was finding a nic that plays nice with FreeBSD.
Stupid question, but on the SUpermicro H12SSL MB where is the Chassis Power and LED Header located? I do not seem to be able to find it in the diagram based on their naming
I would like to see someone pull off watercooling in a home server rack. The few videos I found on RUclips involved heavily modifying a server rack case (cough Linus cough) and so is out of reach for us plebs. The idea being to see if one can DIY a quiet (aka wife approved factor) home server rack build
4:29 Imagine this had only 4 occulink at 1/2 the price, but it split them out to all 8 ports....... so you'd have full bandwidth to 8 friggn 120GB optane drives
Did Intel really never fix that internal clock issue with a microcode update?
I really want to see an all purpose build!
@5:00 I bought one of those - still can't get it to see more than 1 SSD :-(
Any issues with doing this with true nas scale instead?
Just out of interest is there a reason for using pci passthrough for the NIC's for pfsense as apposed to just using a bridge? I'm already running a system like this using Proxmox with pfsense, truenas and many other services. I was assuming that passthrough might be used to reduce overhead for performance reasons. However I'm just using a bridge with a 10G card and couldn't see a notable difference in performance between a vm with passthrough and one with a bridge. So went with a bridge to still allow for vm migration and less setup headaches that can sometimes come with hardware passthrough. All the best
forbRIDDEN router!
For the algo
same
You look like you lost a lot of weight since this video was taken! Congrats!
am so jelus !!! the file server i have at my home /studio , has only 4 1tb mecanical drives in raird 10 , one 128ssd boot drive , one 250 ssd work drive one 3 tb for movies / music for plex , and 1 tb for plex tv show conected to the rest of the network with quad 1gb nic , and one gtx 960 to help rendering , and run on i5 4570
ME: I virtualized the router!
Family :(
As a reminder, the less expensive version of this router build was promised and is anticipated! 😄
First
the Asus HyperCard fan is annoying as hell. I yeeted it out of there as it is useless with 2 consumer grade NVME anyway
Oh really? What bother you about it?
There are at least 3 variants of the ASUS Hyper M.2 card. You've got to be careful with which card you buy what generation the slot is, and then what generation M.2 devices you use.
It looks simple but it's nuanced.
@@amateurwizard I knew about two variants, now I have to find the third. I have one of the pcie 4.0 hyper cards in my desktop, the fan has never been a problem.
@@applicablerobot the sound
@@LA-MJ wow. I wonder if we have different versions, mine is silent
Cost of electricity (peak consumption, US 2022 prices): $648.24/year
Would cost about £1200 a year in UK fiat to run that monster. Costs approximately £60 per year to run a small chinese fanless mini-PC. No contest.
just why? just used a router...
Most non-DIY routers are garbage. No need to build Frankenstein's router like in this video... You can do it much smaller, cheaper, more efficient etc
@@MichaelSmith-fg8xh a router is designed to route traffic nothing more. I've worked on everything from cisco to draytek there is nothing you can't do on a good router as far as traffic management. More efficiency would be to buying secondhand of ebay.
@@Rickymcdd by "most non-DIY" I meant more something like the typical consumer stuff like Netgear. Ive been using a router I built myself for 5 years and it's much better than I had before with even $US600 consumer stuff.
The build in the video is ridiculous as a "router". It's really a NAS and virtualisation server that happens to have a router VM.
@@MichaelSmith-fg8xh Yes the above is not bad, but you could do it with say a Dell R620 plus some cards and drives... It would be noisy, I've got 2 setup with VMware.