Checking Out The ASRock Rack B650D4U Server Motherboard!
HTML-код
- Опубликовано: 21 авг 2024
- **********************************
Check us out online at the following places!
linktr.ee/leve...
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-------------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons...
Even after all these years, Wendell still makes relaxing videos that completely go over my head in techlevels. Brings a smile to my day.
recently set up a Supermicro 8 node in 3U of space AM5 blade system at work with 7900's and 7950x. it has been working great, single/few threads performance is just off the charts, and heavily threaded performance aint too bad either
I wish you commented on the ECC support with this kind of motherboards. ECC support seems to be a bit confusing, some say DDR5 is ECC by default, others say that it doesn't support all ECC features, some MB vendors support ECC officially, some don't, and even for those who do, the list of officially supported ECC modules is ridiculously small, so going for an enterprise-grade MB like this one seems to be the easiest way to avoid headaches for those of us that cannot test dozens of MBs and RAM modules until we find a working combination.
To put it in simple terms, non-ecc ddr5 had some built-in error correction, but ecc ddr5 has more of it due to additional hardware
regular ddr5 is basically ECC lite. Ryzen supports Unregistered ECC - but there isnt a facility for communicating issues with error correction like there is in the EPYC and Xeon chips.
The NM-AMB13 mounting bars are what you need if you want to mount an NH-D15 cooler and have the fan axis parallel to the DIMMs. It will still interfere with the x4 slot and only has .001'' clearance with the GPU backplate so YMMV
I was able to connect a Taichi 7900XTX through 0.8m of 85Ω passive mini-SAS cable at 16Gbps x16
The latest BIOS won't boot four Micron 32GB EUDIMMs at faster than 4000MT. Bad memory timings aren't recoverable so you have to watch it POST through the LED or IPMI and clear the CMOS when it fails. DIMM voltage is not adjustable and fixed at 1.1V
Contrary to the video, ECC errors are also reported in the management interface
Fans running at 1500RPM are frequently reported as failed, and more often at slower speeds. Below 600 rpm they don't register at all. The motherboard headers have a second tach input so you can run two fans off each header and read their RPM separately
The Broadcom UEFI UI is unbearably slow, it can take 45 seconds to respond to a key press if it does so at all
Hi Wendell! Very cool I like these pro-sumer boards from as-rock. I am really looking forward to that update on ECC RAM on AMD. I am really hoping that the AM4 socket has gotten some improvments as it would be awesome to switch to ECC on my ubunutu storage/plex server.
AM4 only supports unbuffered ECC, which is pretty rare in the used market.
i would love to see ASRock do a X670E ATX board, you could turn all the PCIe lanes from the CPU and chipsets into either M.2, U.2 or PCIe slots, couple that with a 7900 and it would be a perfect AM5 server platform
Would love to see a AM5, mini-ITX, 6x SATA, IPMI, ECC, 1x PCIe...
Last time we had something similar was the E3C226D2I and let me tell u, it got ooooold!
Got a bunch of these Asrockrack boards, put my old dev box cpus into them to run my servers. All my main websites run on my old 2700x on earlier version of these boards. Nice and stable with remote management/kvm. Very nice.
I deployed two of those boards at work, the memory training was very long. At least for the last bios version. Sometimes over 10 min.
These ASRock mobos and servers are great. They are replacing that segment that SuperMicro used to occupy before they went to OEM direct market only like Dell. If you don't want IPMI and the server price-tag the Asus Commercial AM5 motherboards give you ECC support on AM5 with a VGA port for datacenter crash carts. Something like the Asus Prime B650M-A II-CSM come to mind. It's pretty close to your ideal layout if you don't care about IPMI. It's likely what I'll be using for my 7000-series build since the margins are so small for over-clocking that I'd rather just spend money for stability instead.
I'm moving my primary home lab server to the x570 version of this board with a 5950x in a couple weeks. Wish it was full ATX with a couple more PCIe slots but it's going to do all that I need it to do for the next 5 to 8, maybe 10 years.
I just finished a build with this exact setup, unfortunately the chipset is overheating at idle or bios screen. temporarily replaced with an ASUS board until RMA replacement arrives. hopefully the replacement board doesn't have the same issue!
This is why I ended up just going EPC 7282 paired with the AsRock Rack ROME6DU-2L2T (which is an insane mATX board btw. 4 4.0 full size x16 slots with full bandwidth). Satisfies all my needs in terms of plenty of room for adding on drives indefinitely, and all the PCIe lanes I could ask for.
I just finished reading a review on STH for the ASRock Rack AM5D4ID-2T and had to check to make sure it was not the same board.
thanks for explaining the SMBUS, I've seen it so many times in kernel configs and dmesg logs, but never knew what it was actually for.
I was looking at Asrock server boards for AM4 10 minutes before this came out. I thought I was being followed by SIRI. Nice video.
I've got a E5-2630L v4 which does 10k Passmark. Neither Intel nor AMD makes something like it any more. Hasn't since 2016. It's low power, has 10c/20t, and 40 PCIe lanes. There is NOTHING on the market with 40 PCIe lanes at 55W TDP. I'd love to get one of these Intel or AMD boards to replace it, but there just aren't enough PCIe lanes for what I'm currently doing with this v4 chip from 2016.
same, and for the cpu and moherboard cost you can buy a lot of power.
Man, what would I do for more ugly cheap components to build a PC.
Ugly, cheap, and with a post code.
You! Let’s bring back the ugly and useful!
Sadly, making things uglier doesn't make them magically cheaper.
@@Zanpaa Trying to make them ugly doesn't make them cheaper, but trying to make them good looking does make them more expensive.
actualy i got this motherboard with a 7950X about a week ago. very useful board
It's funny. I saw that board on Newegg's Cyber Monday sale (the 1Gb NIC version) and was like "hmm, wonder if there's a version with 10Gb NICs" and then, "Hopefully Wendell will cover it and we an learn more." And here we are!
What you need to look at is Supermicro H13SAE-MF, technically it's a workstation motherboard but it has the server level backbone.
Dual x8 PCIe 5.0 Lanes in x16 Slot
Single x4 PCIe 4.0 Lane
Dual M.2 Slots PCIe 5.0
Dual 1gbe Intel Ethernet
Dedicated IPMI Port
I think this one from Supermicro is better than ASRock Rack offerings.
Edit: ASRock seems to have re-added ECC memory support on their AM5 consumer “non-Rack” motherboards. Has anyone verified it working correctly with current BIOS versions?
Just built a Ryzen 7900X server around one of these boards. What an education for my first AMD build! ProxMox runs like the wind on it.
edit: The on-board graphics are... interesting. Your mileage will vary.
im so sad these boards are so expensive. I would like something with this layout but for consumers.
I miss green PCBs
@@zsmith8632 what about brown blue and red pcb's`? good ole days
Dual 10GB RJ45 NICs and IPMI aren't cheap. If one were to add these features separately on a mid range B650 board it would cost slightly more than what this is retailing for. The price if anything seems quite fair. Especially considering that these features are often sold at a premium in the commercial server market.
@@Ilost11 im talking about LAYOUT. Ram perpendicular to the IO shield makes for great airflow. If I want anything else I can buy a pcie card for it. The only other motherboards using this layout (other than oem stuff) was EVGA DARK motherboards, which are super expensive.
@@hectororestes2752 The layout is because it's made for the front to back direction air flows in rack chassis. Nothing more, nothing less. They just don't really make any other kind of boards for rack chassis.
I've built something like this using an x670E PG lightning and 7950X locked to ~45-55w(and no more than 75c, whichever is first)
no dGPU needed.
better transcode quality/compression ratio than GPU, but is fast enough thanks to 16 cores.
many PCIe lanes for high speed networking, SAS controller, and 1TB optane(hoping the 1.5TB come back in stock, didnt even see those before they sold out)
I really would like OOBM though
--there is a bug in FFMPEG that causes massive file sizes OR horrible quality when converting from MPEG2, to H26x/VPx/AV1 on a GPU, and yes, i've tested all hardare transcoders over the past 7 years, including the DG1, and Apple M1/2 (apple being the best) but all are vastly inferior to apple's own and other CPUs, with most hardware transcoders doubling or trippling the file size, or making 1080p, look like youtube 240p at the same file size, despite H265/AV1 being far supiror to the..... 20+ year old? MPEG2
I have an x570d4u-2l2t in my server now. Works well once the BCM got straightened out a few years ago.
Updated build coming to the l1 forums. 😂
Same board here (with a 45W Ryzen 9 5900) and I love it. I'd be more tempted to upgrade to am5 if there were a x670 edition with more features. As it sits these b650 boards are not a direct replacement for the x570 due to either a lack of sata ports, 10GBe or m.2 ports and that's just sad.
I would not be able to straight board swap without losing some drives.
Edit: As wendell says, there isn't a "perfect layout" am5 server board yet.
Edit 2: There is also a WIP port of OpenBMC to the x570d4u board to keep an eye on.
Great stuff, still running a a server with E3-1231-V3 Xeon in my homelab, this would be an amazing upgrade in terms of performance.
me2
This has been on my buy list for a while. But then I checked ebay for epyc. The 7002 can be easily found with a H12SSL-i and 256G of RAM for 500-1500€ depending on the config.
I would really love a workstation like MB for ryzen 7000. The intel workstation chipset has a lot of pcie. The Ryzen ones all need an HBA for a NAS. None come with IPMI + 8 sata. Intel also requires the workstation chipset for ecc ram.
the idle power of 7002 + reg ECC is ridicolous though
Agree on your description of your ideal motherboard! Have been looking for board like that for small business servers for a year now...
would actually like to see regular consumer mobo, GPU etc. in green again
What do you think about reusing old Epic 7002 series systems for homelab/virtualisation?
From my understanding, the main issue with threadripper is the motherboard support.
If vendors are making server grade am5 motherboards then it may make sense for AMD to make 24-32 core am5 chips.
From my understanding, it’s technologically possibly. I think threadripper customers would much rather buy that then threadripper itself.
Am5 and the consumer line of CPUs are heavily limited by the max memory cap and especially the low number of PCIe lanes. If they increased the am5 pcie lanes to the 40-60 lane range then the platform could be viable. But until then, threadripper and Epyc are the only ones with high max memory and lots of pcie lanes.
Intel does the same thing with core cpus vs Xeon workstation and server skus
This would be perfect for a low power solution, low CPU hz, system using the multitude of passive cases that can push out that heat. Then the use case being basically a *slightly* more advanced NAS build.
Slap a multi-NVME board in one of the PCI slots, and maybe additional 2.5Gb networking on another ... it's perfect.
One use-case those old systems still have are applications that need large amounts of memory.
I was wanting to go ASRock Rack for my 5900X AM4 build but $400+ for the board at end of life - stuck with a MSI Pro board till I need to upgrade.
I don't think the power delivery is much good in these. I have a X470/5750U, and besides IPMI it's honestly not an amazing board.
@@Fay7666 strange power delivery issues? That is an odd one. I've been liking these MSI PRO boards, very compatible with 128gb of 3200 CL14/16. I have found however the very limited pcie bus sucks
The last Asrock server board I bought was an X570D4U-2L2T, and now I have it just sitting in a box in my closet because the IPMI firmware is such crap, it doesn't even support serial-over-LAN properly. The chipset is also in a really bad location -- no cooling if you have a Radeon Pro W5700 in the x16 slot.
That makes me wary of any newer Asrock boards, too. My current server now uses a secondhand Supermicro X11SSZ-TLN4F.
Still waiting for any AM5 motherboard that can route its 24 free CPU PCIe lanes to three x8 slots.
So no lanes to the chipset? No USB, etc.? /confused
@@acubleya lot of usb and crap is built straight into the cpu these days ;-)
@@LiLBitsDK The wonders of thinking sand. TY
@@acubley Actually no, there are still the 4 PCIe lanes for the chipset; AM5 gained 4 more CPU lanes compared to AM4, in total 16 + 4 + 4 + 4 (Chipset).
Hmm... just picked up one of those SM chassis for $80 mostly for the spare parts and dual SAS2 expander board but it came with a rather useless JBOD on/off board so you've given me ideas... Thanks Wendell!
Good luck getting regular BIOS and BMC firmware updates for ASRock Rack models. And may God have mercy on your souls.
It would be nice if full ECC support (assuming UDIMM ECC) was available on AM5 "workstation" motherboards. Kind of like a AM5 HEDT that pushes up against Threadripper HEDT. That said, I think AMD should introduce an official gen 2 AM5 HEDT platform with double the PCIE lanes at 24 PCIE 5.0 and 24 PCIE 4.0 (vs Threadripper TRX50 48 PCIE 5.0 and 32 PCIE 4.0) and working ECC UDIMMs up to 48GB/64GB per module at the current maximum four memory slots. This could be the new X class chipset, also with workstation class stable IOMMU and Bifurcation. SO -> B750 > X770 > TRX40 > WRX90. Currently there is a HUGE gaping hole between X670 and TRX50.
I think for the price the only thing I'd change on this board is the 10GbE chip to an Intel X550, that way you'd still keep the 10GbE ports but they can also be used at 5Gb, 2.5Gb and 1Gb. In addition, the drivers for that card are present in pretty much everything (BSD, thus pfSense/OPNsense, Linux, VMWare etc).
Check out Supermicro H13SAE-MF, it's basically the same as this one but with a x8 PCIe 5.0 slot. It is lacking dual 10 GBe ports but you can install a NIC in the x4 PCIe slot.
the x4 slot is open so it can accept x8 or x16 cards (it seems).
I think that we should build your dream AMD motherboard.
Wendell, why doesn't ASRock Rack make an ATX or EATX version of this, and break out 16 of the PCIe 5.0 lanes from the CPU into 16 PCIe 4.0 and 32 PCIe 3.0 lanes across 6 PCIe x16 slots? It seems like the market segment that would be interested in this type of board would prefer more PCIe lanes...
A 64 lane PCIe 3.0 switch would add at least $200 to the BOM, Gen 5 would be even more. If the BOM is $200 more the price will be $600 more so it's not any cheaper than EPYC or TR PRO and the lanes are still oversubscribed
Why doesn't level1techs commission their own custom am5 motherboards? ;)
@@shanent5793Isn't this what the chipset chip on x670e already does (break out PCIe 5.0 into PCIe 4.0? They could simply link more chipsets together. While not being entirely ideal for performance, it wouldn't add $200 to the BoM either; that might be an acceptable trade-off...
@@Level1TechsWendell, that would be really sweet to see! Level1Techs already had the ear off the market that would be interested in this; seems like a perfect fit to me!
@@johndelabretonne2373 Each Promontory 21 chip has 12 PCIe 4.0 lanes, 8 PCIe 3.0 lanes that can also be used for SATA, a 20Gbps USB, and four 10Gbps USB. So the chip only connects upstream to the CPU at PCIe 4.0 x4, 64Gbps total, tying up four of the Gen4 lanes. Chaining them consumes four more lanes for the downstream, so each chip can add four Gen4 and four Gen3 lanes, plus some USB. The final one has no downstream, so four more Gen4 lanes are also available. The whole chain is still limited to 64Gbps, or 7.8GB/s after encoding. Each chip probably adds another 100 to 200ns latency.
Standalone Promontory 21 chips might be useful for building a NAS, each chip could let you add two or four NVMe SSDs, and the bandwidth isn't as much of an issue if you're only serving a dual 10 or 25 GbE NIC through the CPU lanes anyways
I'm waiting for the mini ITX AM5 version with 4 DDR5 SODIMM slots but not deep ITX that they have available. I'm still rocking two AM4 as rock rack mini ITX boards one in desktop and one in home lab server solid motherboards.
These platforms annoy me. They would knock the snot out on my IvyBridgeE xeon. The problem is my xeon has 3 LSI raid cards in it for 24 HDD's. These platforms have no answer for that.
Late, but you mentioned the Supermicro chassis... but you didn't specifically mention the model, just that it has an 2650 v2 platform in it. Can you specify, please?
love it
Memory details at 14:58. He says "4 dimms per slot" but I think the board is setup to allow 2 dimms per channel and the restrictions mentioned are 5200 with 1 dimm per channel and 3600 at 2 dimms per channel. Wendel is used to working on systems with 16+ slots :)
I was looking at swapping an Epyc system into an older generation SM chassis. Unfortunately between shipping and availability all the good quality chassis seem to cost a good chunk of a whole refurb 14g Dell server
What kind of SM are you looking for? I’ve been doing a deep dive and there’s actually some pretty decently priced options out there, especially in the used market. Also, what size?
Where is the demo and evaluation of the board.
I don't understand why but buying ASRock stuff here is a lot harder nowadays. That mainboard over here would probably be around 700USD
lol this is a server motherboard, its price for everyone is 700+USD already
Sweet! How would this compare to the W680 options in a "son of the fobidden router" scenario? Dollar for dollar would you go with Zen 4 or Alder/Raptor Lake?
I'd honestly say this beats out the W680 version in the forbidden router, been thinking about it myself as a alternative. A step up from this would to use the 4th gen scalable and something like the 5416S for that QAT goodness and use all your networking & encryption on that platform.
gosh i would need those 2011 collers for my supermicro server.....
Do you think is possible to cool this VRM on a regular (non-server) chassis?
I am planing to build a workstation on a Sama IM01 (basically a mATX NR200), and component positions on this board seem to go really well with the airflow pah I want to use.
Its gonna be all air cooling. My new motto in life is minimum maintenance, maximum reliability. Hence trying to get my hands on server grade stuff.
Lol, that board is cheaper than some consumer ones at 500 Bucks.
I know What I am gonna get for my workstation.
09:57 I see what you did there Wendell ( ͡° ͜ʖ ͡°)
i have a cluster of V2s/v3 but, i'd need full ATX to move over all of my cards AND i'd have to move to new version of esxi and that won't support my nvidia grid OG cards with gpu sharing
What does "not report to the management bus" for, let's say, a home-lab environment mean?
What’s the Best Low power Motherboard/CPU/HBA combo for a home lab NAS ?
I've been thinking about putting together a rig for ML purposes, and specifically keeping a bunch of small models in RAM to run inference on a hypothetical desktop "Phoenix" iGPU, should it indeed come in January. I wonder if this would be a good fit?
Xyratex HS-1235T-ATX, sporting 2x Intel(R) Xeon(R) CPU X5660 @ 2.80GHz, 48Gb DDR3 ECC (24Gb /socket) and 10x4Tb SAS (Z2 Pool) + 2x1Tb (Raid 1 OS)
Eating more power than I would like truth be said, question is, can I use a normal ATX mobo to upgrade this to a more reasonable power usage?
nice,
Please check out the Gigabyte AM5 Servers
we need to put an APU there to get idle power right
That motherboard would make me buy an AMD cpu. Fully qualify ECC warranty on the Ryzen would make too many cpu fail I guess, reason why full price on Epyc. I like that motherboard quad rj45, not optical but alright.
Sorry what? this board takes the same CPUs you use on consumer boards. ECC is always validated on the CPU.
What is not (officially) validated in consumer boards is the motherboard/bios, because the board manufacturer does not want to pay for the time to do that.
wish more manufacturers would make computer hardware without the ridiculous "gamer" looks and obscene amount of flashy LED's...
6:31 what video is that build from? nice lil server case
Does this MOBO + CPU combo fully support latest version of ESXi?
I know you can probably fix it with bifurcation, but any similar with 2 m.2 slots and 10gbit?
Where do you find server OS drivers for this board? I can only find Windows 10 and 11 drivers and they don't seem to work for Server 2022.
I've had this board since launch and ECC errors (both organic and deliberately injected at the chip and bus levels) are reported and logged in the management interface which has nothing to do with the user OS or kernel. Why do you keep saying that AM5 ECC support is somehow incomplete? What else is it supposed to do?
What kernel? For me the kernel logs them but never goes to the ipmi. On am4 I created a module to inject the error into ipmi logs because the sysmgmt stub wasn't doing it.
I'll tell you where to shove that motherboard... right up your upgrade path
what's the problem
It is a nice Bord. I have learned over the Last years you dont need al those Features and Performance really for Home Server stuff unless you need those for crunching numbers or Lifting tons of Data. But for me i have only a 50mbit Network to the Internet and a secondarry 15mbit Network over a cellphone Router with a Sim Card. My Personal Home Server needed only be 24/7 available and Power efficient. So i bought a j5040 with 16gb RAM and a Pico psu and uses a old external HDD wall wart with 12V 25W. On this Server ist running a Minecraft Server 3gb 20players with Standard settings and a Samba Network storage Server. And everything i need to Deal can be Done over ssh. I have 1 SSD with 500gb in it and nothing else. I run powertop for deactivating every Thing what i cant in the BIOS and for some system behavier Things that preventing from going down with the c States. And even 20 Players are online the CPU is only to 30% used. The best Thing in this System is, it is dead silent and draws in idle 4W and on CPU Benchmarks and HDD Benchmark parallel only 17W. On Thing say be Said i Ran Ubuntu 22.04 LTS in this Thing Not Ubuntu Server. Box Standard Ubuntu 22.04 LTS. It is the ideal Server vor my use Case. I dont need more. I dont need ecc or ipmi or other fancy stuff. If i need maybe more Power in the future. There is maybe a system Like this available that uses the newest nessesary pc Tech but again packed in a small 10w cpu Like this.
Since this has been brought up this way, I'm curious if I shouldn't do the dual 2699 v5 thing, I currently have a gaming/homelab setup built around a 7950x, but should i just buy this board, and get a 7800x3d, or would 22 x99 cores actually still make any sence. (Power budget isn't a concern)
By the looks of it, it seems that 2x2699 would still be slower than a single 7950X (25k for each of the former, 63k for the latter on the passmark website)
Single thread is over double (1.9k vs 4.3k)
@@iaial0 That's kind of insane. I guess that's settled though, but I would like to mention that I could get a supermicro board and both cpus for the price I paid for my 7950x alone. So idk what that says besides you get what you pay for but oh well. I might anyways because I want to put it in an IBM server tower from the 90s
@@awetisimgaming7473 yeah, raw performance numbers alone aren't enough to make a decision, especially if you don't consider price (CPUs on eBay are absurdly cheap, while motherboards eh idk) or the application.
No RGB juhuuu! Bloody distracting lights.
How can you get the pikvm for o cheap?
I like your videos, but can you talk a little less and get to the building and performance of the products please.
Anybody considering AMD siena 8004 over this?
What's a hyperscaler
big company that has lots of servers. Facebook, Google Microsoft Amazon etc
T?
SUpermicro H13SAE-MF
I went looking the other day at AM5 homelab and found the biostar b650egtq, but I couldn't find anywhere to buy them.....😢
Well, if you manage to get the biostar, post your user experience on the level one forums, Biostar doesnt come to mind when i think of stability- but brands can always redeem themselves😅
I saw people mention that on some forums but other forums in other languages seemed to think they were great so I don't know if it is just a region issue and the products the post to a region. If I can get one though I'll post on the forum.