Another great video man love it, I will to hit up ali express for some dual Xeon motherboards to replace this DD 6300 locked down one and have a nice enclosure to start my own project.
I wish I knew what they were for originally! I didn't know about satadom until it was pointed out by another as well. That would have been quite a bit better than my routing of cables for boot drives.
Kind of a moot point after upgrading - but I had a similar issue with enterprise PCIe cards not booting on my home server. The issue was due to their SMBus misbehaving/ not registering correctly with the host system. I had a Dell H330 card that would not work on my home server until I taped over pins 5 and 6 on the PCIe connector with electrical tape.
Nice! My current server is a core i5 4430, I have plans to eventually upgrade it to either an i7 8700k or ryzen 7 2700. But it's doing its job as a plex server, bedrock MC, server and pi-hole without a sweat, so no real rush atm to upgrade
Good video man, i heard that server racks are better than cheap diy desktop NAS, just because they are sort of vibration proof, is something you have experience? keep the good work
You can get 32GB DDR4 ECC sticks for peanuts, I have 2 on my desk I bough got £35 EACH -- Also, the xeons if you hunt around can be found for really cheap too, even the *top of the range* ones. I upgraded my own server with 2*E5-2696 v4 (22 cores *each*, these are upgraded 2699's) and the CPU were £130 from china. The CPU I use in my workstation is the moderately quicker (fro less cores) E5-2697 v4 (16 cores). Again, DDR4 (2133 or, a bit more rare, 2400) 32GB sticks (samsung, Dell branded etc) are super cheap of you are patient. I put 256GB in my workstation, and there's now 640GB (!!!!) in that Dell rack server I use for builds. Quite frankly with that sort of configuration I can ignore modern CPU craze (and super expensive ram) for the foseable future.
I do agree, you can get great deals if you wait patiently and stalk listings. Prices are coming down. The 2699v3 can already be bought from a US seller for over $15 less than I paid, and it hasn't been long since I built this. I didn't see a point in 32GB sticks with having so many slots. 8x16GB gives me 128GB and even the 64GB I have now is more than plenty. By the time I exceed 128GB (in hopefully many, many years) I'll be looking at building a new system out anyway :)
Hello. I am looking to build something similar to this high core count dual xeon build for video editing. In particular long 4k videos with effects. I have built a workstation before but i never built a rack mount server type system. Do i need to use these type of computers to pull this off? Can i make it in a workstation form factor? If not are there any alternatives?
Aside from that thing of covering the PCIe slot pins, many Supermicro boards have jumpers (JSMB1 and JSMB2 in the manual) to prevent the SMBUS from being exposed on the PCIe slot. Perhaps the jumpers need to be moved to the "off" position, or even to floating (no pins connected)?
120GB drives for OS are fine. You should not have huge swap files, that is only an issue with too little RAM. You plastic shroud for the CPU is missing a side piece that slots into the side to close in air flow.
120GB is too small these days, even if not using such a large swap partition (which you're right, is really not needed). Also yes, I do believe the fan shroud is missing a piece - you have a good eye. Lol.
@@HomeSysAdmin The only reason I run bigger drives is whats available on the market. I am usually running up Hyper-V thus only as a hypervisor and like all good sysadmins, you dont install anything else on the host OS.
I think u should check your bios settings and u may find a place under advanced chipset where it will give u option to manually assign lanes. Many motherboards even though they may have dual 16x wired slots they both can not run at x16 together. The board should automatically detect and select x8 for each slot when u have cards plugged in x16 both slots. But i guess your board is unable to detect and properly bifurcate so u can just set it manually 8x on both slots and that will force the pcie switchs to force 8x on both slots doesn't matter if something is plugged in or not. & This is why server platform is different than desktop. In server cpus u have more pcie lanes that allow more expansion slots to run properly! You did the right thing to switch to server platform it will solve a lot of current and future issues.
I created a new channel for the server content and re-uploaded it after removing from the other channel. I haven't made the official announcement yet but it seems people are finding it on their own. LOL
You might need to cover pin 5 and 6 on the first smallest pcie connector as it sends some information that messes with the boot signal, I had this issue with my z230 board.
Your videos have been very helpful for myself and my chia farming. Could you do a video on setting up the farmer with ubuntu from scratch? I have been using windows and it is a pain... I tried a year ago to set up with ubuntu but failed miserably.
Not sure I'll do a video on it, but I'm more than happy to help. What problems were you running in to? I would recommend the "install from source" method vs the pre-packaged stuff github.com/Chia-Network/chia-blockchain/wiki/INSTALL#install-from-source
have you considered that your BIOS and the network cards firmware do not checkout? is there possibly a BIOS update that fixes that specific issue? considering it works once posted tells me it's a firmware and BIOS handshake issue.
I was running the latest version of the BIOS (which is several years old). I think I was just out of PCIe lanes for the CPU I was using - I'm not sure. I should have tried removing one of the other cards and seeing if it worked.
It was running the latest BIOS and I tired various UEFI modes. I didn't think to check the card's firmware, but I think the problem was PCIe lane exhaustion.
Can't fit 120mm fans in a 2U case. I guess you could use a liquid cooler? I've never heard of anyone doing that on a rack-mount server before but... sure?
I have a supermicro x11sdv-4c-tln2f that has bifurcation on it. I think I set all settings correctly in the bios. And yet I can not get the card to see in ubuntu no matter what I do.
Make sure the bifurcation is set to x4x4 or x4x4x4x4. Some of the SM boards have "auto" mode for bifurcation; however, that never seems to work when I try it. Also, make sure it's being set on the correct slot. The slot numbers don't always match the numbering in the BIOS. The motherboard manual will have a block diagram with the slot number info.
This was a thing back in 2005 to do, but why still do it now? Why not use AWS and be building all the terraform and cfn/json needed for the modern serverless work that we are well into? Cheaper too with more flexibility and expandability.
@@HomeSysAdmin Sure I just stumbled over your video thanks to the YT algorithm and I am indeed not aware of what the bigger picture is. Having said that though I am not aware of what use cases these days - as far as professional training/work goes - where there is a need for building servers and the like, let alone at ones own home. I know that there is still scope in the embedded space for sure. But commodity servers have sort of gone the way of the dodo. Especially on x86 architecture.
@@TurbanLeDurban It's important to note the differences here. The hard drive that stores the plots is pretty much always idle or doing a quick lookup. In fact, I would argue the workload on the hard drive is significantly less than the workload from a typical workstation computer. This is very different from "mining" you hear about that uses graphics cards whereby the graphics cards are pounded to death with a 100% (and often overclocked) workload.
Yes, memory is cheap! But you should still have some space allocated to swap regardless. Linux's memory management does a great job and you'll find it still offloads some infrequently-accessed things after a while.
Supermicro X10SRi-F Server... ebay.us/YQdOs0
Xeon E5-2630v4... ebay.us/HpnfLp
(affiliate links)
anyone else cringe when the motherboard was on the carpet. good video thanks.
Yes! I have never done that in over 35 years of dealing with computer parts.
Eh, I've done that with hundreds of boards and haven't had one fail yet. Not saying it's the "best" idea but...
Excellent video. You deserve much more subscribers who will be coming.
Thanks and good luck with your new Channel...
Thanks! 🙂
i just discovered your channel! amazing video!
Thanks! It's actually a brand new channel spun off of my other, "Lithium Solar" 🙂
I’m old too let’s go, I can still learn!
Great video! Feel free to elaborate more on Chia mining too.
Another great video man love it, I will to hit up ali express for some dual Xeon motherboards to replace this DD 6300 locked down one and have a nice enclosure to start my own project.
Nice upgrade dude. Enjoyed the video.
the 2 yellow ports you can put sata doms in there, and they will provide power to the sata doms and now extra wires.
I wish I knew what they were for originally! I didn't know about satadom until it was pointed out by another as well. That would have been quite a bit better than my routing of cables for boot drives.
Kind of a moot point after upgrading - but I had a similar issue with enterprise PCIe cards not booting on my home server. The issue was due to their SMBus misbehaving/ not registering correctly with the host system. I had a Dell H330 card that would not work on my home server until I taped over pins 5 and 6 on the PCIe connector with electrical tape.
Subscribed from your other channel.
Nice! My current server is a core i5 4430, I have plans to eventually upgrade it to either an i7 8700k or ryzen 7 2700. But it's doing its job as a plex server, bedrock MC, server and pi-hole without a sweat, so no real rush atm to upgrade
Good video man, i heard that server racks are better than cheap diy desktop NAS, just because they are sort of vibration proof, is something you have experience? keep the good work
Good video !
Awesome! OL8 - great OS. I work for Oracle (former Sun Microsystems) as Field Engineer. The setup looks great. Thanks for sharing
You can also control fanspeed via the IPMI, while the machine is running too, I have this same motherboard it is great
you can always buy 2 SataDOM drives(for boot) and install them in the 2 orange sata ports
Those look interesting - didn't even know such a thing existed!
👍perfect
You can get 32GB DDR4 ECC sticks for peanuts, I have 2 on my desk I bough got £35 EACH -- Also, the xeons if you hunt around can be found for really cheap too, even the *top of the range* ones. I upgraded my own server with 2*E5-2696 v4 (22 cores *each*, these are upgraded 2699's) and the CPU were £130 from china. The CPU I use in my workstation is the moderately quicker (fro less cores) E5-2697 v4 (16 cores). Again, DDR4 (2133 or, a bit more rare, 2400) 32GB sticks (samsung, Dell branded etc) are super cheap of you are patient. I put 256GB in my workstation, and there's now 640GB (!!!!) in that Dell rack server I use for builds. Quite frankly with that sort of configuration I can ignore modern CPU craze (and super expensive ram) for the foseable future.
I do agree, you can get great deals if you wait patiently and stalk listings. Prices are coming down. The 2699v3 can already be bought from a US seller for over $15 less than I paid, and it hasn't been long since I built this. I didn't see a point in 32GB sticks with having so many slots. 8x16GB gives me 128GB and even the 64GB I have now is more than plenty. By the time I exceed 128GB (in hopefully many, many years) I'll be looking at building a new system out anyway :)
Oh I'm mixing up my builds now, this one had the 2630v4, not the 2699v3 🙂
Hello. I am looking to build something similar to this high core count dual xeon build for video editing. In particular long 4k videos with effects. I have built a workstation before but i never built a rack mount server type system. Do i need to use these type of computers to pull this off? Can i make it in a workstation form factor? If not are there any alternatives?
Aside from that thing of covering the PCIe slot pins, many Supermicro boards have jumpers (JSMB1 and JSMB2 in the manual) to prevent the SMBUS from being exposed on the PCIe slot. Perhaps the jumpers need to be moved to the "off" position, or even to floating (no pins connected)?
120GB drives for OS are fine. You should not have huge swap files, that is only an issue with too little RAM. You plastic shroud for the CPU is missing a side piece that slots into the side to close in air flow.
120GB is too small these days, even if not using such a large swap partition (which you're right, is really not needed). Also yes, I do believe the fan shroud is missing a piece - you have a good eye. Lol.
@@HomeSysAdmin The only reason I run bigger drives is whats available on the market. I am usually running up Hyper-V thus only as a hypervisor and like all good sysadmins, you dont install anything else on the host OS.
I think u should check your bios settings and u may find a place under advanced chipset where it will give u option to manually assign lanes. Many motherboards even though they may have dual 16x wired slots they both can not run at x16 together. The board should automatically detect and select x8 for each slot when u have cards plugged in x16 both slots. But i guess your board is unable to detect and properly bifurcate so u can just set it manually 8x on both slots and that will force the pcie switchs to force 8x on both slots doesn't matter if something is plugged in or not.
& This is why server platform is different than desktop.
In server cpus u have more pcie lanes that allow more expansion slots to run properly!
You did the right thing to switch to server platform it will solve a lot of current and future issues.
You actually don't need Putty. It works with PowerShell to connect to the server and PowerShell comes with Windows. :)
You are correct! but yuck, powershell LOL 😀
Wait, is this a re-upload? I swear I watched this already
( Edit: Just realized that this wasn't your main channel :p )
I created a new channel for the server content and re-uploaded it after removing from the other channel. I haven't made the official announcement yet but it seems people are finding it on their own. LOL
@@HomeSysAdmin Not a bad idea honestly and yeah, I found it naturally in my recommended
You might need to cover pin 5 and 6 on the first smallest pcie connector as it sends some information that messes with the boot signal, I had this issue with my z230 board.
Your videos have been very helpful for myself and my chia farming. Could you do a video on setting up the farmer with ubuntu from scratch? I have been using windows and it is a pain... I tried a year ago to set up with ubuntu but failed miserably.
Not sure I'll do a video on it, but I'm more than happy to help. What problems were you running in to? I would recommend the "install from source" method vs the pre-packaged stuff github.com/Chia-Network/chia-blockchain/wiki/INSTALL#install-from-source
❤️
have you considered that your BIOS and the network cards firmware do not checkout? is there possibly a BIOS update that fixes that specific issue? considering it works once posted tells me it's a firmware and BIOS handshake issue.
I was running the latest version of the BIOS (which is several years old). I think I was just out of PCIe lanes for the CPU I was using - I'm not sure. I should have tried removing one of the other cards and seeing if it worked.
@@HomeSysAdmin hey eitherway that was a nice upgrade and it was probably worth the cost.
may be worthwhile to check out epyc 7002 series and Supermicro H11 series motherboards(not compatible with 7003 milan tho).
No AMD in my house 🙂
$11? What a steal!
There are still a pile of them in stock if you want one ;)
could be an issue with uefi, try updating the bios and firmware of card and motherboard.
It was running the latest BIOS and I tired various UEFI modes. I didn't think to check the card's firmware, but I think the problem was PCIe lane exhaustion.
How do you find a good mini-PC? Any tips with getting one? Look for, deals...
If you're referring to the Dell that I have on the rack shelf, it's a Dell Optiplex Micro 5050. You can find them on ebay for cheap.
I have a BIG IMPORTANT question, can you use 360mm AIO Liquid cooler, and 3 120mm fans?
Can't fit 120mm fans in a 2U case. I guess you could use a liquid cooler? I've never heard of anyone doing that on a rack-mount server before but... sure?
@@HomeSysAdmin Cool, if you ever do a video about it LMK :D :D :D :D
I have a supermicro x11sdv-4c-tln2f that has bifurcation on it. I think I set all settings correctly in the bios. And yet I can not get the card to see in ubuntu no matter what I do.
Make sure the bifurcation is set to x4x4 or x4x4x4x4. Some of the SM boards have "auto" mode for bifurcation; however, that never seems to work when I try it. Also, make sure it's being set on the correct slot. The slot numbers don't always match the numbering in the BIOS. The motherboard manual will have a block diagram with the slot number info.
is 10 core Xeon strong enough for editing 1080p and 4K videos ?
It's plenty fast for video editing. What will help more than the CPU is having a good graphics card that supports hardware acceleration.
but can it run crysis 3?
LOL probably not very well. There's no dedicated GPU and the low clock rate of the Xeons is designed for server work-loads, not gaming.
This was a thing back in 2005 to do, but why still do it now? Why not use AWS and be building all the terraform and cfn/json needed for the modern serverless work that we are well into? Cheaper too with more flexibility and expandability.
The technologies you're proposing have nothing to do with what I'm trying to accomplish here. You must be misunderstanding?
@@HomeSysAdmin Sure I just stumbled over your video thanks to the YT algorithm and I am indeed not aware of what the bigger picture is. Having said that though I am not aware of what use cases these days - as far as professional training/work goes - where there is a need for building servers and the like, let alone at ones own home. I know that there is still scope in the embedded space for sure. But commodity servers have sort of gone the way of the dodo. Especially on x86 architecture.
If you want big electric bill. This is one way to do it.
No idea what you're talking about. This is a very energy-efficient setup.
Placing motherboard on mat like that triggers me.
Why? It's a clean carpet. Static concerns?
check with Linus
Repurpose the old system to a opnsense box,etc
What the heck is a CHIA farm? Bitcoin?
It's a cryptocurrency like bitcoin but it mines (farms) with plot files on hard drives instead of power-hogging expensive graphics cards.
@@HomeSysAdmin sounds like a lot of failed hard drives haha
@@TurbanLeDurban Why would you say that?
@@HomeSysAdmin well mining is constant usage and constant usage of a hard drive can make it fail, unless I'm mistaken
@@TurbanLeDurban It's important to note the differences here. The hard drive that stores the plots is pretty much always idle or doing a quick lookup. In fact, I would argue the workload on the hard drive is significantly less than the workload from a typical workstation computer. This is very different from "mining" you hear about that uses graphics cards whereby the graphics cards are pounded to death with a 100% (and often overclocked) workload.
hey mate
Hello :)
Swap? RAM is cheap enough, you shouldn’t need swap.
Yes, memory is cheap! But you should still have some space allocated to swap regardless. Linux's memory management does a great job and you'll find it still offloads some infrequently-accessed things after a while.
ethe br0
ethtool br0
For what do you use your nas? Work files?
Pretty much everything that gets stored at home goes on the NAS instead of storing on any specific computer's hard drive.