Not sure how people watch this video and come to the conclusion that they have a 10Gbps link to their ISP. The 10Gbps speeds are just for their internal network. Their "Internet" speed I imagine is just standard business fibre at 1Gbps
Casey Stellar Well yeah it depends where your business is located but business's get the option of paying outright to get a leased connection installed, even if it is ridiculously expensive.
@Bald new you pretty much get what you pay for - want better than 1050ti you'll need to pay for better than 1050ti. A used GTX 970 should be similar in cost but a lot faster. Slightly more for a GTX 980.
+LinusTechTips Btw Linus, what do you think about a video comparing nvidia Shield streaming to the Moonlight stream app on a random tablet/phone that is compatible with nvidia gamestream?
I'm surprised that the video is again washed out :/ You guys have a problem with a monitor calibration or someone doesn't know what they're doing... It doesn't destroy the video, but it's not very pleasing to watch :( Edit: Mistakes happen, that's normal, but I would hope you guys get it under control soon.
it's not that it's washed out, it's confirming video levels for the web.. the have color calibrated monitors and use certain colors so that it's easier to distinguish whites and Black's and details in the shadows and highlights are preserved.. otherwise you lose data in the video and lose video quality as well
+jack black It's actually a lack of contrast. Most people set up their cameras to shoot as much latitude as possible to give more headroom in color correction. This has probably just been rushed out. Skin tones look a little weird, probably down to LED lighting which can ave a really poor CRI. Tbh, it's not worth complaining about, but the tech world is filled with pedants.
I'd love to get some 10 gigabit between my PC and NAS, but I'd need to upgrade the card in the NAS, my switch, the cabling, and the NIC in my PC - so essentially every piece of hardware involved :P lol
I've been really waiting for some coverage on this topic. I picked up a couple SFP+ cards and a cable to run between my 2 main studio rigs and am setting up a 10gbe connection between them, but I've yet to find an "affordable" 10gb switch for my budget, haha.
the people who visit this channel are already kinda into tech stuff, and yet so many of them don't even know what this video is talking about and start comparing their internet speeds to this, makes one wonder what knowledge the general public has on these kind of stuff.
First, this is NOT an attack on your comment. It always amazes me also at what some people are totally oblivious to. Not everyone who watches this channel has a computer tech ability at all. Many watch this channel to try to learn about specific technologies to increase their knowledge. So yeah, the general public may be into using computers, but many have no working concept of how or why they work. Many people do not even understand the differences between RAM and what a Hard Drive is. It is all storage in many people's minds. Add SSD, which really is memory, and you have completely lost them. Forget about even more basic concepts like BIOS and what ROMs are. It comes down to what a person really wants to do with a computer. If they have no interest in programming, they typically will never learn basic programming concepts. If they have no interest in how the computer works inside, they often will never learn what the basic components and terms are. If you just drive a car for transportation, you may not know what size your engine is, how many cylinders it has, or even what type, or where the oil goes! It is human nature to not be concerned about what does not matter to you. Who knows what they ate for breakfast 2 weeks ago? Who knows their cholesterol levels? Who knows their blood type? How about all the birth dates of everyone in the family? These are all basic points of information, but unless a person actually cares about them, it is unlikely they will know the answers. Years ago, the government was horrified at the revelation that people were so illiterate, that many did not know how to even set the clock on their VCRs! It gets very old to set a clock all the time, you do not need, since you may never use the VCR for timer recording. Especially when power bumps occur and it has to be done again EVERY time! People did not know how to set the time, because it did not matter to them. Many people are the same with Microwave Ovens today! At the same time, many of these people have amazing skills at things I would never want to know, such as being medical physicians and molecular biologists (I think there is such a field right?), and even more basic needed skills such as physical strength and endurance to build and maintain our roads and freeways! We are all different, and the best goal is to help each other as needed, and be willing to also accept help from others. We can all learn, and we can all teach. It just depends on what the topic is at the time.
why is everyone going on about internet speeds? he is mainly using it for LAN. So production guys can transfer/work with large files from the fileserver.
Have you tried disabling UDP/TCP offload and Large Send Offload for IPv4 and IPv6? I know I've had similar problems with Hyper-V where TCP/UDP offload severely throttles network speeds because it's offloading work to the CPU.
holy christ. Network speed =/= internet speed. Network speed is the data rate between their in house servers and computers, i.e. moving videos from one machine to another. Internet speed is what you use for downloading crap off the internet.
10Gb networking really is quite tricky to get right. You are on the right path with the receive side scaling (if you do not use that, all networking is done on core 0, which coincidentally is also the one used mainly for the OS itself, and even with the latest CPU, you won't get more than about 4Gb/s out of one core alone). Since you are using a 2 port card - have you made sure that the receive queues do not use an overlapping CPU core set? You can set a max processor / base processor number for each NIC, try using something like 2 to 8 for one NIC and 10 to 16 for the other for a first test and 4 receive queues for each (only even numbers=physical cores count). With Intel drivers, you should also be OK using all the offloading features. If you are using jumbo frames, make sure they are active everywhere (even your NAS). Also make sure flow control is active on the cards and on the switch ports, it helps a bit. Does your NAS support SMB3 if you're using Win10 / Server 2012R2 clients? With the exact same card you are using, I am getting live migrations with 2x9.8Gb/s utilization between Hyper-V hosts (SMB3 multichannel over two ports in different VLANs), so it is definitely possible :-)
Does this switch have shared or dedicated port buffers? And it's probably not non-blocking ( overbooked backplane ). Also watch out with jumbo frames. It can improve throughput, but I have seen very wierd cases with iscsi and nfs where packets were dropped because of unexpected packet sizes. In your case with native SMB3 please check the tcp window frame sizing. You can tune your nics with netsh.
Linus, I'm gonna go out on a limb here. Maybe the machines that you were experiencing slower transfer speeds had a cable length/distance of greater than 100m? Limitations of CAT6A/7 with 10GBaseT.
Hi Linus, I remember I made some tests on my 1 Gbit Local network and I figured out that pulling data is faster than pushing. this said in my own words describing what I figured out. an example : you are on machine 1 > cut or copy a file from machine 2 and paste it on machine 1 ( pulling) is much faster than (Pushing). Pushing > you are still on machine one > cut or copy an item from machine one and paste it on machine two while you are still on machine 1 ( pushing to machine 2) is slower
People in the comments need to understand this isn't internet speeds. It's their internal network from their group of computers. Basically used for fast file transfer throughout the office.
FYI: Jumbo Frames is a horrible creation. It scales very poorly, especially with respect to switch resource usage, and it gives you no benefit when you are transferring small files. Naturally the overhead is multiplied greatly whenever packet loss occurs congesting what is already an unstable link even further. In theory the maximum difference in performance on a 10Gb link with or without Jumbo Frames for large payloads would be in the area of 5-6% meaning that the maximum benefit they should give you is 75 MB/s faster transfer. While that is a noticeable difference it does not explain what you are seeing. I would wager that the other changes you made to the NIC, number of CPU threads and to a lesser degree the send buffers, had a much larger impact.
You guys should do more networking videos. I think you guys are very entertaining and believe you could bring some humor to the subject that could make it less painful.
The one thing I can think about with having this much speed is virtualization especially if you want to stream enterprise apps to all of your users as well as doing a NAS setup.
We have had 10 gig for years @ work, using sfp+ and om2 multimode fiber. If we are transferring lots of tiny files, the performance is never very good, single giant files are always fast. I'd check your server to see if any kind of read-ahead and write-back are enabled on the array. Stripe size of the raid(s) can also make a huge difference when writing. For giant files like you describe, I usually make an array w/512K stripes. The higher stripe size always seems to affect write speed, not so much read. I took a local ssd array and changed it from 64 to 512 and got 1600 - >2200 MB/'s difference on writes. Anyway, good luck on your 10gig network!
I have always been told that networks at that speed can not be actually handled due to the length of the able, if you ping one PC to the other you will get
+Xuvial Sylvester I like the vessel app more than the new RUclips layout, considering that it is a band new service it will only get better with time. also no need to deal with Google bullshit
I don't care about video quality, I come to this channel for information quality. I work at best buy, so what I gather will go towards helping others that come to my store.
Me and my friend were amazed in 1994 November when we both got 28.8 modems. He came from a 14.4 and me a 2400. We were so excited to see the numbers spin on a file transfer. Then we found IceZmodeM and you could type to each other during file transfers!! TYPE. TO. EACH. OTHER!...So amazing.
Just to be clear for some people on the comments section: #1 - 10 Gigabit (10Gb) - 10 Gigabyte (10GB) Are not the same, bits are mostly used in transfer or internet speed while bytes are used mostly for file sizes. #2 This video is about the Local Network used just in the office. For example, the video editors sending their projects to the server(s) for backup purposes. This has Nothing to do with their internet bandwith. Linus never mentioned their ISP or their internet connection. Sooo the comments like "And I'm just sitting here downloading at 512KB/s" are a little out of place. Still they have decent bandwith in the office, I believe it's 200Mb/s upload and download. (Sorry if my grammar is bad, English is my second language)
So lets see, the comments thus far, the video is washed out, this is something that people care about obviously because they incessantly go on about it. The next group can't comprehend the content so they ramble on about what they think it's about then another group has to explain to them what the actual content is about. Then you have the trolls who have such crappy lives they literally post stupidity where ever they can so they can show the world what an idiot looks like. I don't know how Linus does this for a living, the idjits out number the people you want watching your channel it seems. My take on this video, it was well done as always, I could care less if it's washed out a bit, the channel and equipment are a work in progress. I don't watch Linus for perfectly done videos, I watch for the content. As for 10 gigabit, I'm hoping to make the move so at least my server and my main workstation have 10 gigabit connectivity and I hope to attain at least 500 megabyte transfers. I've been happy with 110 to 129 megabyte sustained transfers (the max Gigabit can hit) but ever since the move to NVMe and SSD's in the server, I need the speed!
+KlipschHead281 The only things these problems prove is Linus is getting popular. Who knows, he may advance to the point of disabling RUclips comments altogether
Dunno whether someone has mentioned it, but if the MTU on both ends doesn't match, you'll end up bottlenecking at the switch, cause it would need to repackage every single freakin frame (I've run into this multiple times with corporate clients). Also the first thing you do whenever you get above 1GbE is to setup jumbos and increase the Rx and Tx buffers. TOE also helps. The faster/slower transfer inconsistency with spindle drives is usually due to the block sizes discrepancies or lack of fine tuning (depending on the majority of files being stored it makes a HUGE difference whether you're running with 4K or 512K blocks), but since you're using SSDs and RAM disk, I honestly have no idea what might be the reason there.. Also if you're using Windows all across your infra and have enough ports, given the fact you have dualport cards and especially since they're 10X (meaning rj45), I would consider LACP or some sort of balancing trunking - LACP would be best for Win8+ and WS2012r2+ (completely new network stack on both). I could get into more detail if you're interested why LACP and so on, but I guess that's enough for now. Cheers!
What's crazy is that gigabit Ethernet cards in the 90's used to cost $1000, then one day in July of 2000 Steve Jobs went on stage and announced that it would be standard in all new PowerMac G4 models making gigabit networking adorable almost instantly.
Maybe using a Linux or BSD would work better? I would never trust Windows for performance... (Also, I am aware that there are others, but those are the ones I know that are best for networking)
I do have 20 symetric :p but this video is about 10 gbits speed in their own networking (just betwen the computers at their office) not their internet speed (ips) also to mention not to confuse gbits with gbytes
the said ssd should be as fast as ram or faster for it to be used efficently as cache which in this case is pretty much useless since ram for storage is already faster than any known type of storage
It's more likely because you always have a certain overhead on any network. So you would never ever get 100% theoretical value. Hence why 80% better reflect what you can expect in real life
Hey there, do not try to cut the files because if you do that, the original file has to do two things at once , it has to transfer the file and clean it from the original PC. The transfer speed depends on the ratio between the TCP package and the file length. Note: remember that the network cards and operating systems have buffers , so that's the difference when you transfer files from one place to another.
Linus, not sure if its the extra confidence or space the new office has give you - but dam - you presentation skills are on fire, 30% better I would say - love it
Linus, do you think that the length of the cable run from your test bench to the server/switch room had something to do with the variable speeds? Have you tried it right beside the switch? Maybe the quality of the connectors used in the network installation might contribute.
I mean 1GB/s in general would be freaking awesome no doubt, but the ~500MB/s you get sometimes are quite good as well. I can't really remember when I did such big transfers that often. Ok, sometimes I copy films on my Home-Streaming PC and my network is running on about 300MB/s, It usually doesn't take longer than 10 minutes, which I think is quite OK. Especially when I do that 3 to 6 times a year. And yeah, I get that your channel is all about CRAZY STUFF haha :D, but as I said in general I would be happy with ~500MB/s.
@LinusTechTips Make sure both the PC & server is set to high performance mode. I'm not 100% sure if you can tweak the SMB buffers on windows, but for my FreeNAS based server with bonded 10Ge, I needed to raise the SMB buffers significantly before I got good throughput. Also, do note that because SMB is single threaded it has interesting behavior to say the least. You'll notice that if you have a client doing multiple transfers at once, you'll be limited to the slowest connection. So let's say you have a theoretical 700MB/s limit, but for some reason one transfer is going at 100MB/s, the other transfer will be capped at 100MB/s on the windows client. If the slowest transfer goes up, as does the other one. Once that was done, and the client machines weren't in low power states, I was capable of pushing 700-800MB/s on spinning disks from our ZFS servers.
I need a serious upgrade to my caring. I'm still haven't installed caring 2.0 yet. ...and that leaves me with a blissful lack of caring that allows me to not care,..at all. And that's something I care very deeply about. I'd say thanks for the video if I cared. I'd wish you peace and health if I cared :)
The issue you're seeing is because of the inherent issues with TCP/IP. The reliable and consistent performance you're hoping for can only be seen with SAN systems with direct connect 8Gb/16Gb fibre channel from the client to the storage and an outboard metadata link connected via 1Gb or 10Gb. (Metadata doesn't need much in terms of speed.) For how much you spent, if 500MB/s is the worst speeds you are getting, you're doing great.
Packet size is negotiated which is why it didn't matter, but the switch has a maximum packet size which is why that matters. For SUPER cheap, home 10Gbit, use CX4 cables/switch/card. Find a used switch like a Woven LB4 for $200 or less. For Linux, use a Myricom card which should be $50-100 on Ebay each. For Windows machines, go Intel NetEffect NE020 for, well, about the same price. Last I checked, the Myricom had no Windows drivers and that particular Intel no Linux drivers. (But this was a few years ago.) Woven LB4 has 4xCX4 + 48x1Gbit. Note: Woven LB4 is LOUD.
Nice. I was so worried about my old cat5e network when i got my 300mbit internet but it could handle it. Even tho most of the internet is lagging behind (pretty much only steam can do those speeds) its clear that you shouldnt invest in less than cat6. These very high speeds also question if you really need large local space as you can pull data so fast from afar that itll feel like its "local on a old hdd" or similar.
The variation in the speed are due to the caching in memory. When you are reading from a server, to your machine with a lot of memory, you are actually writing to a cache in RAM, and OS writes back in background this data to the harddrive (or ssd). This means ssd/hdd is not a bottleneck. If you would actually copy very very big file (bigger than your memory), you will see the decrees in speed. The same happens when you read data back in opposite direction. You will be only able to read at a speed of sata3 interface, and not faster, unless it is already cached. But I guess, Windows is cleaning the cache in the background quiet quickly.
I don't know the details of the server hardware configuration, but if my understanding is correct he's running FreeNAS, whith ZFS, on a hardware RAID array. That may be adding some IO delay. Enabling jumbo frames on the network would be the first step as he did, but I don't think the bottleneck was ever on the network because with wire speed switching you should have no trouble achieving max throughput.
oh yeah if your frame/MTU sizes are different, the individual frames will get broken up along the way and have to be reassembled, this will add tremendous overhead and in that case the bottle neck would be the network.
10Gbps networking has been in my roadmap for a year now. It sucks having to wait 3 hours for a file to transfer at 1Gbps speeds. I prewired with cat6A when I was doing the runs to my server.
For those who won't need RJ45-specific cabling, then I just picked up an Emulex P004096-01K network card with 2x SFP+'s for $50, and a 48-slot Quanta LB4M switch with 2x SFP+ uplinks for $95 That's $145 in ebay money (excl. ship n tax) for a cheapo 10Gbit setup.
The issue may be your MTU sizes (aka Jumbo Packet size), you mention "it doesn't seem to matter if your values don't match up exactly", this isn't entirely true, what you are doing is causing an MTU mismatch between your machines, this will cause lots of weird issues because you're dropping packets. If you have one machine's MTU set higher than the other's then you'll have the exact symptoms you are describing. Try setting all of your MTUs or Jumbo Packets to the same size.
Oh, one more thing. The XS712T has a bug on the new firmware when you run LACP on the SFP+ ports. Sometimes the balancing algorithm will decide to spew lots of heartbeat frames and essentially bottleneck the connection. I haven't seen this happen on the copper ports, but who knows. The latest firmware also quiets down the fans too :D
I get 100% network utilization and the full 1gigabyte per sec network transfer speed. I have a Dell T5610 with the same INTEL NIC card mentioned in this video. I frequently transfer single files that are over 100GB in size to an 8 drive Synology Diskstation running a 24 TB RAID 5 setup(7 drives and 1 hot spare). The individual hard drives are the WD RED models. It took quite a bit of tuning and research but it is totally possible to hit 100% network utilization and ultimately achieve the performance you are looking for.
I am liking it Linus but I am asking you to put some time and effort into it and put a glass top on that desk and actually make it look nice to fit that beautiful new office I wish I was working in.
Do you know how old-school 'ttcp' is? hint: the original code was written by the guy who wrote 'ping'. I'm not sure if it's still a thing in modern versions of Windows; I haven't used the OS in an administrative capacity since XP, but there was a network service called "QoS RSVP'" that was single-handedly responsible for opening up the pipes over high-bandwidth connections. If that service has been turned off for some reason, it may be why you needed to twiddle with TCP parameters, while others get satisfactory performance 'right out of the box'. I quite enjoy your 'As Fast as Possible' videos: It's not at all easy to condense technical topics you do into ADHD-sized bites. But to do it in such a way that preserves the integrity of the topic is genius. Cheers, -nevin
I thoroughly enjoy these prosumer videos. I feel like it's a higher tier of youtube that almost no one else does (well enough). Great work and I hope you can figure out to untap the full potential of your beast network!
Two questions: #1 "More higher", can you really say that?? And #2 A single SSD may be limited to the 600MBps limite but what if you are running RAID?? Genuinely curious...
A friend of mine shared one of your videos two days ago on facebook, and i'm slowly falling in love with this channel. Just awesome stuff here. I might actually try to assemble something as epic as you guys do. PD: I'm planning to build a render farm on the next year. Basically multiple boards connected any tips?
Haha, I'ts fun to remind ourselves how things have changed since 2015 : ) At home I'm using Infiniband FDR for the vertical (and servers/SAN) and Infiniband DDR for the horizontal to the 7 clients (mellanox cards). It's all over QSFP connectors. The theoretical 20Gbps DDR connection to clients is only giving about 14-16 Gbps and the vertical plane is theoretically 56Gbps but we're only seeing about 50Gbps of that. The primary uplink is a Hyperoptic 1gigabit synchronous for which we're only paying £45/mo. _(about $55 USD / $73 CAD)_.... the uplink is giving us about 900 reliably, both ways. I wish I'd just opted for straight 10Gbe ethernet to the clients. Also, the SAN can only push about 40Gbe max which is a shame. The servers are twin Dell m1000e's with 16 blades and 8 blades respectively... these have local gigabit full-mesh interconnects, separate management net and mellanox IO modules to the SAN/WAN I made several mistakes with the build. Sometimes I hope the house would burn down so I could start again... ... I'm insured though... so... y'know... if anyone out there has a balaclava, some matches and a little free time in the evenings - I'd love to hear from you ; ) LOL
My studio has a 10 Gigabit Network, all use macs with thunderbolt connections. Done. And we actually do get 10Gigabit connection speeds, unlike Windows machines. don't get me wrong having a modular system is great when you want to make a cheap powerful machine for compiling data, but modular systems have their weaknesses same as Macs. Some of those weaknesses just happen to be fragmented network capability.
![MISSING: SLIM SHADY [Expanded Mourner’s Edition Trailer]](http://i.ytimg.com/vi/xh8qgHrOO1g/mqdefault.jpg)
Not sure how people watch this video and come to the conclusion that they have a 10Gbps link to their ISP.
The 10Gbps speeds are just for their internal network. Their "Internet" speed I imagine is just standard business fibre at 1Gbps
+NoZephy I do remember on one of Linus's videos, he said "200Mbit up and down"
Yeah the amount of people in the comments comparing this to their internet speeds is too damn high xD
+NoZephy Standard business fibre at 1Gbps? HA ! Maybe in the Eastern world.
Casey Stellar
Well yeah it depends where your business is located but business's get the option of paying outright to get a leased connection installed, even if it is ridiculously expensive.
NoZephy It just isn't common here in the states.. Maybe California but, businesses typically get shit speeds countrywide..
It's crazy to think that in 4 years, 10GbE has become so affordable.
Even 40Gb isn't too bad, with used equipment, though the cables are exxy - planning to go 40Gb between my servers, and then 10Gb to my desktop.
@Bald new you pretty much get what you pay for - want better than 1050ti you'll need to pay for better than 1050ti. A used GTX 970 should be similar in cost but a lot faster. Slightly more for a GTX 980.
Well it is now
@@ruripapi that's literally what I said...
Then look now
DESK COMPUTER BUILD LOG HYPE
Came out like 20 min ago
+Ben G I don't see it (vessel doesn't count)
+Bainanaz I actually pay monthly
+Bainanaz It's cheap per month ,plus better compression when downloading
+Ben G AH OK
Show me that desk/pc build
It's coming
+LinusTechTips lol i know, and its going to be insane
+Cory Allen Vessel.
+Cory Allen Ha, I already saw it on vessel!
+LinusTechTips Btw Linus, what do you think about a video comparing nvidia Shield streaming to the Moonlight stream app on a random tablet/phone that is compatible with nvidia gamestream?
We want *Scrapyard Wars 3!*
+BloodMaster If you don't know. They're working on it.
+BloodMaster and austin is on it as well!!! this is better be good
Bloode obozavam tvoj kanal
+BloodMaster Odkud ti tu :D?!??!
+BloodMaster Saw on his twitter that they finished shooting the finale today! Gonna start soon!
I'm surprised that the video is again washed out :/
You guys have a problem with a monitor calibration or someone doesn't know what they're doing...
It doesn't destroy the video, but it's not very pleasing to watch :(
Edit: Mistakes happen, that's normal, but I would hope you guys get it under control soon.
+sawyer ptak Dude, it's a legitimate criticism. Fuck off.
+Queen Bitch KaNoMiko your right, after the intro at 0:45 it is particularly washed out.
Washed out for me too on all my computers, pads... please fix the lighting situation
it's not that it's washed out, it's confirming video levels for the web.. the have color calibrated monitors and use certain colors so that it's easier to distinguish whites and Black's and details in the shadows and highlights are preserved.. otherwise you lose data in the video and lose video quality as well
+jack black It's actually a lack of contrast. Most people set up their cameras to shoot as much latitude as possible to give more headroom in color correction. This has probably just been rushed out. Skin tones look a little weird, probably down to LED lighting which can ave a really poor CRI. Tbh, it's not worth complaining about, but the tech world is filled with pedants.
I'd love to get some 10 gigabit between my PC and NAS, but I'd need to upgrade the card in the NAS, my switch, the cabling, and the NIC in my PC - so essentially every piece of hardware involved :P lol
Anyone know where exactly they get their stock photos from? So gold.
Yeah
I've been really waiting for some coverage on this topic.
I picked up a couple SFP+ cards and a cable to run between my 2 main studio rigs and am setting up a 10gbe connection between them, but I've yet to find an "affordable" 10gb switch for my budget, haha.
the people who visit this channel are already kinda into tech stuff, and yet so many of them don't even know what this video is talking about and start comparing their internet speeds to this, makes one wonder what knowledge the general public has on these kind of stuff.
+footwearfoot I feel they are either all trolls or did not watch the video. lol
they must not know the difference
First, this is NOT an attack on your comment. It always amazes me also at what some people are totally oblivious to.
Not everyone who watches this channel has a computer tech ability at all. Many watch this channel to try to learn about specific technologies to increase their knowledge. So yeah, the general public may be into using computers, but many have no working concept of how or why they work. Many people do not even understand the differences between RAM and what a Hard Drive is. It is all storage in many people's minds. Add SSD, which really is memory, and you have completely lost them. Forget about even more basic concepts like BIOS and what ROMs are.
It comes down to what a person really wants to do with a computer. If they have no interest in programming, they typically will never learn basic programming concepts. If they have no interest in how the computer works inside, they often will never learn what the basic components and terms are. If you just drive a car for transportation, you may not know what size your engine is, how many cylinders it has, or even what type, or where the oil goes! It is human nature to not be concerned about what does not matter to you. Who knows what they ate for breakfast 2 weeks ago? Who knows their cholesterol levels? Who knows their blood type? How about all the birth dates of everyone in the family? These are all basic points of information, but unless a person actually cares about them, it is unlikely they will know the answers.
Years ago, the government was horrified at the revelation that people were so illiterate, that many did not know how to even set the clock on their VCRs! It gets very old to set a clock all the time, you do not need, since you may never use the VCR for timer recording. Especially when power bumps occur and it has to be done again EVERY time! People did not know how to set the time, because it did not matter to them. Many people are the same with Microwave Ovens today!
At the same time, many of these people have amazing skills at things I would never want to know, such as being medical physicians and molecular biologists (I think there is such a field right?), and even more basic needed skills such as physical strength and endurance to build and maintain our roads and freeways! We are all different, and the best goal is to help each other as needed, and be willing to also accept help from others. We can all learn, and we can all teach. It just depends on what the topic is at the time.
I think most come here expecting one thing and just start tearing through the comments without listening to what Linus is saying.
why do colors look washed out? is it just me?
+fcoronel1 cause of the green screen?
This isn't green screened.
+fatbloke have you watched the moving blog m8 it ain't green screen there not using them anymore and actually haven't in ages
yea probably cause the background is too far back that it has kinda the same effect as a green screen
rendering in premiere will cause washed up colors sometimes
why is everyone going on about internet speeds? he is mainly using it for LAN. So production guys can transfer/work with large files from the fileserver.
Exactly.
+Tyler Taupe It's like watching one of those powerline adapter reviews. comment section. "CAN I GET FREE INTERNET WITH THESE?!?!?!"
norge696 Yeah, I have never seen any videos about them but obviously the uneducated would assume really dumb things.
Have you tried disabling UDP/TCP offload and Large Send Offload for IPv4 and IPv6?
I know I've had similar problems with Hyper-V where TCP/UDP offload severely throttles network speeds because it's offloading work to the CPU.
Disable TCP Checksum Offload (IPv4 & IPv6)
Disable UDP Checksum Offload (IPv4 & IPv6)
Disable Large Send Offload (IPv4 & IPv6)
Disable IPv4 Checksum Offload
+Tom Martin If that offload isn't buggy as hell, turning it off should *slow down* the network.
Great tips! Helped increase my 10Gbit speeds from 360MB/sec to 1.1GB/Sec. Keep it coming!
holy christ.
Network speed =/= internet speed.
Network speed is the data rate between their in house servers and computers, i.e. moving videos from one machine to another.
Internet speed is what you use for downloading crap off the internet.
yes
JgHaverty 3
i thought wifi = internet and now you're telling me even network speed =/= internet. Gosh i need to process this xD *sarcasm*
Holy Christchurch!
DxCBuG Can I use this to download more facebooks?
2019 and finally I can afford 10 gigabit switch.
And now surplus SFP+ NICs can be had for
Go get 48 port SFP+ switch.
0:23 Every time Linus says something like "on the subject of" or "speaking of" I am conditioned to think it's going to be about a sponsor.
simbot15 its his job
This is awesome, I'm currently in the middle of putting together my own fast Home network! Thanks for all of the tips and tricks!
10Gb networking really is quite tricky to get right. You are on the right path with the receive side scaling (if you do not use that, all networking is done on core 0, which coincidentally is also the one used mainly for the OS itself, and even with the latest CPU, you won't get more than about 4Gb/s out of one core alone). Since you are using a 2 port card - have you made sure that the receive queues do not use an overlapping CPU core set? You can set a max processor / base processor number for each NIC, try using something like 2 to 8 for one NIC and 10 to 16 for the other for a first test and 4 receive queues for each (only even numbers=physical cores count). With Intel drivers, you should also be OK using all the offloading features. If you are using jumbo frames, make sure they are active everywhere (even your NAS). Also make sure flow control is active on the cards and on the switch ports, it helps a bit. Does your NAS support SMB3 if you're using Win10 / Server 2012R2 clients? With the exact same card you are using, I am getting live migrations with 2x9.8Gb/s utilization between Hyper-V hosts (SMB3 multichannel over two ports in different VLANs), so it is definitely possible :-)
and here i'm sitting at 10 Mb/sec...
+TheAlienpope 4mbps reporting in
Here I am with 2mb
+Sam Bilney 12mbps here.
30mbps :/
+TheAlienpope 10KB/sec Dial Up reporting in lol
We are waiting for the 40Gig upgrade😜😁
Does this switch have shared or dedicated port buffers? And it's probably not non-blocking ( overbooked backplane ). Also watch out with jumbo frames. It can improve throughput, but I have seen very wierd cases with iscsi and nfs where packets were dropped because of unexpected packet sizes.
In your case with native SMB3 please check the tcp window frame sizing. You can tune your nics with netsh.
Linus, I'm gonna go out on a limb here. Maybe the machines that you were experiencing slower transfer speeds had a cable length/distance of greater than 100m? Limitations of CAT6A/7 with 10GBaseT.
Hi Linus, I remember I made some tests on my 1 Gbit Local network and I figured out that pulling data is faster than pushing. this said in my own words describing what I figured out. an example : you are on machine 1 > cut or copy a file from machine 2 and paste it on machine 1 ( pulling) is much faster than (Pushing). Pushing > you are still on machine one > cut or copy an item from machine one and paste it on machine two while you are still on machine 1 ( pushing to machine 2) is slower
People in the comments need to understand this isn't internet speeds. It's their internal network from their group of computers. Basically used for fast file transfer throughout the office.
FYI: Jumbo Frames is a horrible creation. It scales very poorly, especially with respect to switch resource usage, and it gives you no benefit when you are transferring small files. Naturally the overhead is multiplied greatly whenever packet loss occurs congesting what is already an unstable link even further. In theory the maximum difference in performance on a 10Gb link with or without Jumbo Frames for large payloads would be in the area of 5-6% meaning that the maximum benefit they should give you is 75 MB/s faster transfer. While that is a noticeable difference it does not explain what you are seeing. I would wager that the other changes you made to the NIC, number of CPU threads and to a lesser degree the send buffers, had a much larger impact.
I love how he just ran at the servers with a tub of ethernet cables...
You guys should do more networking videos. I think you guys are very entertaining and believe you could bring some humor to the subject that could make it less painful.
The one thing I can think about with having this much speed is virtualization especially if you want to stream enterprise apps to all of your users as well as doing a NAS setup.
Anyone else notice he unplugged something from the testbench around 5:22?
yeah he did
literally read your comment right as it happened. slides it out of the way like a pro and keeps going. lol
Traetuus s
yea lol.
I was reading this and missed it happening lol
Ok but.. where is my DDR8?
2:56 Isn't that Doc's drawing of the "Flux Capacitor" in Back to the Future ?
We have had 10 gig for years @ work, using sfp+ and om2 multimode fiber. If we are transferring lots of tiny files, the performance is never very good, single giant files are always fast. I'd check your server to see if any kind of read-ahead and write-back are enabled on the array. Stripe size of the raid(s) can also make a huge difference when writing. For giant files like you describe, I usually make an array w/512K stripes. The higher stripe size always seems to affect write speed, not so much read. I took a local ssd array and changed it from 64 to 512 and got 1600 - >2200 MB/'s difference on writes. Anyway, good luck on your 10gig network!
I have always been told that networks at that speed can not be actually handled due to the length of the able, if you ping one PC to the other you will get
Evil Vessel spoilt Everything!!!
since when?
Then don't go to Vessel? I'm perfectly happy watching Linus videos on youtube, delayed or not.
+Xuvial Sylvester You're happy not directly supporting Linus?
Lumia Vessel is shit and their mobile app sucks balls. I support Linus through youtube and the majority of his subs are doing that.
+Xuvial Sylvester I like the vessel app more than the new RUclips layout, considering that it is a band new service it will only get better with time. also no need to deal with Google bullshit
Watching Linus run with the tub of cables. 🤣😂🤣
"They are both somewhat affordable" 300$ - Damn it linus!
It wasn't that long ago, that we were happy with 56kbit/s on dialup connections, and the local lan was 10Mb/s
I don't care about video quality, I come to this channel for information quality. I work at best buy, so what I gather will go towards helping others that come to my store.
10gigabit is WEAK i want to see 100gigabit! Its not to expensive
+me eq NOT_GRAMMAR_NAZI
call lol wow just wow
I WANT 10000Tbit/SECOND!!!!!!
all jokes aside, it would be cool to see 40 Gbit/s networking :D
they have 1tbps in my time xD
PSHT... 100Eb/s, bitches.
GOTTA GO FAST
UR 2 SLO!
I think I see your problem Linus, you're using Windows.
Yup, that's it.
that a good one... shame you have to *pay* for it.
pun intend
Me and my friend were amazed in 1994 November when we both got 28.8 modems. He came from a 14.4 and me a 2400. We were so excited to see the numbers spin on a file transfer. Then we found IceZmodeM and you could type to each other during file transfers!! TYPE. TO. EACH. OTHER!...So amazing.
Just to be clear for some people on the comments section:
#1
- 10 Gigabit (10Gb)
- 10 Gigabyte (10GB)
Are not the same, bits are mostly used in transfer or internet speed while bytes are used mostly for file sizes.
#2
This video is about the Local Network used just in the office. For example, the video editors sending their projects to the server(s) for backup purposes. This has Nothing to do with their internet bandwith. Linus never mentioned their ISP or their internet connection.
Sooo the comments like "And I'm just sitting here downloading at 512KB/s" are a little out of place.
Still they have decent bandwith in the office, I believe it's 200Mb/s upload and download.
(Sorry if my grammar is bad, English is my second language)
Internet speed ≠ local network
CreativePixels95 what's your point?
How many Mbps does your network run at?
ITS OVER NINE THOUSAAAANNDDD!!!!!!!
Its my bday, can I be on top?
+RunHao Lin I don't know what would be more surprising. Your comment ? Or if that's exactly what he/she wanted ?
still miss the kitchen but it's great that you guys have grown so much since I've been subscribed to you guys!
the brightness of this is really high, linus black shirt looks grey-ish instead of normal black
Why do you not try Linux on this setup?
(Because he doesn't know anything about computers, as evident from this video, and the channel is a commercial.)
In a city in NZ you can get 1GB/s internet speeds
and in some places in that town its up to
6GB/s
Luke Jamieson bullshit
That's horseshit.
which city brother
I'm in Auckland and I have gigabit lol
lol you guys on drugs we have it here in the us, xfinity has it for a fair price+
So lets see, the comments thus far, the video is washed out, this is something that people care about obviously because they incessantly go on about it. The next group can't comprehend the content so they ramble on about what they think it's about then another group has to explain to them what the actual content is about. Then you have the trolls who have such crappy lives they literally post stupidity where ever they can so they can show the world what an idiot looks like. I don't know how Linus does this for a living, the idjits out number the people you want watching your channel it seems.
My take on this video, it was well done as always, I could care less if it's washed out a bit, the channel and equipment are a work in progress. I don't watch Linus for perfectly done videos, I watch for the content. As for 10 gigabit, I'm hoping to make the move so at least my server and my main workstation have 10 gigabit connectivity and I hope to attain at least 500 megabyte transfers. I've been happy with 110 to 129 megabyte sustained transfers (the max Gigabit can hit) but ever since the move to NVMe and SSD's in the server, I need the speed!
+KlipschHead281 The only things these problems prove is Linus is getting popular. Who knows, he may advance to the point of disabling RUclips comments altogether
Ahem .. cough pewdiepie cough
MarioDragon I doubt anybody gets that far. :)
+KlipschHead281 A few of the top guys have...
MarioDragon Kudos to them, damn...
Dunno whether someone has mentioned it, but if the MTU on both ends doesn't match, you'll end up bottlenecking at the switch, cause it would need to repackage every single freakin frame (I've run into this multiple times with corporate clients). Also the first thing you do whenever you get above 1GbE is to setup jumbos and increase the Rx and Tx buffers. TOE also helps. The faster/slower transfer inconsistency with spindle drives is usually due to the block sizes discrepancies or lack of fine tuning (depending on the majority of files being stored it makes a HUGE difference whether you're running with 4K or 512K blocks), but since you're using SSDs and RAM disk, I honestly have no idea what might be the reason there.. Also if you're using Windows all across your infra and have enough ports, given the fact you have dualport cards and especially since they're 10X (meaning rj45), I would consider LACP or some sort of balancing trunking - LACP would be best for Win8+ and WS2012r2+ (completely new network stack on both). I could get into more detail if you're interested why LACP and so on, but I guess that's enough for now. Cheers!
What's crazy is that gigabit Ethernet cards in the 90's used to cost $1000, then one day in July of 2000 Steve Jobs went on stage and announced that it would be standard in all new PowerMac G4 models making gigabit networking adorable almost instantly.
Maybe using a Linux or BSD would work better? I would never trust Windows for performance...
(Also, I am aware that there are others, but those are the ones I know that are best for networking)
Yeah apparently you don't know shit so shut the fuck up.
teksyndicate got it working lol. phone wendell
And I just installed 10/100 Ethernet. I didn't even know there is 10Gbit/s until this Video :/
Thats the kind of vid i subscibed for, detailed, visual walkthrough, real world results. THANK YOU !
When you say "cat 6a cable" at 3:52 it makes my google home go nuts.
And here I am with 3Mb internet T-T
+SIKORO #5 You should really be doing better than that in Germany of all places. Even 3G Mobile is faster than that O_o
I do have 20 symetric :p
but this video is about 10 gbits speed in their own networking (just betwen the computers at their office) not their internet speed (ips)
also to mention not to confuse gbits with gbytes
This isn't internet ISP speeds, this is internal network transfer speeds over LAN.
+Random_Red 100Mb/s internet...
imagine an ssd with a 1 tb cache
so... an ssd with another ssd as cache?
the said ssd should be as fast as ram or faster for it to be used efficently as cache which in this case is pretty much useless since ram for storage is already faster than any known type of storage
Levi Lapsley Gimme that cash/cache XD
(Not the same of course)
Intro: a byte is 8 bits not 10... (Kind of strange... Or is the remaining 2 gigabits for sending addresses and stuff?)
Plus it's 1,024 megabytes to be a gig.
+New American Fishkeeper funnily enough it actually isnt. You're referring to a 'mebibyte', but everyone calls them megabytes
@New American Fishkeeper
that's not a gigabyte, that's a gibibyte
Not 10, because 1->2->4->8.
It's more likely because you always have a certain overhead on any network. So you would never ever get 100% theoretical value. Hence why 80% better reflect what you can expect in real life
Hey there, do not try to cut the files because if you do that, the original file has to do two things at once , it has to transfer the file and clean it from the original PC.
The transfer speed depends on the ratio between the TCP package and the file length.
Note: remember that the network cards and operating systems have buffers , so that's the difference when you transfer files from one place to another.
Linus, not sure if its the extra confidence or space the new office has give you - but dam - you presentation skills are on fire, 30% better I would say - love it
Is it me or does Linus talk really really fast in this video
He does, and his voice breaks more than my rj45 clip switches.
He always talks fast, especially when he gets excited!
1Gigabyte has 1024 not 1000MB !!! plzz thos 24MB count!
+Fify Lv 1000*
actually they changed the standards like 17 years ago, 1 Gigabyte = 1000 Megabytes, but 1 Gibibyte = 1024 Mebibytes
Wow didnnt know that :(
+A CuteTeddyBear but gibibyte is
***** I said Gibibyte, not Gigibyte...
relax dude your going to bust a vein
Most of the times I have no idea what he is saying. But I love watching those videos !
Linus, do you think that the length of the cable run from your test bench to the server/switch room had something to do with the variable speeds? Have you tried it right beside the switch? Maybe the quality of the connectors used in the network installation might contribute.
I mean 1GB/s in general would be freaking awesome no doubt, but the ~500MB/s you get sometimes are quite good as well. I can't really remember when I did such big transfers that often. Ok, sometimes I copy films on my Home-Streaming PC and my network is running on about 300MB/s, It usually doesn't take longer than 10 minutes, which I think is quite OK. Especially when I do that 3 to 6 times a year.
And yeah, I get that your channel is all about CRAZY STUFF haha :D, but as I said in general I would be happy with ~500MB/s.
@LinusTechTips
Make sure both the PC & server is set to high performance mode. I'm not 100% sure if you can tweak the SMB buffers on windows, but for my FreeNAS based server with bonded 10Ge, I needed to raise the SMB buffers significantly before I got good throughput.
Also, do note that because SMB is single threaded it has interesting behavior to say the least. You'll notice that if you have a client doing multiple transfers at once, you'll be limited to the slowest connection. So let's say you have a theoretical 700MB/s limit, but for some reason one transfer is going at 100MB/s, the other transfer will be capped at 100MB/s on the windows client. If the slowest transfer goes up, as does the other one.
Once that was done, and the client machines weren't in low power states, I was capable of pushing 700-800MB/s on spinning disks from our ZFS servers.
Linus mid 2015: 1 gigabit
Linus 2018: 10gigabit
Linus 2021: 100 gigabit
Linus 2023: 1 terabit
I need a serious upgrade to my caring. I'm still haven't installed caring 2.0 yet. ...and that leaves me with a blissful lack of caring that allows me to not care,..at all. And that's something I care very deeply about. I'd say thanks for the video if I cared. I'd wish you peace and health if I cared :)
The issue you're seeing is because of the inherent issues with TCP/IP.
The reliable and consistent performance you're hoping for can only be seen with SAN systems with direct connect 8Gb/16Gb fibre channel from the client to the storage and an outboard metadata link connected via 1Gb or 10Gb. (Metadata doesn't need much in terms of speed.)
For how much you spent, if 500MB/s is the worst speeds you are getting, you're doing great.
Packet size is negotiated which is why it didn't matter, but the switch has a maximum packet size which is why that matters.
For SUPER cheap, home 10Gbit, use CX4 cables/switch/card. Find a used switch like a Woven LB4 for $200 or less. For Linux, use a Myricom card which should be $50-100 on Ebay each. For Windows machines, go Intel NetEffect NE020 for, well, about the same price. Last I checked, the Myricom had no Windows drivers and that particular Intel no Linux drivers. (But this was a few years ago.)
Woven LB4 has 4xCX4 + 48x1Gbit.
Note: Woven LB4 is LOUD.
Nice. I was so worried about my old cat5e network when i got my 300mbit internet but it could handle it. Even tho most of the internet is lagging behind (pretty much only steam can do those speeds) its clear that you shouldnt invest in less than cat6. These very high speeds also question if you really need large local space as you can pull data so fast from afar that itll feel like its "local on a old hdd" or similar.
The variation in the speed are due to the caching in memory. When you are reading from a server, to your machine with a lot of memory, you are actually writing to a cache in RAM, and OS writes back in background this data to the harddrive (or ssd). This means ssd/hdd is not a bottleneck. If you would actually copy very very big file (bigger than your memory), you will see the decrees in speed. The same happens when you read data back in opposite direction. You will be only able to read at a speed of sata3 interface, and not faster, unless it is already cached. But I guess, Windows is cleaning the cache in the background quiet quickly.
Imagine a glass desk top with the PC in a case underneath. THAT would be awesome
I don't know the details of the server hardware configuration, but if my understanding is correct he's running FreeNAS, whith ZFS, on a hardware RAID array. That may be adding some IO delay. Enabling jumbo frames on the network would be the first step as he did, but I don't think the bottleneck was ever on the network because with wire speed switching you should have no trouble achieving max throughput.
oh yeah if your frame/MTU sizes are different, the individual frames will get broken up along the way and have to be reassembled, this will add tremendous overhead and in that case the bottle neck would be the network.
10Gbps networking has been in my roadmap for a year now. It sucks having to wait 3 hours for a file to transfer at 1Gbps speeds. I prewired with cat6A when I was doing the runs to my server.
For those who won't need RJ45-specific cabling, then I just picked up an Emulex P004096-01K network card with 2x SFP+'s for $50, and a 48-slot Quanta LB4M switch with 2x SFP+ uplinks for $95
That's $145 in ebay money (excl. ship n tax) for a cheapo 10Gbit setup.
The issue may be your MTU sizes (aka Jumbo Packet size), you mention "it doesn't seem to matter if your values don't match up exactly", this isn't entirely true, what you are doing is causing an MTU mismatch between your machines, this will cause lots of weird issues because you're dropping packets. If you have one machine's MTU set higher than the other's then you'll have the exact symptoms you are describing. Try setting all of your MTUs or Jumbo Packets to the same size.
Oh, one more thing. The XS712T has a bug on the new firmware when you run LACP on the SFP+ ports. Sometimes the balancing algorithm will decide to spew lots of heartbeat frames and essentially bottleneck the connection.
I haven't seen this happen on the copper ports, but who knows. The latest firmware also quiets down the fans too :D
I get 100% network utilization and the full 1gigabyte per sec network transfer speed. I have a Dell T5610 with the same INTEL NIC card mentioned in this video. I frequently transfer single files that are over 100GB in size to an 8 drive Synology Diskstation running a 24 TB RAID 5 setup(7 drives and 1 hot spare). The individual hard drives are the WD RED models. It took quite a bit of tuning and research but it is totally possible to hit 100% network utilization and ultimately achieve the performance you are looking for.
You are the best friend of starter I.T. engineers, Linus. We love you
I am liking it Linus but I am asking you to put some time and effort into it and put a glass top on that desk and actually make it look nice to fit that beautiful new office I wish I was working in.
i don´t understand a thing he´s saying, but i just love watching these videos and listen to thousand complicated words he says :D
Do you know how old-school 'ttcp' is? hint: the original code was written by the guy who wrote 'ping'.
I'm not sure if it's still a thing in modern versions of Windows; I haven't used the OS in an administrative capacity since XP, but there was a network service called "QoS RSVP'" that was single-handedly responsible for opening up the pipes over high-bandwidth connections. If that service has been turned off for some reason, it may be why you needed to twiddle with TCP parameters, while others get satisfactory performance 'right out of the box'.
I quite enjoy your 'As Fast as Possible' videos: It's not at all easy to condense technical topics you do into ADHD-sized bites. But to do it in such a way that preserves the integrity of the topic is genius.
Cheers,
-nevin
I thoroughly enjoy these prosumer videos. I feel like it's a higher tier of youtube that almost no one else does (well enough). Great work and I hope you can figure out to untap the full potential of your beast network!
Haha I remember when the ad spaces actually made sense.... "speaking of computers... Dollar Shave Club! Wipe you're butt like a gentleman :D" xD
Disk Sector Alignment Linus, Disk Sector Alignment. Do it on your Raid 50, and you'll see proper throughput.
Two questions: #1 "More higher", can you really say that??
And #2 A single SSD may be limited to the 600MBps limite but what if you are running RAID?? Genuinely curious...
A friend of mine shared one of your videos two days ago on facebook, and i'm slowly falling in love with this channel. Just awesome stuff here. I might actually try to assemble something as epic as you guys do.
PD: I'm planning to build a render farm on the next year. Basically multiple boards connected any tips?
Was just going to point out that more than 10Gbit has existed for a long time, but you mentioned it :)
Haha, I'ts fun to remind ourselves how things have changed since 2015 : )
At home I'm using Infiniband FDR for the vertical (and servers/SAN) and Infiniband DDR for the horizontal to the 7 clients (mellanox cards). It's all over QSFP connectors. The theoretical 20Gbps DDR connection to clients is only giving about 14-16 Gbps and the vertical plane is theoretically 56Gbps but we're only seeing about 50Gbps of that. The primary uplink is a Hyperoptic 1gigabit synchronous for which we're only paying £45/mo. _(about $55 USD / $73 CAD)_.... the uplink is giving us about 900 reliably, both ways.
I wish I'd just opted for straight 10Gbe ethernet to the clients. Also, the SAN can only push about 40Gbe max which is a shame.
The servers are twin Dell m1000e's with 16 blades and 8 blades respectively... these have local gigabit full-mesh interconnects, separate management net and mellanox IO modules to the SAN/WAN
I made several mistakes with the build. Sometimes I hope the house would burn down so I could start again...
... I'm insured though... so... y'know... if anyone out there has a balaclava, some matches and a little free time in the evenings - I'd love to hear from you ; ) LOL
99% of this went completely over my head, and I loved it
This RUclips channel is good because this guy shows that homosexuals can do more than just interior design or hairdressing.
My studio has a 10 Gigabit Network, all use macs with thunderbolt connections. Done. And we actually do get 10Gigabit connection speeds, unlike Windows machines. don't get me wrong having a modular system is great when you want to make a cheap powerful machine for compiling data, but modular systems have their weaknesses same as Macs. Some of those weaknesses just happen to be fragmented network capability.
Meanwhile im pausing this video for half hour so it buffers and let me see without stopping!