Latency Difference Between 10GBASE-T SFP+, AOC & DAC Cables - 1286
HTML-код
- Опубликовано: 21 авг 2024
- Was talking about DAC and Fiber cables in a video last week,, and it seemed that this was an subject, that many had an opinion and was interested in. So a little follow up video.
Where I found some of the info in this video : community.fs.c...
[Affiliate Links]
Link US - DAC Cables : amzn.to/3USrwAw
Link UK - DAC Cables : amzn.to/3H1dtTL
___________________________________________________________________________________________________
/ myplayhouse
For 3$ a month, you get an extra weekly "What's UP" update video. Just for my Patrons. The Support I resave on patreon is all used on stuff to make interesting videos on RUclips.
My PlayHouse is a channel where i will show, what i am working on. I have this house, it is 168 Square Meters / 1808.3ft² and it is full, of half-finished projects.
I love working with heating, insulation, Servers, computers, Datacenter, green power, alternative energy, solar, wind and more. It all costs, but I'm trying to get the most out of my money, and my time.
SFP28 - Cause it has a RAW bandwidth of 28Gbit/s but drops down to 25Gbit/s after adding encoding and overhead
Uhh nice,, I did not know that. Thank You!
SFP normally bring out a MAC connection to the cages, so you are adding the latency from the MAC to the PHY layer in the cases of the two SFP-based interconnects. So to be fair, you should also take into account the latency in the PHY's on the board that the RJ45's are used with...if this makes sense.....
That would be different from card to card,, This was just about the cables.
@@MyPlayHouse But it's not , because some of the NIC components are in the SFP RJ-45 10G transceivers.
Having said that, I've proven to myself that there shouldn't be any difference at all between a Intel NIC chip and copper wire regardless of the relative location and proximity of various components in the circuit.
Right?
On the NIC right behind the connection or in the cage.
SFP28 is named after the base data rate of the connection. It gets down to 25Gbit after the usual packet padding and other tags and so on that data gets added to it to get where it needs to be.
Hi Allison Pell
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
In case people don't know: there are replacement clips for rj45 jacks: search either for "Delock 86420" or I also found them as "RJ45 Hyper: Clip Repair Clip".
Thank You,, I did not know of those.
Thank you Morten for bringing up considerations for networking our Personal Confusers (and servers). 😂
You are welcome :-)
Using a 10GBASE-T SFP+ isn't allways the worest solution, it depends on the circumstands. In my case I wanted to upgrade a 1Gig connection between my main rack (standing in my hallway) and a room above my garage. Inbetween is a kitchen and routing a cable through a kitchen is really time consuming. The 1Gig connection is based on a CAT6 cable and could handle the 10Gig. Therefore the cheapest solution (counting my spare time too) was to use the 10GBASE-T SFP+ connectors. If there were no cable already there, I would have used a fibre cable 😄
If the equipment does not have SFP,, or if you need to go from 10 to 1 gig 10GBASE-T is good. It's not as if it does not work :-)
Good explanation of electrons moving through copper, I like to think of it as a pipe full of marbles, one in one end, one out the other.
That is a good way to see it..
8:47 another fun fact : Light traveling speed in a medium is related to refractive index of the medium
Humm what is the latency of a reflection,, does light just instantly go the other way :-?
Interesting question! I searched and found, on graded-index fibre, refractive index of the center of the core is higher than that of near the clad, and carefully designed to match light traveling time of zig-zag route with straight through route by adjusting refractive index gradation. Therefore light traveling speed is determined by the refractive index of the center of the core on graded-index fibre. On step index fibre, light with zig-zag route reaches late and signal is blurred
One reason for increased latency of a DAC vs fiber is an incorrect assumption about the speed of light in copper. The speed of propagation depends on the characteristic impedance of the line, which depends on the dialetric material, distance between waveguides and groundplanes, etc. Paraphrasing from another commenter, transmission lines are complicated.
I still think it’s weird 😳
Neither of these matter when we, on the receiving network card, run with typically 70us coalescing. This avoids interrupting the CPU at every packet, instead giving the NIC time to do its hardware acceleration things, and the CPU can process packets in bundles.
The reason for both the low power and latency of fiber interfaces, and the high same of 10GBASE-T is bandwidth: Fiber has oodles of it, and the bits can be shoved across as-is with bandwidth-inefficient coding. 10GBASE-T is up against the limit of the cable, and requires complicated line coding to make it work; There's a presentation from Broadcom in 2006 showing the crazy complexity of a 10GBASE-T 'modem'.
So the network card is kind of caching for the CPU. ?
@@MyPlayHouse It's doing more than that. It will read data from main memory (it goes across PCIe and through the CPUs memory controller), chop up data into packets, add headers, and calculate checksums on transmit, and verify checksums and assemble packets into larger blocks on receive. Not just on Ethernet level; the NIC understands IPv4/IPv6/UDP/TCP/GRE enough to do this. It writes the result into main memory where the driver wants them, and only when all that's done does it notify the CPU. The quad-gigs you used to sell also handle priority queueing and map traffic flows to CPU cores. Desktop NICs have less of this, as 1Gbps/2.5Gbps is relatively slow, modern CPUs are plenty fast, and this market focuses entirely on BOM cost.
Interesting!
Fun fact: there is lots of engineering on getting a fast electrical signal to travel over a wire. We tend to think of a wire as something that just outputs whatever signal you inject into it, but that's a huge oversimplification. Instead, the universe has a tendency to work in quite complicated ways: en.wikipedia.org/wiki/Transmission_line
Well I have difficulty enough, with the part that it is not the electron you put in, that is comming out :-)
I think the biggest factor to these connectors is the heat and wattage that they consume. This is especially true when you have full switches that are populated. For one or two connections, an extra 3-4 watts per port and btu is not a big deal, but when you have 48+... now that is going to be a big deal if you have to pay for expensive power rates and AC to cool.
It all adds up,, I have also seen big switches with just a few connections. :-/
The logic of your math presents a problem. A DAC is a 1:1 circuit connection, and no conversion is necessary. This contrasts with fiber, which does require conversion. Although it's not something measurable externally, nearly every switch IC engineer I've consulted advises using a DAC if space and cable length allow.
I did not come up with the numbers :-/
Very informative thanks!
Hi David Anderson
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Should mention not all DAC cables are direct copper. Some do have active components inside and convert to coax over the cable. Look up passive and active copper DAC cables.
True but they are more expensive than AOC or just SR SFP+'s and the fibre. Personally I hate AOC cables. You have heavy lumps of metal on the end of your fibre just saying break me now, and if anything goes wrong you are into replacing the while dam cable, where if you had just used a couple of transceiver and some fibre you would only need to replace the duff component.
Hi BlueFoxTV
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
AOC = Active Optical Cable :)
Hi @alexfischer9493
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
For Short Distances DAC cables are better as they are cheaper and no conversion is needed (From Electricity to Light). For Longer distances fiber with transceivers are better as copper wires have Resistance and the current Cat7a cable can do 10Gbps @ 100M max.
AOC Cables on the other hand stands for Active Optical Cable.
Advantages of DAC Cables :
More Cost-effective- Generally speaking, the price of copper cables is much lower than that of optical fibers. The cost of passive copper cables is 2 to 5 times cheaper than fiber cables of the same length. Therefore, the use of high-speed cables will also reduce the cabling costs of the entire data center.
Lower Power Consumption- High-speed DAC (direct attach cable) consumes less power(the power consumption is almost zero) since passive cables do not require a power supply. The power consumption of active copper cables is generally around 440mW. If you use direct attach copper cables instead of AOC fiber cables, you can save hundreds of thousands of kilowatts of electrical power.
More durable-It is designed with a seamless connection form of optical module and optical cable, which reduces the cost and ensures that the optical port is not exposed to dust and other pollutants. Therefore, DAC is less susceptible to damage.
Advantages of AOC Cables :
Lighter weight-An active optical cable is composed of two optical transceivers and a fiber optic patch cable, whose weight is only a quarter of the direct attach copper cable, and the bulk is about half of the copper cable.
Longer distances-AOC fiber can provide a greater and longer transmission reach of up to 100-300m due to its better heat dissipation in the computer room’s wiring system and smaller bending radius of optical cable.
More Reliable- Active optical cable is less vulnerable to electromagnetic interference since optical fiber is a kind of dielectric that can sustain a static electric field within it. The bit error rate of the product transmission performance is also better, and the BER can reach 10^-15.
Those SFP to RJ45 converters/adapters... Well they are useless in most cases. The transceivers gets hot and heating up the whole switch unnecessary. The proper way is to convert the signal is a proper media converter designed for this purpose.
Hi Victor Shane
Thank You very much! that was some assay :-)
The SFP to RJ45 works fine for connecting a management port like IMM,ILO from a server,, without having to add an extra cobber switch.
Thank you for watching! :-)
The SFP+ DAC (passive) should be a lot better because there is no signal processing in the transceiver. It literally connects the contacts from one sfp+ slot to the other via a copper conductor. That's also why it does not reach far beyond 5 m. I think you mixed up the numbers of the SFP+ Fibre and SFP+ DAC transceivers. All other methods have some sort of signal processing and or medium conversion, and that is the mayor contributor to the latency. The medium itself is mostly irrelevant in this case, the (long) distance has a bigger influence in latency than if the signal travels over copper or optical fibre. Of course signal degradation over very long distances and the need for repeaters do play a role.
So we have two contributors:
- Medium conversion
- Distance
But to be blunt, in a homelab, I don't think latency matters enough. I'd just avoid the SFP(+) to RJ45 transceiver modules, not just because of latency issues, but also because they get hot af and draw a lot of power.
I see up to 7meter DAC cables,, after that they become active AOC :-) Thank You!
@@MyPlayHouse Yes, that’s why i said „not far beyond“ and not „not longer than“.
Also, apparently active DAC (yes, copper!) cables exist that go a bit further, but of course don’t reach the fibre distances.
Hey Morten, Cindy needs some help with her solar system. Can you help her please? You are such a genius with these things. Thanks!
As you might have seen, I have been by to help her out,, and I believe I will be in tomorrows video as well.
Amazing videos. Im enjoying 40gb infiniband, also called qsfp+. I am a fan of fiber too because of its long distances
I have never done much infiniband,, but 40Gbit is fast :-)
@MyPlayHouse I have been watching your channel since 2016, I really enjoy the process you go through when trying to solve a problem. My father also enjoys your videos. We like when you say "virtual machine". I wish you the best, and you always have happy viewers watching.
Thank You very much,,, and why is it funny when I say "virtual machine" ?? :-)
@MyPlayHouse we like it when you say it. We know your channel from watching videos about vms. It is a way we remember you
Okay,, Thank You for watching,,,me say virtual machine :-)
Active Optical Cables: AOC😊
Hi @FarsiAli-fv7oo
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
@@MyPlayHouse my pleasure, Sir.... You're a tech class genius professor 😊🙏❤️
What are the power and heat numbers for each type..? This very important for cost and durability of the gbic and switch port.
I do not have anything to mesura that.
Another great video... See the youtube comments counter is not working showing 0 comments lol
That happens quite often :-/
Any body got any recommendations on a 10gb SFP++ Switch at least 8 ports
Mikrotik CRS309-1G-8S+IN
Seems like the one I have is not sold anymore :-/
Ubiquiti UniFi USG and US-16-XG 10G Switch - Challenges - video 653
There are two types of copper dac. Twinax and reg copper. What Are there latency and cost differences?
Hi @jeffselkowitz5842
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Modern bend insensitive fibre cable is not particularly fragile.
Well those thin fiber cables kink really easily :-/
@@MyPlayHouse then stop buying the 2.0mm ones and buy the 3.0mm ones. I would point out that you are not supposed to kink Cat6a cables and they have minimum bend radius too that are larger than bend insensitive fibre. I have doing fibre optic cabling in the data centre for nearly 20 years now, it is not as delicate as people make out. The main thing is to keep the caps on the ends. For LC connectors you want those clip on ones, not the tiny push on things that fall of as soon as you look at them.
I admittedly did not watch the whole video, but skipped to the part with the whiteboard that says Fiber is faster than DAC.
You may have covered this in your video, but if not - This is wrong. DAC will always be faster than fiber.
Over the distances that you can run a DAC, it will virtually always be faster (lower latency) than fiber.
This is because fiber requires the electrical signal be converted to a light signal, then back to an electrical signal.
This introduces more latency than the speed advantage that fiber has over the distances a DAC can be run.
Over long enough distances, yes a fiber would eventually be lower latency than a DAC - but the point is moot, because DACs don’t run that far anyways.
TLDR; For all intents and purposes DAC is always faster than Fiber
I also thought that,, but could not find any data to back up that.
Did you do any benchmarks or just the math?@@MyPlayHouse
I was always told that DAC is the lowest latency but i could be wrong. The advantage of fibre is that its thinner and easy to manage compared to DAC. Unless you work with " John loop the fibre" who uses a fibre patch lead 4m too long and loops it up and down the rack....... and then loses his console cable for the 1000 time.
I would also have expected that DAC would have had the lowest latency,, I can't get into my head that the other is faster.
Another thing is interference. Copper wires can be affected by high current power lines where as light on the other hand is not.
It is posable,, not usually a big issue.
Any Issues If you use a single sfp module with single cord connecting switches to OLT
Sorry I do not know... :-/
For comparison, on 10Gbps networking, 1500 bytes packet is transferred in 1.2 microseconds.
Hi yamamoto65536
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Isn't there a fiber cable that is more protected from breaking with a thicker shirt? I'm not talking about the 3mm variant.
Yes you can get thicker, and cables with like two wires side by side,, they are stronger.
These thin ones you have to be careful with.
What are they called so I can buy some?
Can someone please help me understand, when you have top of rack switches which are linked back to spines in spine and leaf, and you choose sfp28 ports on your top of rack switches: where does the servers ilo rj45 plug into? Do I need an sfp28 switch and rj45 switch separately just for ilo or do I convert some sfp28 to rj45 with converters?
My work uses all rj45 and I really want to move us to 25G with 100g or higher uplinks to spines but the ilo/idrac question is confusing me
You can put one of these in your switch,, and a cat6 -ish cable to your Ilo.
@My PlayHouse yes I think cheap rj45 sfp modules are the way to go thanks
You put a separate management switch into the rack. You could put 1Gbps SFP modules into your super expensive top of the rack switch but honestly, it would be a really daft move. We have historically cycled down old switches for that purpose but that won't work in the future as all the switches are SFP+ ports only😞
👍
Hi SKY NET CYBER SYSTEM 3 TECH
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Electricity doesn't flow in the wire. It travels in the electromagnetic fields surrounding the wire so the speed of travel is limited by the dielectric. Free electrons in a conductor actually only move about 0.5 cm/second or something. It's actually really all a mess. It's only at DC that something like 44% of the energy is carried by the electrons. Above 100 Hz it's all fields. If you thought electromagnetic theory is a disaster, you should look at solid state physics. Those guys must be clamoring to throw themselves off tall buildings. If you actually heard the "theories" they've come up with to explain free electron flow the scales would fall from your eyes.
Nothing is ever simple when you gig in to it,,, like if you think a compass points towards north,,, is does,,-ish :-)
somewhere out there must be a 40Gbit/s NIC for RJ45, because there are RJ45 de-embedded modules from metz-connect btr for this speed...
The cat8.1 will do 40GBASE-T,, I have never seen a card..
@@MyPlayHouse Cat6a will do 40GBaseT as well. Yesterday I found a posting that it's possible to cross-flash a Mellanox connext-X3 to 40G.
@@matthiaslange392 There is no IEEE 802.3 standard for 40Gbps Ethernet over Cat6a. It needs Cat8 with the special CG45 connectors and only goes up to 30m point to point, no patch panels or outlets are allowed. To the best of my knowledge, not a single piece of equipment has ever been released to the market supporting it and 40Gbps Ethernet is a dead end now, so it is unlikely there ever will be.
the connect-x3 has sfp ports, not rj45.
@@jonathanbuzzard1376 it's like SATA-express: some mainboards, but not a single drive ever released.
you forget the first 10 Gbase conection, cx4 connector :)
Humm,, think I have one,, but never used it!!
Do not believe fiber cabling offers lower latency compared to a DAC connection. I know for a fact that that is not the case 🙂 You did not measure your experiment, but based your conclusion on information provided by a single cable vendor.
I did see that they did not invent the numbers them self.
AOC: Active Optical Cable
Hi John G.
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
lol "It's not flat, it's round".
Did you lose any subscribers on that one?
(thanks for the videos, they great)
:-) I can't see for a few days...
I just use cat8 for my home network
Humm up to 40GBASE-T,, nice..
the DAC added latency comes from the fact that the SFP interface was made with optics in mind, and the DAC has to do some signal processing to convert it to electrical signaling
Hi zMeul
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
There is no signal processing in passive DACs.
aoc = active optical cable
Hi shep husted
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
@@MyPlayHousearen't all cable ls with transimeivers active? Even the dac does some conditioning?
10th
Your counting is ever so slightly off...