Mikrotik's a fantastic company. We had the honor of receiving the very first RouterBoard in the United States, thanks to CEO John Tully providing it to family members traveling in Riga Latvia while we were using their RouterOS extensively on Intel systems. Have fun with your switch.
2:33 "Mikrotik... never heard about them." - Woaw, that's what I didn't expect to hear from anyone who talks about networks to public :-D 5:37 and let's be clear here - When you use RouterOS and L3 routing functions you throughput downs to something about 300Mbps, so if you want fast L3 switch you have to spend more money. But when you use SwitchOS w/o L3 it is full 10Gbps. And, wait for it, they wrote it in docs :-)
Thanks for the clever comments :) I don't use the RouterOS/L3 functionality and as you pointed out its limitations are very clearly documented. I just use this switch as basic 10Gbe switch and for me it works very well for this purpose. It has many options that I like -- SFP+, low power, and most importantly its fan-less. I still have my InfiniBand Switch, and do turn it on from time to time. So, who knows maybe I'll go back to it at some point but that my friend would be a different video :)
@@vmexplorer Is there ever a time when a home or even power/enthusiast user would need L3 functionality in a switch? I remember learning about about L3 switches back in networking and TCP/IP classes but it was always mentioned in a large enterprise network context.
@@DuaLeaD It really depends on what you are trying to accomplish. For me yes, a L3 switch was needed, as I wanted to do VLAN/IP routing, ACLs, Netflow, virtualization hosts on different and discrete vlans, etc. The cost difference isn't to much between a L3 vs L2. L2 24 Port Gbe might be $20-50, L3 $40-100.
I discovered mikrotik about this time last year and was super impressed wt. The features you get on each device - i mean cmon - their 50€ routerboard has certain functionality for a fraction of the price of something like cisco. So if you dont need heavy duty datacenter switching and cisco reliability id go for these any day. Feature rich and the price is more than acceptable for what you get.
I've bought a CRS317-1G-16S+, witch is 16 SFP+ ports, and migrated my lab to it (It had 4x1GbE + 2x4Gb FC for SAN). The 4 port 1GbE cards were not compatible with ESXi 7 so it was time to upgrade. Bought some dual port BMC57810 adapters and it works quite well! "Pro" tip, SAN based SFP+ modules (that support FC 8Gbps) are compatible with the Mikrotik switch. Even thou they are made for SAN and 8Gbps, the module is exactly the same. It worked fine with several SFP+ brands, and they are much cheaper than Ethernet FSP+ modules, for some reason. Also, at least were I work, 8Gbps FSP+ modules are obsolete, so I have a lot to spare. Is there a way to flash those Infiniband Mellanox cards to Ethernet?
Hey Andi, sorry for the late response. The ConnectX 3 InfiniBand cards I use can be used for InfiniBand or Ethernet and they do this via an inherent auto detect mode, so no flashing or additionally settings needed.
Have you had any issues with the 10gb card shown in the video? My HP nc523sfp cards get *very* hot, even with the firmware upgrade, so much that I had to swap them out for single-port Intel 10gb AF XFP cards.
@@vmexplorer Oops, not sure how I missed that. BTW I also have an IS5022 and it works well enough with ConnectX 2 Pro cards, though I will upgrade my ESXI box to a ConnectX 3 because of occasional link speed issues. And I do have the 305 which is a great little switch.
@@klnnurv No worries, I went back to all my videos where these notes were and brought them to the top of the list. I like the IS5022, the only challenge was where to put the subnet manager. As I'm sure you know the IS5022 requires an external subnet manager, so I went with a MTS-3600 that was licensed to support an internal subnet manager and it worked like a charm.
Thanks. Aside from the learning aspect, I am just curious what you guys do with 40gbps or 10gbps networking at home. I use a gigabit switch, and periodically salivate at the prospect of buying a 10gbps switch, but I can't find any computational needs to justify such a move. Such a move will involve finding a NAS that supports 10gbps, replacing all spinning disks in my computers, NICs, and may be even the cabling (cat 6 now). Can you please elaborate on some use cases, and the complete infrastructure that goes with it?
Great Question! 40gbp/s InifiBand is like having a Ferrari in the garage that you never take out to track and 10gbp/s is a really good fit for my VMware vSAN Environment. I work for VMware and as a Technical Account Manager its very common that I get questions from my customers around how something works or how do VMware products work. I help VMware Embedded OEM partners built compute platforms that go in to the Industrial Manufacturing space. So yes, with all the VMWare products out there, I use the 10gbp/s quite a bit and I'm actually planning on hooking up my InfiniBand network to do some more RDMA testing soon. Check out my blog there's lots of links around building up VMware based Home labs and do reach out if I can help in any way. vmexplorer.com/home-labs-a-definitive-guide/
Thanks for the comment! I didn't realize my accent was British :) I have not done that setup nor explored it. I might recommend the MikroTik User forum or check out my buddy at vSwitchZero.com he did do a fair bit more with this switch than I. I hope this helps. forum.mikrotik.com/
Hello did you use any specific hardware when you connected the HP 10Gbe SPF+ to the Mikrotik ? as apparently i been advised it uses a different type of communication than the HP cable and will not work ?
Check out "ServeTheHomeVideo" channel they review a lot of mikrotik stuff. Im sure HP and Cisco are using some sort of proprietary ways of cabling or connection. Mikrotik should be less problematic regardin that.
I dont understand why people dont run fiber... I stopped using copper any place I can and just use LC to LC OM4 fiber and I never have to re run anything anymore and all i need to do is swap sfp+ adapters for faster speeds in the future. The OM4 fiber is same price as copper depending where you get it.
Thanks for the comment, and your right Fibre is a great option for Home Labs. For me I wanted to learn more about Infiniband and then went for a simplistic 10Gbe switch in this video. And, that's the fun of Home Labs finding something you want to learn about and doing it. Fibre is on my list, just not right away.
Umm kinda pointless for 30-40 you could get a connectx3 card that can do infiniband OR 10/40 Gbe . I got dual port connect x3 cards running infiniband on one port and 10Gbe ethernet on the other.
Thanks for the comment. You are correct about the ConnectX 3 cards, and I've run them for years now. For my home lab needs, silence and low heat are a must. I work from home and my home lab is located in my home office. So, I'm always looking for value, low heat, and low noise. I have the Mellanox MTS3600. Its normal fans are so loud, even with fan mods, its just to loud. So the point of this video blog post is to show an option where you can get 10gbe that is completely silent and at a good price point. I hope this clear is up, if not post up and Happy computing!
I could not agree more. However, for my purposes with a Home Lab that is right next to where I do customer calls - IB can be loud, hot, and if you are not accustom to it difficult to first start working with vs. this 10gbe switch has none of these issues.
Bingo, the InfiniBand switches I had access to were loud and run hot. I like to have my home labs quiet and cool as they sit right next to me while I'm working. Also, 40Gbe was nice but its like having a Ferrari the garage that you never use. My home lab rarely even comes close to maxing out 10Gbe.
By RJ45 to your Gbe Lan, do you mean the single RJ45 port that is part of MirkoTik CRS309-1G-8S+? If so, I use it for a management interface. I'm not sure of its other uses. Best to read the manual. More information found here - mikrotik.com/product/crs309_1g_8s_in
![NLE Choppa - Or What ft. 41 (Orchestra Mix) [Official Music Video]](http://i.ytimg.com/vi/biOFASEgSOM/mqdefault.jpg)
Mikrotik's a fantastic company. We had the honor of receiving the very first RouterBoard in the United States, thanks to CEO John Tully providing it to family members traveling in Riga Latvia while we were using their RouterOS extensively on Intel systems. Have fun with your switch.
Thanks so much for the post. It's been a great switch, in fact I bought a second one too!
2:33 "Mikrotik... never heard about them." - Woaw, that's what I didn't expect to hear from anyone who talks about networks to public :-D
5:37 and let's be clear here - When you use RouterOS and L3 routing functions you throughput downs to something about 300Mbps, so if you want fast L3 switch you have to spend more money. But when you use SwitchOS w/o L3 it is full 10Gbps. And, wait for it, they wrote it in docs :-)
Thanks for the clever comments :) I don't use the RouterOS/L3 functionality and as you pointed out its limitations are very clearly documented. I just use this switch as basic 10Gbe switch and for me it works very well for this purpose. It has many options that I like -- SFP+, low power, and most importantly its fan-less. I still have my InfiniBand Switch, and do turn it on from time to time. So, who knows maybe I'll go back to it at some point but that my friend would be a different video :)
@@vmexplorer Is there ever a time when a home or even power/enthusiast user would need L3 functionality in a switch? I remember learning about about L3 switches back in networking and TCP/IP classes but it was always mentioned in a large enterprise network context.
@@DuaLeaD It really depends on what you are trying to accomplish. For me yes, a L3 switch was needed, as I wanted to do VLAN/IP routing, ACLs, Netflow, virtualization hosts on different and discrete vlans, etc. The cost difference isn't to much between a L3 vs L2. L2 24 Port Gbe might be $20-50, L3 $40-100.
MicroTik makes excellent LTE dish routers! Huge feature sets for the price points of their products.
I discovered mikrotik about this time last year and was super impressed wt. The features you get on each device - i mean cmon - their 50€ routerboard has certain functionality for a fraction of the price of something like cisco. So if you dont need heavy duty datacenter switching and cisco reliability id go for these any day. Feature rich and the price is more than acceptable for what you get.
I have MT HapAC2 running 3dhcp with firewall rules each dhcp + hotspot + pppoe server 50 clients. No problem. 20-30% cpu usage.
I've bought a CRS317-1G-16S+, witch is 16 SFP+ ports, and migrated my lab to it (It had 4x1GbE + 2x4Gb FC for SAN). The 4 port 1GbE cards were not compatible with ESXi 7 so it was time to upgrade. Bought some dual port BMC57810 adapters and it works quite well! "Pro" tip, SAN based SFP+ modules (that support FC 8Gbps) are compatible with the Mikrotik switch. Even thou they are made for SAN and 8Gbps, the module is exactly the same. It worked fine with several SFP+ brands, and they are much cheaper than Ethernet FSP+ modules, for some reason. Also, at least were I work, 8Gbps FSP+ modules are obsolete, so I have a lot to spare.
Is there a way to flash those Infiniband Mellanox cards to Ethernet?
Hey Andi, sorry for the late response. The ConnectX 3 InfiniBand cards I use can be used for InfiniBand or Ethernet and they do this via an inherent auto detect mode, so no flashing or additionally settings needed.
Have you had any issues with the 10gb card shown in the video? My HP nc523sfp cards get *very* hot, even with the firmware upgrade, so much that I had to swap them out for single-port Intel 10gb AF XFP cards.
Yes, please see my notes in this video.
@@vmexplorer Oops, not sure how I missed that.
BTW I also have an IS5022 and it works well enough with ConnectX 2 Pro cards, though I will upgrade my ESXI box to a ConnectX 3 because of occasional link speed issues. And I do have the 305 which is a great little switch.
@@klnnurv No worries, I went back to all my videos where these notes were and brought them to the top of the list. I like the IS5022, the only challenge was where to put the subnet manager. As I'm sure you know the IS5022 requires an external subnet manager, so I went with a MTS-3600 that was licensed to support an internal subnet manager and it worked like a charm.
Thanks for the videos man!
You are most welcome.
Thanks. Aside from the learning aspect, I am just curious what you guys do with 40gbps or 10gbps networking at home. I use a gigabit switch, and periodically salivate at the prospect of buying a 10gbps switch, but I can't find any computational needs to justify such a move. Such a move will involve finding a NAS that supports 10gbps, replacing all spinning disks in my computers, NICs, and may be even the cabling (cat 6 now). Can you please elaborate on some use cases, and the complete infrastructure that goes with it?
Great Question! 40gbp/s InifiBand is like having a Ferrari in the garage that you never take out to track and 10gbp/s is a really good fit for my VMware vSAN Environment. I work for VMware and as a Technical Account Manager its very common that I get questions from my customers around how something works or how do VMware products work. I help VMware Embedded OEM partners built compute platforms that go in to the Industrial Manufacturing space. So yes, with all the VMWare products out there, I use the 10gbp/s quite a bit and I'm actually planning on hooking up my InfiniBand network to do some more RDMA testing soon. Check out my blog there's lots of links around building up VMware based Home labs and do reach out if I can help in any way. vmexplorer.com/home-labs-a-definitive-guide/
@@vmexplorer Thank you for the detailed answer. I will be checking out your blog too. Cheers
Your voice Sounds like Jackfrags voice i swear and also i got question,would i be able to hook up this switch with 2x2.5gbit lacp right with Qnap nas?
Thanks for the comment! I didn't realize my accent was British :) I have not done that setup nor explored it. I might recommend the MikroTik User forum or check out my buddy at vSwitchZero.com he did do a fair bit more with this switch than I. I hope this helps. forum.mikrotik.com/
@@vmexplorer thank u,cheers
Great video, we use this CRS309 in house and we love it.
Hello did you use any specific hardware when you connected the HP 10Gbe SPF+ to the Mikrotik ? as apparently i been advised it uses a different type of communication than the HP cable and will not work ?
I didn't have any issues with the cards or the cables I used.
Check out
"ServeTheHomeVideo" channel they review a lot of mikrotik stuff. Im sure HP and Cisco are using some sort of proprietary ways of cabling or connection. Mikrotik should be less problematic regardin that.
I dont understand why people dont run fiber... I stopped using copper any place I can and just use LC to LC OM4 fiber and I never have to re run anything anymore and all i need to do is swap sfp+ adapters for faster speeds in the future. The OM4 fiber is same price as copper depending where you get it.
Thanks for the comment, and your right Fibre is a great option for Home Labs. For me I wanted to learn more about Infiniband and then went for a simplistic 10Gbe switch in this video. And, that's the fun of Home Labs finding something you want to learn about and doing it. Fibre is on my list, just not right away.
Liked and subscribed
Umm kinda pointless for 30-40 you could get a connectx3 card that can do infiniband OR 10/40 Gbe . I got dual port connect x3 cards running infiniband on one port and 10Gbe ethernet on the other.
Thanks for the comment. You are correct about the ConnectX 3 cards, and I've run them for years now. For my home lab needs, silence and low heat are a must. I work from home and my home lab is located in my home office. So, I'm always looking for value, low heat, and low noise. I have the Mellanox MTS3600. Its normal fans are so loud, even with fan mods, its just to loud. So the point of this video blog post is to show an option where you can get 10gbe that is completely silent and at a good price point. I hope this clear is up, if not post up and Happy computing!
Are you using a qsfp to sfp+ adapter to get 10gbe?
The DAC cable you link is not $5, but $25.....
Thanks for the comment... I got mine for $4.99 apiece. Upon a quick look on eBay you are right, the prices have gone way up...
Well, I still think IB mode is superior due its 4 times better bandwidth and awesome latency
I could not agree more. However, for my purposes with a Home Lab that is right next to where I do customer calls - IB can be loud, hot, and if you are not accustom to it difficult to first start working with vs. this 10gbe switch has none of these issues.
Why not to 40GBE?
He seems to have a goal of noise reduction, both the cards and the switch for 10Ge are fanless that he showed, but 40Ge would be back to fans again
Bingo, the InfiniBand switches I had access to were loud and run hot. I like to have my home labs quiet and cool as they sit right next to me while I'm working. Also, 40Gbe was nice but its like having a Ferrari the garage that you never use. My home lab rarely even comes close to maxing out 10Gbe.
Yan i use the RJ45 to connect it to my gigabit Lan ?
By RJ45 to your Gbe Lan, do you mean the single RJ45 port that is part of MirkoTik CRS309-1G-8S+? If so, I use it for a management interface. I'm not sure of its other uses. Best to read the manual. More information found here - mikrotik.com/product/crs309_1g_8s_in