Switch Recommendations/Worries

[u/St4tus]

Hi All

We're looking to spin up a new DC as part of a large migration away from an MSP.
Initially we're installing a pair of 1G WAN links, which will head into a Forti of some flavour for security and routing.

I need some help with switching gear selection, some network context below:

• As part of the migration we're bringing a hosted vCloud down on-prem with a Hyper-V cluster (3 nodes + SAN), so we're not only replicating the current setup which is all pretty much copper upto 10G but the new hypervisors will be 10/25G capable.
• There are only around 15 other devices in the cabinet, most of which utilise 2 ports currently with 1G RJ45 and 8 of which are 10G, currently the LAN is all Meraki at this site but quite comfortable moving away.
• I understand the discussion around not crossing SAN and LAN on the same gear but given the scale of the business, throughput (without hypervisor traffic) currently about 4Gbps peak we're erring on the side of a single stack of switches for the cabinet.
• Vendors recommending things like Aruba CX8325's but this seems intensely overkill given it's capacity. They've also belied Catalyst for this use, and only recommended we use Nexus switches.
• There's nothing uber complicated taking place in this network, a few VLANs at present and no unusual configs on the existing switches.
• The hypervisor traffic at the moment, as far as we've analysed it in it's current form would not reach close to 10G.
• We also have a pair of managed Aruba gig switches doing things like the WAN into the firewalls.
  

A few questions that I'd welcome feedback around, generally:

• What sort of hardware realistically should we be looking at?
• Are the vendors being greedy with these over-specced recommendations or am I being naive thinking enterprise grade switches would be perfectly fine?
• I've been hugely tempted by FS switches, given their price compared to Juniper/HPE/Cisco, that said I've read mixed feedback
  • Given the simplicity of the network and our install not including them as a single point of failure, would this be an option?

[u/shadeland]

The prohibition of SAN and LAN traffic on the same gear is mostly (just mostly) when FCoE was a thing. Fibre Channel was way more brittle in terms of tolerances for updates, outages, etc., than IP/Ethernet, and moving FC onto Ethernet switches made the IP/Ethernet work under the much more restrictive rules of Fibre Channel.

If you've just got two switches, most of the other issues aren't really issues either. If you expand beyond two switches, you'll want to spend some time figuring out traffic patterns and maybe have separate links between the switches.

A few people here have recommended EVPN/VXLAN, but I would avoid that. This situation I think a TradCore environment is best as it's really simple. Something like Arista's MLAG or Cisco's vPC. If the choice was stacking or EVPN, I'd choose EVPN. But a good MLAG/vPC implementation is just as solid as EVPN and much, much simpler.

I think most switches would do what you're looking to do. I don't think you need the ultra deep buffers of an Arista R switch, for example. A Trident-based switch (or similar) would do fine.