Top of Rack Switching, No Single Point of Failure

Assume that you have 1 rack in a reliable colo facility. The colo (obviously) has advanced chassis switches and can provide any reasonable manner of drops specified (but a limited number of drops). That is to say, you can specify two GB cat6 drops configured such that (specify additional config here).

Also assume that you have N (say 10) "servers" each with 2 GB ethernet ports. Each server needs to have one always accessible, routable ip address. That is to say, each server has an IP address WWW.XXX.YYY.ZZZ that should be pingable from any properly configured host on the internet.

What is the simplest logical and phsical network topology you can install top of rack such that there is no one single point of failure leading to ip connectivity issues between the servers and the gateway provided by the colo?

By simple, I mean, generally speaking, cheapest to implement using Cisco networking gear. That is a rough definition, but I think it should correlate well with the answer I am after.

From serverfault SvrGuy

• You need two switches hook your colo/isps uplinks to each of these. Between the two switches set up two patches and enable rapid spanning tree on each of the switches. Doing this makes sure only one of the two patches are used, and only one of the uplinks.

  Then on each server set up bonding http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding This is assuming Linux servers. Other OSes has their own approach to this but most if not all supports this.

  THis setup relies heavily on rapid spanning tree and can be achieved with most hardware.

  For a more complicated setup, not really what youre looking for though, you can use switches with routing capability, terminate each server on its own vlan and usee VRRP or HSRP to make the servers gateway redundant and use rapid spanningtrees on each vlan to make sure it doesnt loop throught the two links between the switches. Then finally use BGP to handle automatic failover between upstream links. If you used switches with little memory you can have your ISP announce 0/0 rather than a full routing table.

  Hope this helps :)

  womble : Questioner is using WinXP. Channel bonding fun for the whole family.

  Rune Nilssen : Well, usually ethernetbonding is a server os feature ;]
  From Rune Nilssen

• A couple of refinements to Rune's suggestion:

  • if you use Cisco Catalyst 3750's, the stacking cable will eliminate spanning-tree between the switches, provide greater bandwidth and redundancy without using host ports.
  • If the colo has Cisco 6500 core's with the sup720 VSS blades, then you can do Etherchannel on the 3750's to virtual-Etherchannel on the cores (switch1 -> core1 & switch2 -> core2) further reducing spanning-tree
  • If supported by the colo:
    • use HSRP, VRRP or GLBP for gateway redundancy
    • implement UDLD aggressive on the uplinks (assuming they are fiber)
  • consider using RPS units for power redundancy on the 3750's
  From Peter