If your switching infrastructure needs something bigger than a standalone LAN switch but not quite as big (and expensive) as a modular chassis, you might want to consider stacking your switches. Cisco offers their StackWise feature on the Cisco Catalyst 3750 series switches, and StackWise Plus on Catalyst 3750-E series.

What StackWise basically does is build one big switch by intelligently joining several individual switches together (with a maximum of nice switches total). The switches are united into a single logical unit using special stack interconnect cables that create a bidirectional closed-loop path. The bidirectional path acts as a switch fabric for all the connected switches, where network topology and routing information is updated continuously through the stack. All stack members have full access to the stack interconnect bandwidth (32 Gb).

The stack is managed as a single unit by a master switch, which is elected from one of the stack member switches. Any stack member switch can become master, and election is done my one of these methods:

1. User priority-The network manager can select a switch to be master.
2. Hardware and software priority-This will default to the unit with the most extensive feature set.
3. Default configuration-If a switch has preexisting configuration information, it will take precedence over switches that have not been configured.
4. Uptime-The switch that has been running the longest is selected.
5. MAC address-Each switch reports its MAC address to all its neighbors for comparison. The switch with the lowest MAC address is selected.

When one master switch becomes inactive and while a new master is elected, the stack continues to function.

To build StackWise connections you need (at least two) of either of these cables:

• CAB-STACK-50CM= (0.5-meter cable)
• CAB-STACK-1M= (1-meter cable)
• CAB-STACK-3M= (3-meter cable)

The reason you need at least two cables (or more precisely: X + 1, where X is the number of switches in the stack) is that the stacking cables must form a ring, as the connections are bidirectional, and without the ring a failure of one of the cables would break the stack with all manner of nasty things happening to your network. In a correct configuration a stack cable failure will halve the capacity but the stack continues functioning, while after the failed cable starts functioning again full capacity is automatically restored.

The main differences between StackWise (supported on 3750 and 3750-E series) and StackWise plus (only supported on 3750-E series) are:

• StackWise plus performs destination stripping, where StackWise only supports source stripping. What this means is that with source stripping a unicast packet that is sent on the stack ring goes around the whole ring and is removed (i.e.: stripped) by the sender of the packet (i.e.: the source switch). With destination stripping the target switch of the packet removes it from the ring, freeing up bandwidth on the ring sooner.
• StackWise Plus supports local switching, where StackWise doesn’t, forcing all packets on the stack ring. On a StackWise Plus stack any traffic destined for a node connected to the same member switch stays within that member switch, and doesn’t use the stack ring at all.
• StackWise Plus will support up to 2 line rate 10 Gb Ethernet ports per switch.

You can build a stack consisting of a mix of 3750 and 3750-E switches, in that case only StackWise features are available with the exception of the local switching, the 3750-E’s will still do that.

All in all the StackWise feature is quite nice. Just as with RAID arrays you can build a resilient and easy to manage system by using relatively inexpensive devices. Management is almost as easy as managing a single switch, and most of the features that make StackWise worthwhile and in some respects even great are automatic and do their thing quietly and without fuss. Good stuff.

When building high availability networks or systems the three golden rules are:

1. No single points of failure
2. All failovers are automatic and swift
3. Use non-proprietary technologies where possible

When coupling server to the network I myself quite like using EtherChannel. EtherChannel allows multiple physical Ethernet links to combine into one logical channel, which allows the links in the channel to share traffic load, as well as redundancy in the event that one or more links in the channel fail. So you have at least two cables linking a server to a switch, and with some intelligence in setting up the link you both double the bandwidth available to that server and make sure that a dead cable doesn’t leave your server dead in the water and you racing to the data center.

Equipment used

As a big friend of Cisco equipment I’ll use a Cisco switch to show the configuration at the switch end, although many other managed switches from other brands do support the EtherChannel standards as well. Of course the server (or in fact the network card) also needs to support the configuration, I’ll be using an HP ProLiant server today with an HP Dual Port ProLiant Network Adapter (again, other brands of course also support this configuration).
Another note: All commands used below are Cisco IOS commands. if you’re using a switch running CatOS please scroll to the end of this article for a short recap of the used commands for CatOS.

First you’ll need to check if your switch supports EtherChannel by using the show command on one of the ports you’d like to use, and look for the ‘Channel’ key word:

asp-lsw-01#show interface Gi1/5 capabilities
GigabitEthernet1/5
Model: WS-C4948-RJ-45
Type: 10/100/1000-TX
Speed: 10,100,1000,auto
Duplex: half,full,auto
Auto-MDIX: no
Trunk encap. type: 802.1Q,ISL
Trunk mode: on,off,desirable,nonegotiate
Channel: yes
Broadcast suppression: percentage(0-100), hw
Flowcontrol: rx-(off,on,desired),tx-(off,on,desired)
VLAN Membership: static, dynamic
Fast Start: yes
CoS rewrite: yes
ToS rewrite: yes
Inline power: no
SPAN: source/destination
UDLD: yes
Link Debounce: no
Link Debounce Time: no
Port Security: yes
Dot1x: yes
Maximum MTU: 9198 bytes (Jumbo Frames)
Multiple Media Types: no
Diagnostic Monitoring: N/A
Queuing: rx-(N/A), tx-(1p3q1t, Sharing/Shaping)
asp-lsw-01#

Hooray, my switch understands EtherChannels. To check the capabilities of your network card you’ll have to refer to the manufacturer’s information I’m afraid.

Cabling

As Cisco puts it:

The EtherChannel should start on a single device and end on another single device. The device can be a switch, a switch stack, a workstation, or a server.

• Within a single switch chassis, the EtherChannel can start or end on different modules. This setup is applicable for Cisco Catalyst 4000/4500/6000/6500 switches.
• Within a single switch stack, the EtherChannel can start or end on different stack members.

Do note that CatOS-based Cisco switches do have stricter requirements on EtherChannels, basically you can only create EtherChannels on adjoining odd/even pairs, otherwise the switch will not accept your port selections.

Rule of thumb on cabling: never ever connect a cable to a port on your network without configuring it first! Especially connecting multiple cables to the same end device can produce some very unexpected and nasty results on your network as for example STP goes haywire on ill-configured ports and disables entire trees of switches on your LAN. Connect the cables only after finishing your configuration.

Understanding EtherChannel negotiation and port mode

EtherChannel ports can use two protocols for EtherChannel negotiation: either the Cisco-proprietary PAgP or LACP (IEEE 802.3ad). I generally prefer using accepted cross-brand standards (see HA golden rule 3 above) , so we’ll use LACP here as well. When setting an EtherChannel mode in a Cisco switch, your choices are ‘on’, ‘off’, ‘active’, ‘passive’, ‘desirable’ or ‘auto’. Each mode setting forces a particular negotiation protocol and behaviour as such:

I shall not bore you with the pros and cons of using negotiation vs. fixed configuration, you will have to decide for yourself here whether you’ll use on or active mode. In this example I’ll be using active mode.

Configuring the switch

As an EtherChannel uses multiple ports, you will have to repeat these steps for all ports you wish to couple in this channel.

1. Set the port to switchport mode (optional, only necessary on L2/L3 switches):

asp-lsw-01(config)#int Gi1/5
asp-lsw-01(config-if)#switchport

2. If you’re using VLANs, set the port to access mode for the correct VLAN (I’m using VLAN 2 in this example):

asp-lsw-01(config-if)#switchport access vlan 2
asp-lsw-01(config-if)#switchport mode access

3. As this ports connects to a server directly (as opposed to a part of the network infrastructure such as another switch) you can enable portfast, thus shortening the time the port takes to become active. Never ever enable portfast on ports connected to other switches etc.

asp-lsw-01(config-if)#spanning-tree portfast
%Warning: portfast should only be enabled on ports connected to
a single host. Connecting hubs, concentrators, switches, bridges,
etc... to this interface  when portfast is enabled, can cause
temporary bridging loops. Use with CAUTION

%Portfast has been configured on GigabitEthernet1/5 but will
only have effect when the interface is in a non-trunking mode.

4. Configure the port for EtherChannel using the mode you chose earlier. Make sure you use an unused channel group number.

asp-lsw-01(config-if)#channel-group 1 mode active
Creating a port-channel interface Port-channel 1

This concludes the configuration of the switch port (don’t forget to perform these steps on all ports that have to be linked in the EtherChannel!).

Configuring the server

As we’re using an HP ProLiant server we can configure the EtherChannel using the HP Network Configuration Utility. Other brands and types will have their own drivers and software to get this done. One thing to keep in mind is that the NIC software may not call the setup “EtherChannel” but may use other terms like “Teaming” (which is what HP calls it).

1. Start the HP Network Configuration Utility (from the task bar of the Start menu):

2. Select both listed NICs and click the ‘Team’ button. This will immediately create the team:

3. Select the team and click the ‘Properties’ button:

4. Pick the appropriate Team Type Selection, in our case “802.3ad” (which is the IEEE standard number for LACP):

5. Choose the Transmit Load Balancing Method from the next drop-down list, and click ‘Ok’ after you’ve made your selection.

6. Apply the configuration, after which the server will want to reboot. After the reboot, go back to the HP Network Configuration Utility to check that the Teaming is functioning as expected (i.e.: showing a green icon).

If all went well you should now have a new network adapter in your Windows network configuration. This is the adapter you will use to configure the network from now on (i.e.: IP address, file and printer sharing, etc.).

More, more, more…

Now that you have an EtherChannel link to your switch, ponder in this: what if the switch fails? You would in fact still lose connectivity to the server. Yowsa. Remember HA golden rule number 1: No single points of failure. You could actually create two EtherChannels (so using four network cables in total), each to a separate but interconnected switch, thus ensuring that even a switch failure wouldn’t kill the traffic to your server. If both switches are configured properly STP (Spanning Tree Protocol) will disable one of the EtherChannels while the primary is active, but will fail over to the second link if the first one dies.
How far are you willing to go for HA?

CatOS: a slightly different approach

If your Cisco switch happens to be running CatOS, the EtherChannel modes and the commands to configure the switch ports are somewhat different. The modes are:

Using the same example as before the commands on CatOS would be:

1. assign the ports to the correct VLAN (VLAN 2 in this example, being set on ports 5/1 and 5/2 (as CatOS works with port sets)):

old-sw-01 (enable)set vlan 2 5/1-2
old-sw-01 (enable)set trunk 5/1-2 off

2. Enable portfast (optional):

old-sw-01 (enable)set spantree portfast 5/1-2 enable

3. Enable EtherChannel on the port set:

old-sw-01 (enable)set port channel 5/1-2 mode on

4. Set the load distribution method:

old-sw-01 (enable)set port channel all distribution mac source

This concludes the configuration of the switch ports, as you can see you don’t need to repeat these steps on CatOS as you’ve just configured both ports in the set in one go.