Back up inside two hours is pretty good though, tip ‘o the hat to the engineers over there.

1. No single points of failure
2. All failovers are automatic and swift
3. Use non-proprietary technologies where possible

When coupling server to the network I myself quite like using EtherChannel. EtherChannel allows multiple physical Ethernet links to combine into one logical channel, which allows the links in the channel to share traffic load, as well as redundancy in the event that one or more links in the channel fail. So you have at least two cables linking a server to a switch, and with some intelligence in setting up the link you both double the bandwidth available to that server and make sure that a dead cable doesn’t leave your server dead in the water and you racing to the data center.

Equipment used

As a big friend of Cisco equipment I’ll use a Cisco switch to show the configuration at the switch end, although many other managed switches from other brands do support the EtherChannel standards as well. Of course the server (or in fact the network card) also needs to support the configuration, I’ll be using an HP ProLiant server today with an HP Dual Port ProLiant Network Adapter (again, other brands of course also support this configuration).
Another note: All commands used below are Cisco IOS commands. if you’re using a switch running CatOS please scroll to the end of this article for a short recap of the used commands for CatOS.

First you’ll need to check if your switch supports EtherChannel by using the show command on one of the ports you’d like to use, and look for the ‘Channel’ key word:

asp-lsw-01#show interface Gi1/5 capabilities
GigabitEthernet1/5
Model: WS-C4948-RJ-45
Type: 10/100/1000-TX
Speed: 10,100,1000,auto
Duplex: half,full,auto
Auto-MDIX: no
Trunk encap. type: 802.1Q,ISL
Trunk mode: on,off,desirable,nonegotiate
Channel: yes
Broadcast suppression: percentage(0-100), hw
Flowcontrol: rx-(off,on,desired),tx-(off,on,desired)
VLAN Membership: static, dynamic
Fast Start: yes
CoS rewrite: yes
ToS rewrite: yes
Inline power: no
SPAN: source/destination
UDLD: yes
Link Debounce: no
Link Debounce Time: no
Port Security: yes
Dot1x: yes
Maximum MTU: 9198 bytes (Jumbo Frames)
Multiple Media Types: no
Diagnostic Monitoring: N/A
Queuing: rx-(N/A), tx-(1p3q1t, Sharing/Shaping)
asp-lsw-01#

Hooray, my switch understands EtherChannels. To check the capabilities of your network card you’ll have to refer to the manufacturer’s information I’m afraid.

Cabling

As Cisco puts it:

The EtherChannel should start on a single device and end on another single device. The device can be a switch, a switch stack, a workstation, or a server.

• Within a single switch chassis, the EtherChannel can start or end on different modules. This setup is applicable for Cisco Catalyst 4000/4500/6000/6500 switches.
• Within a single switch stack, the EtherChannel can start or end on different stack members.

Do note that CatOS-based Cisco switches do have stricter requirements on EtherChannels, basically you can only create EtherChannels on adjoining odd/even pairs, otherwise the switch will not accept your port selections.

Rule of thumb on cabling: never ever connect a cable to a port on your network without configuring it first! Especially connecting multiple cables to the same end device can produce some very unexpected and nasty results on your network as for example STP goes haywire on ill-configured ports and disables entire trees of switches on your LAN. Connect the cables only after finishing your configuration.

Understanding EtherChannel negotiation and port mode

EtherChannel ports can use two protocols for EtherChannel negotiation: either the Cisco-proprietary PAgP or LACP (IEEE 802.3ad). I generally prefer using accepted cross-brand standards (see HA golden rule 3 above) , so we’ll use LACP here as well. When setting an EtherChannel mode in a Cisco switch, your choices are ‘on’, ‘off’, ‘active’, ‘passive’, ‘desirable’ or ‘auto’. Each mode setting forces a particular negotiation protocol and behaviour as such:

I shall not bore you with the pros and cons of using negotiation vs. fixed configuration, you will have to decide for yourself here whether you’ll use on or active mode. In this example I’ll be using active mode.

Configuring the switch

As an EtherChannel uses multiple ports, you will have to repeat these steps for all ports you wish to couple in this channel.

1. Set the port to switchport mode (optional, only necessary on L2/L3 switches):

asp-lsw-01(config)#int Gi1/5
asp-lsw-01(config-if)#switchport

2. If you’re using VLANs, set the port to access mode for the correct VLAN (I’m using VLAN 2 in this example):

asp-lsw-01(config-if)#switchport access vlan 2
asp-lsw-01(config-if)#switchport mode access

3. As this ports connects to a server directly (as opposed to a part of the network infrastructure such as another switch) you can enable portfast, thus shortening the time the port takes to become active. Never ever enable portfast on ports connected to other switches etc.

asp-lsw-01(config-if)#spanning-tree portfast
%Warning: portfast should only be enabled on ports connected to
a single host. Connecting hubs, concentrators, switches, bridges,
etc... to this interface  when portfast is enabled, can cause
temporary bridging loops. Use with CAUTION

%Portfast has been configured on GigabitEthernet1/5 but will
only have effect when the interface is in a non-trunking mode.

4. Configure the port for EtherChannel using the mode you chose earlier. Make sure you use an unused channel group number.

asp-lsw-01(config-if)#channel-group 1 mode active
Creating a port-channel interface Port-channel 1

This concludes the configuration of the switch port (don’t forget to perform these steps on all ports that have to be linked in the EtherChannel!).

Configuring the server

As we’re using an HP ProLiant server we can configure the EtherChannel using the HP Network Configuration Utility. Other brands and types will have their own drivers and software to get this done. One thing to keep in mind is that the NIC software may not call the setup “EtherChannel” but may use other terms like “Teaming” (which is what HP calls it).

1. Start the HP Network Configuration Utility (from the task bar of the Start menu):

2. Select both listed NICs and click the ‘Team’ button. This will immediately create the team:

3. Select the team and click the ‘Properties’ button:

4. Pick the appropriate Team Type Selection, in our case “802.3ad” (which is the IEEE standard number for LACP):

5. Choose the Transmit Load Balancing Method from the next drop-down list, and click ‘Ok’ after you’ve made your selection.

6. Apply the configuration, after which the server will want to reboot. After the reboot, go back to the HP Network Configuration Utility to check that the Teaming is functioning as expected (i.e.: showing a green icon).

If all went well you should now have a new network adapter in your Windows network configuration. This is the adapter you will use to configure the network from now on (i.e.: IP address, file and printer sharing, etc.).

More, more, more…

Now that you have an EtherChannel link to your switch, ponder in this: what if the switch fails? You would in fact still lose connectivity to the server. Yowsa. Remember HA golden rule number 1: No single points of failure. You could actually create two EtherChannels (so using four network cables in total), each to a separate but interconnected switch, thus ensuring that even a switch failure wouldn’t kill the traffic to your server. If both switches are configured properly STP (Spanning Tree Protocol) will disable one of the EtherChannels while the primary is active, but will fail over to the second link if the first one dies.
How far are you willing to go for HA?

CatOS: a slightly different approach

If your Cisco switch happens to be running CatOS, the EtherChannel modes and the commands to configure the switch ports are somewhat different. The modes are:

Using the same example as before the commands on CatOS would be:

1. assign the ports to the correct VLAN (VLAN 2 in this example, being set on ports 5/1 and 5/2 (as CatOS works with port sets)):

old-sw-01 (enable)set vlan 2 5/1-2
old-sw-01 (enable)set trunk 5/1-2 off

2. Enable portfast (optional):

old-sw-01 (enable)set spantree portfast 5/1-2 enable

3. Enable EtherChannel on the port set:

old-sw-01 (enable)set port channel 5/1-2 mode on

4. Set the load distribution method:

old-sw-01 (enable)set port channel all distribution mac source

This concludes the configuration of the switch ports, as you can see you don’t need to repeat these steps on CatOS as you’ve just configured both ports in the set in one go.

Last week I noticed a major dip in the server’s performance, which got worse and worse nearing the weekend. A quick scan revealed that my Apache processes were taking up around 90 – 95% of the CPU capacity with no real direct sign of the offending site. It took my 30 minutes to write and run a script that disabled the hosted sites one by one and then checked the CPU results. The ‘culprit’ appeared to be moqub.com, a (Dutch) blog run my a friend of mine with a steady following of readers.

As I had recently upgraded her WordPress to version 2.5 my first suspicion was that the new software was messing up my server…but then again my own site also runs WordPress 2.5, and if I put some stress on this blog the CPU was still quietly ticking over, not stressing out as with her site. But still…her WordPress was the result of upgrade upon upgrade, each version having been added on to with lots of plugins, so I just couldn’t be sure. I set up a fresh site on the server, did a clean WordPress 2.5 install, imported Moqub’s posts, comment and links, added the theme (after verifying its 2.5 compatibility) and diverted the visitors to the new site. Hey presto, the CPU was back up to 90% again, even when I disabled all customisations and the extra theme. Yikes.

As I now knew that the problem was not with WordPress itself, I started digging around in the logs. I found that there was an extraordinary amount of requests from a single IP address, identifying itself as a “Microsoft Search Bot 4.0″. Well, that should be easy to fix: I built a custom robots.txt that should have shushed all robot traffic except for GoogleBot, but to no avail. The bot never even tried to read the text file, it just went straight for the content, running several threads at once at high speed and thus maxing out the CPU.
A little research showed the IP address belonged to the Provincial Central Library in Drenthe (a province in the east of The Netherlands). This was no coincidence as Moqub writes about libraries and the use of information systems in them. Still, as their robot misbehaved I had no alternative but to completely block the IP address the robot was originating from. Ahh…peace and quiet on the server at last.

Now the question bugs me: why do robots still misbehave and completely ignore the robots.txt file, accepted (as far as I know) as the de facto standard in blocking or guiding robot traffic? And this was no home-brew, this was a Microsoft robot. Am I just being silly and naive in expecting “professional” software to behave according to the rules?

The lesson for me here was that badly run scripts can really mess up your server, especially if they decide to dig in to dynamically generated pages. And there really is not a whole lot you can do about it if they decide to completely ignore the standards in place.