We recently purchased Emulex 10 Gb CNA Cards for our new and existing VMware ESX Hosts and other large bandwidth servers. However we have seen nothing but problems with them crashing. After going round and round with VMWare, Cisco and Emulex we seem to have a stable build using a beta driver build we received from Emulex. This driver is only for the Ethernet controller on the card; in other words the be2net driver.
The problem seems to be with any version of the driver that has TCP Offload enabled. On the windows drivers we were able to configure the driver to disable this "Feature" which made the cards stable. On Vmware the drivers at current time only go up to VMware ESX/ESXi 4.x Driver CD for ServerEngines BladeEngine 10Gb Version "2.102.440.0" released on 2010/09/16. Problems you'll notice with this driver is if you change VLAN ID’s on a Network the ESX host will crash with a purple screen. Other problems will arise like hosts would lose the ability to talk to each other intermittently. However the switching the VLAN ID case was used to case the crash on demand for testing.
Our network guy pushed and we were able to get a beta release of the be2net driver. The build is version be2net-2.102.474.1 and from what we read in it the notes we got. This build allows you to enabled and disable vlan offloading with the default being disabled. Like the Windows driver we worked with on windows 2008 R2 that appears to be all that’s needed to make the driver stable.
Below is the screen shot of the ESX with the beta unsigned driver running an ESX Host that’s been stable for last 2 weeks and pasted every test we could think of to try.
We also tested a QLogic 10 Gb CNA since we considered switching to it however it had the same issue with crashing with TCP offload. We wonder if they are both using the same chip.
No fix of yet would allow us to enable this feature.
Update 1: Newer Beta Drivers Listed
So I just wanted the first comment on your site. You know I don't really understand any of this yet, but you are great with teaching me so of it. I like the background too. ^_^
ReplyDeleteAnyway, I love you.
Danielle
I am dealing with a different problem with the currently available driver as well. If the vNIC loses network work for whatever reason the link will show up, but no one is home to pass the traffic. I have been playing games with emulex and IBM to get the beta driver with no luck. Your network guy must be more persuasive then I.
ReplyDeleteNot sure about the IBM side yet, deploying to our AIX Servers next week. But if possible turn off any TCP offload and give it a try. That works for our windows servers. The issue with Vmware was that the driver didn't have the option to turn it off untill the beta version our Network guy got.
ReplyDelete.....And yes my Network Guy can be very persuasive.
hi chris, i have similar issue in my current deployment. Can you drop me an email at net_storm@msn.com? I need some assistance from you.
ReplyDeletethanks.