BigFoot Networks Killer NIC: Killer Marketing or Killer Product?

Testing Methodology

As we alluded to on the front page, designing a set of testing methodologies that would be consistent and repeatable for the Killer NIC is nearly impossible. It still is to be quite honest. However, we feel like the test criteria we devised is fair for both BigFoot Networks and our readers. There is a phrase that we had to abide by during testing and it makes sense when reviewing our results: Sometimes close enough is good enough.

How do you test a card when the two most critical variables in the online gaming experience, our network connection and the game's server performance, are totally out of our control? Besides these two variables it is impossible to guarantee the movements, actions, experience level, character class, and quantity of players on the servers we connected to would be the same or at the very least close enough for each of our tests. Our testing methodology is by no means perfect we found it to be good enough so this is what we decided to do.

We set up two identical systems for testing as our main focus is to determine if the $279.99 Killer NIC provides a better overall online gaming experience in a wide variety of games than the onboard and essentially free NIC on the Asus P5N32-SLI Premium. Each system had the exact same software image with the only differences being the driver load for the Killer NIC and the nForce 590SLI. Each system was connected to a D-Link Gamer Lounge router with all GameFuel options disabled. We have both Cable and DSL service but decided to strictly use the DSL connection to further reduce the variables. Our Cable connection speed varies greatly depending upon the time of day so this option was not viable. We have enough variables to contend with and adding yet another one was not in our best interest.

We utilized FRAPS to capture our frame rates in each game and the ping time measurements came from the games internal ping rate value where applicable. The ping rates were captured by a third party every sixty seconds and then averaged over the length of game play. We played each game for an average of fifteen minutes online for a total of ten separate sessions at different periods during the day or night. We typically logged on to servers that were fully populated when possible or were close to the maximum amount of players for the selected map.

We utilized each test system at the same time, each player played as the same character class when possible, and connected to the same game server simultaneously. Each player stayed together, followed the same path, and performed the same actions as much as possible during testing. If one player was killed during action then we started the process over until we reached our fifteen minute mark except in Counter Strike: Source and Quake 4 where the time limits ranged anywhere from seven to twenty minutes. Our World of WarCraft play time was 30 minutes in each section due to our play locations and character travel requirements.

Our testing routine was fairly methodical in nature for the FPS and MMO games. We will not mention names but one particular tester took a significant amount of time to learn how to stay out of harms way in most games. It was so bad at one point that we thought about calling Dick Cheney to give him a lesson on how to use a shotgun. Overall, we spent about 300 hours online to bring you around a 100 hours worth of results.

In our RTS testing we had to change our testing methods slightly. Each player played the same map and as the same class each time in a single versus single player game. We scripted out a series of events for each player to follow and probably had a 98% success rate in following the script. We also tried random two on two player matches on the same map and server to see if there would be any noticeable differences in our results. There were not any real differences so we will present our one on one player results. Our outside test party hosted the game utilizing the same DSL service and had a similar system setup on their end. Ping rates could not be accurately measured in our RTS games so they will not be presented, although they were typically very low at game startup due to our test routine.

We also will present results with our D-Link Gamer Lounge DGL-4100 router with GameFuel technology implemented. This router basically promises to accomplish several of the Killer NIC features on the router. This includes providing a platform that reduces latency and boosts network efficiency and performance while intelligently managing and automatically prioritizing network traffic. Sounds familiar right? We are including results from our D-Link DGE-550T PCI Gigabit network adapter to show the results from using a standard PCI NIC in your system with and without our D-Link DGL-4100 optimized.

Since we did not have four identical systems our test plan for the Gaming Router and PCI NIC consisted of powering down our test systems, inserting our D-Link PCI NIC card into each system, removing the Killer NIC and disabling the nForce 590SLI NIC in the BIOS, swapping out the hard drives for a D-Link NIC image, rebooting, and then setting up the D-Link router for optimized operations with our games. We then tested each machine in the same manner listed earlier. The difference being that we ran our tests with GameFuel on and then ran the tests again with GameFuel off. This meant there was usually a ninety minute difference in test results between our Killer NIC / nForce 590SLI combination and our D-Link Router / D-Link NIC results.

A lot can happen in ninety minutes when playing online but fortunately our variables were not off the map after each individual test session. The greatest variability we noticed in the results actually came from the time differences between running our D-Link NIC with GameFuel enabled and disabled. Please be aware of this when reviewing the numbers. However, considering our results are averaged from ten different sessions the overall variability is fairly minimal between test sessions. We set the Killer NIC to Game mode for all of our gaming benchmarks.

There were so many different combinations that we could have tried but in the end we decided to utilize this combination of components to present our results. We could have utilized a standalone PCI-E NIC, used a different video card or processor, and tested strictly on a LAN to remove most variables. We actually did all of that and the results were different but the percentage differences in scores between each solution were always about the same except when using a single processor in World of WarCraft. We will explain that issue shortly. It is time to take a look at the benchmarks and see how well this Killer NIC performs in a high-end gaming system.