Experience Testing

Because we couldn't perform as many useful repeatable tests as we wanted, we have done quite a bit of just plain gaming. We played with the hardware and without the hardware. We tested EVE Online and Team Fortress 2. Bigfoot reports that Team Fortress 2 sees some of the highest benefit from their technology, and we included EVE in order to gauge impact on network games / MMOs that were not singled out by Bigfoot. We played around with WoW for a while, but we don't have a high enough character to do anything where latency could really matter (large parties playing end-game content). These tests were done the way we normally game: with nothing running in the background and no downloading going on.

In playing on our Core i7 965 system with an NVIDIA GeForce GTX 280 and 6GB of RAM, we spent a couple hours with each game. Half of our time was with onboard networking and the other half with the Killer Xeno Pro. Both games were run at their highest quality settings and resolution on our 30" panel.

In EVE we ran some missions and got into a little PvP action. While we made more isk (EVE's in-game currency) playing with the Killer Xeno Pro, this was just the result of the missions we were handed. Neither PvE nor PvP situations felt any different with the onboard NIC versus the Killer Xeno Pro. Action was just as smooth and the UI was just as responsive no matter what was going on. We felt the same sort of loading hiccups when changing areas with both networking solutions as well: the Killer Xeno Pro just didn't deliver any tangible benefit in EVE Online.

Our Team Fortress 2 testing consisted of lots of different games played on both the on-board NIC and the Killer Xeno Pro.

We do need to preface this by acknowledging the fact that none of us are really twitch shooter experts. Sure, we all played and loved Counter Strike and CS:S, Unreal Tournament in all its incarnations, and many other FPS games, but we aren't the kind of people who run moderate resolutions with 16-bit color and most of the options turned as low as possible in order to get every single possible advantage. We are also not professional gamers; but we do love to game.

That being said, we really didn't notice any difference in our gaming experience with or without the Killer Xeno Pro. I tend to like sniping in games, and typically even non-twitch gamers can tell if they're being screwed out of kills by network issues. I didn't experience this sort of frustration with either solution. Game play was smooth and not jerky or problematic even in larger fire fights when there were no other issues at play. When playing both with and without the Killer Xeno Pro, we experienced some issues when on servers with issues.

It is just a fact that the most important factor is going to be finding a game where you and all the other players have a low latency connection to the server. The slight difference of a minimally reduced client side latency is not going to have a higher impact than any sort of other network issues.

In other words (and to sum up), when you have a bad connection, the Killer Xeno Pro is not going to fix it; when you have a good connection, the Killer Xeno Pro is not going to make the experience any better.

Mostly Deterministic Testing Final Words
Comments Locked

121 Comments

View All Comments

  • has407 - Saturday, July 4, 2009 - link

    Doubtful; a decent NIC is going to provide about the same at lower cost. However, 600MB/min = 10MB/sec = pretty slow. Sounds like you have another problem (your network infrastructure is 100Mbs?). Or do you--is 600MB/min really a problem or is that all the clients are demanding? What does your network configuration look like?
  • has407 - Friday, July 3, 2009 - link

    There is an SDK (and other tools available). I've never used it, but it's the only reason I've bothered looking at the card. It's also a bit old...

    http://www.bigfootnetworks.com/Support/index.php?_...">http://www.bigfootnetworks.com/Support/...arentcat...
  • croc - Friday, July 3, 2009 - link

    Nice try Derek... but there are so many network variables involved in testing like this that any reliable test results would be almost impossible to obtain. Nice job trying, but the testing just showed the limitations of the variables involved.

    Maybe with a 250k Cisco Spider platform some reliable tests could be performed. Think Anand would go for that? :-)

    I would have liked to see some packet captures from something like Wireshark, but the average reader may not get much more information from them, and unless you had several captures running on all points of the system in question, it would have been just another pointless excercise. (I doubt that you would get permission to run Wireshark on a WOW server, for instance)

    Way back when, I tried testing an Anthem Eagle 802.10 vs. a Compaq server nic to see if there was any advantage to paying three times the cost for an exchange server's outgoing nic. It was an internal exchange server we were testing to, so I could also capture against that as well. And we had a pretty good network management system for that time as well, even if it was only Openview.... (No spider probes back then...) After poring over all the captures and relevant data from the Openview platform, our best guess was that it would take 50 years for the high end nic to pay for itself.
  • DerekWilson - Sunday, July 5, 2009 - link

    That's sort of a conclusion in itself though ...

    I know my tests aren't perfect, but I really couldn't build anything better in a real world environment ...

    Of course, while I couldn't answer the question "is the Killer Xeno Pro always more efficient and higher performant than an onboard NIC" I could answer the question "does any difference in real performance delivered by the Killer Xeno Pro equate to a real difference in experience while gaming" ... the answer to the latter question is definitely no while the answer to the previous question (i suspect) is that the Killer does do more faster ... just not faster enough to make a big difference in modern games or for people to notice in the general case.
  • siberus - Friday, July 3, 2009 - link

    "We played around with WoW for a while, but we don't have a high enough character to do anything where latency could really matter."

    Gotta respect anyone who admits to being a noob :)
  • DerekWilson - Sunday, July 5, 2009 - link

    Yeah, I'm a no0b in WoW, but my EVE character is slightly uber :-P

    Not as uber as it should be for the 6 years I've been playing the game ... but definitely up there.

    Besides, EVE is still where it's at for PvP play in any MMO. Nothing matches it in my opinion.
  • JimmiG - Friday, July 3, 2009 - link

    Well I've already got a Quad core CPU where one or two cores remain at least partially dormant while gaming - no need to "offload" anything...
  • DerekWilson - Sunday, July 5, 2009 - link

    well, bypassing the OS could have an impact -- and in fact very likely does have a very real measurable impact -- on response time (and thus lag reduction) ...

    but the problem is that this reduction in latency is very small relative to the rest of the network performance ...

    it really doesn't seem as much to me to be that it doesn't do anything useful -- just that games are currently coded to handle 100+ms latencies in shooters and can even handle many-hundred millisecond latencies for MMOs especially in the style of EVE.

    if network infrastructure continues to improve and game developers demand lower latency performance to accomplish certain tasks (like to support more people on a server for an FPS or MMOs with larger shards), something like the Killer might become more useful.
  • flashbacck - Friday, July 3, 2009 - link

    "This hardware at this price is just not for everyone..."

    What the hell? This hardware at this price is not for ANYONE!
  • DerekWilson - Sunday, July 5, 2009 - link

    well ... while we didn't find much use for the scenarios we tested, it very well might have more beneficial uses in other applications.

    we are actually interested in doing testing, specifically with the voice chat acceleration, with multiple Killer cards to see if has any measurable or noticeable impact on voice lag.

Log in

Don't have an account? Sign up now