Here is something you probably didn’t know about reviewing hardware: benchmarking sucks.

Sure, at first it’s kind of interesting and you can learn and infer a lot about the scores you receive at different points of any given benchmark, but eventually it becomes so atrociously repetitive that gouging your eyeballs out with a spoon seems like a more attractive alternative to sitting through the remainder of the process.

At FPSLabs, we employ the use of repetition to minimize the effect of variance in our benchmarks. This ensures accuracy with all of the scores we report. Unfortunately, it takes a hell of a long time to do. There are many tests we repeat as much as ten times to acquire an average; ten times of the exact same thing.

Also problematic is the fact that you actually have to be attentive to the whole benchmarking process. If, for instance, you go out to grab a bite to eat during a benchmark and some sort of glitch happens that skews the final result, you have no idea it happened. Also, if you leave the system idle for long periods of time in between benchmarks, you will end up taking the better part of a year to get through all the testing.

Occasionally we get to have a shred of fun testing hardware when we play games with no benchmarking functionality built in. For these games we have to actually play the game and use FRAPS to record the frame rates. Granted, you have to do the same thing over and over, and over again to ensure reliable data, but at least you get to do something.

But then again, what really keeps us going in this repetitive and often tedious world of hardware reviewing is witnessing the latest hardware absolutely demolish the competition. We got to see this twice last year with the Core 2 family of processor from Intel and the G80-based GeForce 8800GTX from NVIDIA. In contrast, this year has been pretty slow.

So slow, in fact, that it has prompted at least one notable hardware journalist to blog about what he considers the death of the PC Enthusiast Industry. After reading KK’s post on the matter, I find it very difficult to disagree. Whereas in years past the enthusiast industry was driven by the excitement generated by product launches and technology bumps from all sorts of companies. The variation in products and the way they worked was genuinely interesting back then. Nowadays, it seems like the market is saturated with products that do exactly the same thing, just slightly differently. It’s kinda boring. From a purely hardware perspective it’s hard to argue with him. However, the bigger picture- the one FPSLabs is interested in- displays, among other things, something that promises to keep the PC enthusiast industry alive and kicking: games.

For how long have we been touting new technology on a “killer app” basis? Waiting for one application to come along and validate the existence of some before-its-time product can grow tiring, but whether we like or not, it drives this industry. Nobody thought the KillerNIC network card was any good, and arguably for the price it isn’t, but the performance benefits it has been found to provide in various select applications cannot be denied. People may have had a sneaking suspicion that AGEIA’s PhysX card could do more than crappy games like GRAW allowed it to, but it wasn’t until CellFactor: Revolution and its evolutionary in-game physics came along that we really saw what the product was capable of. Processors and video cards in general have the same kind of thing going for them, only far less pronounced or recognized.

And then occasionally a “killer app” comes along that quite literally kills new technology. Crysis is, in my humble opinion and from a strictly hardware perspective, the single most important application of this year and probably the next couple as well. Perhaps on some level KK is correct in saying the PC enthusiast industry is dead. Perhaps one day I will look back on this blog and think “wow, how could I have been so stupid?” But for now, I think it is reasonable if not logical to consider Crysis somewhat of a savior of the PC enthusiast industry. Crysis will not only push hardware manufacturers to produce bigger and better products due to its graphical splendor, it will also drive sales of high-end parts (which has its own effect on manufacturers) because people genuinely want to experience said graphical splendor.

To test this theory I propose the following simple test:

Find a person in your life that has basically no interest in video games. Show that person Crysis on “Very High”. Challenge them to not be impressed.

Everybody has some premonition of what video games are supposed to look like. I have yet to find anyone who was not utterly flabbergasted by what Crysis has to offer.

So for now, let’s not panic about the death of the PC enthusiast industry. Let’s have faith that software will drive hardware and vice versa. Let’s all sit back, relax, and watch this damn Crysis fly-by loop on this GeForce 8800GTS 512 for hours on end.

PS: In case you didn’t notice this was entirely a rant. Sorry. You may now go back to your life.

Popularity: 6% [?]

You Should Also Check Out These Post:

• Nightly News Links for 2008-11-23
• Valve Joins the Cloud Computing Trend
• NVIDIA Updates
• Sapphire's Radeon HD4850X2
• Get out and vote

More Active Posts:

• Some Updates (8)
• NVIDIA has an overcomplicated lineup (8)
• What happened... (8)
• Welcome To FPSLabs V3 With Added Awesome! (7)
• NVIDIA GeForce 9800GTX on April 1st (7)
• Razer Piranha Review (7)
• Microsoft to sell XP until 2010 (6)
• Weekly Hardware Recap (6)
• The Weekly Hardware Recap (5)
• Newbie's Guide to Vista: The Look (5)