The Difference Between Reviews and the Real World
Samsung was recently called to task over what appears to be their involvement in a scam -- though neither Samsung nor their marketing agency has admitted wrong doing. Specifically, Samsung has been accused of sending high quality LCD screens to reviewers with the same model number as lower quality units found in retail stores.
As Ian Bell notes in his editorial, this is not the first time a hardware company has tried such a stunt. It would surprise me if it wasn't a regular occurrence, one that the hardware companies get away with more than not.
After all, sometimes the difference is legitimate. Reviewers, eager to have the first article on a new computer component, will test early pre-release models. These often perform differently than those which eventually find their way to the retail shelves. If a discrepancy is later revealed, this can be used as an excuse.
Recently, both nVidia and ATI, the two leading graphics chip manufactures, were involved in a similar scandal. They were both, at different points, accused of rigging their video card drivers so that the cards would appear to perform better than they really did.
Reviewers tend to run the same tests against video cards over time. This gives the reviewer -- and the consumer -- an idea of the performance gains over older cards. With this knowledge, ATI and nVidia allegedly "optimized" their video card drivers so that they would perform better in the tests than they would have otherwise.
As the scandal surfaced, reviewers found that if they varied the tests even slightly, that performance dropped radically. In many cases, the cards were unstable: the picture would break up, leaving video artifacts on the screen.
The point here is that you can't expect a card to perform the same in real life as it does in a review. Hardware manufactures would be fools not to optimize their hardware and software drivers for the tests that they know reviewers will use.
We've often found that video cards which reviewers said were stable, were anything but. This is not to say that the reviewers were wrong: the cards performed well in their tests. In most cases, it's not even the hardware manufacture's fault: it's simply impossible for a company to test their hardware against every possible combination of hardware and software on the market.
Nevertheless, it's obvious that many hardware companies rush their products to market. They test the hardware and software drivers just enough to get respectable reviews, ship the boxed products to retail stores, and start working on "the next generation".
So, despite the review you read last night which said the new card was faster, despite the corroborating evidence of a news group post from a 14 year old kid, and especially despite the fact that the new card may be cheaper, it's often in your best interest to trust the professional standing in front of you. They've installed hundreds of cards in as many different combinations of hardware and software.
As for Samsung, well, it would be difficult to chalk this up to an accident, just as it's difficult to believe that the nVidia and ATI incidents were the result of software bugs. Nevertheless, having been called to task on it, I would expect Samsung and other LCD manufactures to tread carefully -- at least for the time being.