Gamers with Jobs reports on Shadowrun developer Mitch Gitelman's disappointment with the 7.0 reviews that his game has been getting. He laments that reviewers are being too harsh, have unreal expectations, and don't appreciate the game for being innovative. Having reviewed the game for SCI-FI (and giving it a C+) I figured I'd offer my two cents on this.
What makes a Classic?
First, the game. I need to listen to the podcast to get the specific details, but from my perspective the lack of options in the multiplayer mode -- you could play team capture the flag or team deathmatch, but that's it -- and the lack of map diversity were ultimately what made the game less than a classic. The gimmicks were great, and the game was fun ... for the first week or so.
What made Halo 2 such a classic -- such a true classic -- is that it offered so much diversity in its online play, giving rise to hundreds of custom games and giving it game life of years rather than days. It was this lack in Shadowrun that proved to be its ultimate undoing among my gaming friends and I suspect game reviewers as well.
On Rating Scales
Second, Gamers with Jobs writer Cory "Demiurge" Banks uses Gitelman's comments to criticize the 10-star rating system in general. I think he raises some good points -- all things being equal, a "5" would be an average rating, and "7" would be above average, "8" would be exceptional, "9" would be out of this world and "10" would be all but impossible.
As is, the rating system really runs from around 6-10, with most games coming in at around a 7. I think the reason for this is that, on average, most games are above average. And what I mean by that is that a game with a 5/10 is probably evenly balanced between its good and bad parts; at 4/10 you're on the downward slope to terrible, at 6/10 you're starting the climb to goodness.
At 7/10, the game is more playable than not, and has enough going for it to attract a group of gamers willing to overlook its flaws. And honestly, I think most video games that actually make it to market are able to hit this benchmark.
If you look at it this way -- as a rating being an indication of its ratio of good:bad, then the current system works. If you're looking at it as a measure of how the game compares to the average, well then yeah, the system needs some work.
In this, I think SCI-FI probably comes out better than the gaming magazines because of its letter grade scale; everyone (at least in America) understands that a C is an average grade, and C+ a slightly above average one. This hasn't stopped some people from lamenting the B- I've given their games (which, to my mind, is a pretty good rating) but I think overall it's a better system than the straight 10 point scale.