Game Criticism - Is Scoring Accuracy A Good Thing?
Over at the New York Times, Seth Schiesel asks an interesting question about game criticism, given that a UBS survey reveals the relatively unsurprising fact that "top-rated games sell a lot more copies than bad ones."
What he asks, simply enough, is that "...if you look at film and popular music, for instance, there is almost no correlation between what critics say and what the public buy" - so therefore: "Are gamers actually more discerning than consumers of other media? Or is it just that game critics have more in common with game players than film and music reviewers do with the unwashed masses? And is this good or bad for the creative health of video games?"
This is a massively complex question, well-researched by the NYT writer, and I think the most interesting part of it is this comment from Schiesel: "It is worth pointing out that the only game among last year’s Top 10 sellers with a Metacritic score less than 80 was Brain Age, the mental-exercise software from Nintendo meant to appeal to users much older than the typical gamer."
So, does it look like the 'blind spot' for game reviewers (where the audience's enjoyment far outstrips critical reception) is casual games? I rather think so - especially given that a lot of major PC casual games don't even get reviewed by the majority of critics.









Comments
That's weird, I had heard the opposite just a couple years ago - that aggregate review scores (eg. metacritic) showed no correlation to sales. (Or was it only that metacritic scores didn't correlate to being a 'best-seller', which I guess might not be exactly the same thing?)
Posted by: josh g. | July 16, 2007 4:44 PM
"Or is it just that game critics have more in common with game players than film and music reviewers do with the unwashed masses?"
This one. There's several instances where the relationship breaks down, and it apparently frustrates reviewers no end.
Posted by: Merus | July 16, 2007 6:25 PM
What about these reports then:
http://www.gamasutra.com/php-bin/news_index.php?story=7453
http://www.gamasutra.com/php-bin/news_index.php?story=10924
Those state that the opposite is true; that game scores have no relation to sales.
Posted by: Ollie Barder | July 17, 2007 1:56 AM
I've seen other studies that say that Metacritic is one variable that helps predict sales, but there are, of course, others.
I think the reason you see such a close tie is twofold. (1) Games cost a lot more to experience than movies or CDs (in both money and time invested), so people have to be extra careful that they don't waste their money hence they read reviews to figure out what to purchase.
(2) Nerd culture is so inbred that we tell each other what is good and that _obviously_ makes it good so we tell other people how good something is. Let's face it, we breathe hype.
Posted by: Zack | July 17, 2007 7:48 AM
i think it simply has to do with the internet. it's so intertwined with our lifestyles by now, that something as electronically/community/computer-based as videogames would clearly find its home on the internet. the internet often serves as the basic mode of gameplay for some gamers.
whereas in other mediums, the criticism was always external to the works themselves, in obscure or even just unappealing places. who's really going to open up the newspaper to see what ebert said?
also, game reviews tend to come out very early in comparison with other stuff.
my theory.
Posted by: parkbench | July 17, 2007 8:26 AM
This article is really reaching in its conclusions. There are plenty of bad games that sell and more that don't. It's more about marketing and licenses than criticism and scores.
As for matching scores, few game critics go against the prevailing view, lest they incur the wrath of the angry fanboy. Film critics are given a lot more leeway.
While that's partially a problem with cowardly game critics who are afraid of, say, criticizing a Zelda game, I think more of the game folk are like the mass audience for videogames than film critics are for movies. The mass movie audience is pretty broad; you'd need reviewers who were soccer moms, blue collar workers, and whomever goes to see Norbit. The people buying Zelda are much more narrowly defined as "gamers," therefore it's easier to match the audience.
Posted by: steve | July 17, 2007 9:45 AM
Some major reasons:
1. Game reviewers are gamers first, so they're really embedded in the gaming subculture.
2. There is no formal education for game analysis and criticism, so there's a lack of divide between a writer and a gamer (see #1). Many reviewers have no formal education in writing, and those that do adapt an existing writing major (film, music, etc.) to what they write about. Those people are few and far between. If more reviewers wrote like N'Gai Croal then we'd see something much more comparable to movie and music criticism, where you have the general public going a different direction than more "learned" reviewers.
Posted by: Chris McDougall | July 17, 2007 11:17 AM
But that wouldn't explain how Beyond Good And Evil, Okami, Ico and Shadow Of The Colossus all bombed or how well Cars: The Video Game did which given the reviews.
Posted by: Spot778 | July 17, 2007 12:59 PM
"If more reviewers wrote like N'Gai Croal then we'd see something much more comparable to movie and music criticism, where you have the general public going a different direction than more "learned" reviewers."
The main problem is the public at large don't care about such a standard, and are more than happy getting 'gamer' reviews instead of someone who actually has some journalistic talent / experience. So why would the publishers who hire these reviewers push for a higher standard? It might even alienate their support base.
Posted by: Aaron | July 17, 2007 10:32 PM
The message is clear to producers: if you want a Top 10 hit, you essentially must have an 80+% rating.
Which comes first, the sales or the review?
The review.
Draw the inevitable conclusions.
Posted by: Percival | July 18, 2007 4:14 PM
Even if the reviews comes first, the scores may mean little to nothing in game sales. If most reviewers are of similar mindset to the buyers, then scores will be high for games that are popular, particularly when you cut it down to the ten best sellers. (Heck, some magazines/sites might find it in their interest to give higher scores to games that they know will sell well, just to encourage the idea that they know how their readers feel. Some games certainly seem to get free passes in areas that others get docked points for.)
The Ten Best Sellers might be dominated by games that scored 80%+, but how many 80%+ games sold extremely well, and how many sold poorly?
As well, isn't 80%+ a pretty decent range for "good" games? One of the top ten sellers didn't even manage an 80%? (Some might get away with arguing an 80% in a game review is pretty close to two-stars in a movie review.)
Posted by: Baines | July 19, 2007 1:38 AM
"Draw the inevitable conclusions."
Causation isn't the same as correlation.
Previews do more to influence sales than reviews. Most gamers have made up their minds about a purchase before the first review hits.
Posted by: steve | July 19, 2007 11:16 AM