Sign on Options
Theme: [Light Selected] To Dark»
29Jun 07

Hey there! Welcome to the other side. It's been a full week since we announced details of our new rating system, and today marks the first full work week of us using that new rating system in all our reviews. So far, I'm pretty excited with how it's worked out. I feel like a jerk for even having to address this, but yeah, I actually mean that. I'm actually pretty excited. A few of you even insinuated that we, the GameSpot reviews department, has been "put up to this" by our "soulless corporate overlords" or something. A couple of things:

1) This rating system overhaul originated right here in the reviews department. We've been talking off and on about our rating system for years now. Late last year, we started formulating this current system.

2) Our corporate overlords aren't soulless. Weird, I know.

Naturally, the reaction to GameSpot's new rating system hasn't been 100 percent positive. It's perfectly normal to fear change, and changing something as serious as the way we recommend games is clearly going to cause some backlash. We've been doing this whole "web site" thing for a while now, so we knew going in that it'd set the message boards on fire. Actually, I'm surprised that we got as much positive feedback as we did. After all, it's a pretty major change. Regardless of your thoughts, thanks for posting in this blog and on our message boards in such a passionate way.

So, now that those posts have been stacking up for some time now, I felt it'd be good to take a few of the more common complaints, misconceptions, and questions and try to address them here. Let's start with the biggest one I've seen so far...

"You're dumbing down the review system by removing the component scores."
I'd say there's more to the rating system than there's ever been. Dropping the component scores has paved the way for the medal system, which is like taking the five component scores that we had previously and splitting them 60+ different ways. Before, when we gave a game a Sound 8, for example, you had to go in and figure out if that was because of the voice work, the music, the effects, and so on. Now when a game has particularly great voice acting, or an outstanding art style, or awesome competitive multiplayer, we can call that out specifically, right at the top of the review. On a similar note, if a game doesn't receive any medals at all, good or bad, you can take that to mean that the game has a lot of average of inoffensive qualities. Final Fantasy Anniversary Edition is our first no-medal review, and with a score of 6.5 backing it up, it's the sort of game that doesn't break anything--including new ground.

"Where's the difficulty and learning curve?"
Though they haven't actually been used yet, we have medals called Too Easy, Punishing Difficulty, and Just Right to reflect the things that matter about a game's difficulty. Once games start earning them, I think you'll agree that they say a lot more about a game than a box that just says "Medium, 30 minute learning curve."

"How was 8.2 vs. 8.4 difficult to understand?"
The old rating system was never really designed for the type of comparisons that many people used it for. In some ways, it still isn't. Our rating has always been designed to be an overall rating of that game's quality compared to the standards of the system it appears on at the time of its review. Once you start factoring time into this, comparing two similar scores and saying "well, obviously the 8.4 is better than the 8.2" actually might not be accurate. This becomes even more of an issue once you start comparing games from different genres. All we wanted to say with either of those scores is that the game is great, overall. This new scoring system better reflects our position on games.

To some extent, though, changing the review scale to 19 points lets it be a bit more meaningful for cross-platform comparisons, too. We've been deliberately avoiding any discussion about the re-rating of old games, primarily because we're not going to go back into the database and change any scores. But let's play out one "what if?" scenario, just for the heck of it. Take, for example, last year's Nintendo release, The Legend of Zelda: Twilight Princess. It received an 8.8 on the Wii. It received an 8.9 on the GameCube. But when it came time to assign the "best" tag, that tag went to the Wii. On the surface, that seems like utter madness, doesn't it? Now, of course, if you dig into the pages about our rating system, you'd get that we're comparing games to other games on the same console and so on and so forth. Under the new system, both of those games would have received the same score, leaving the "best" tag as your indicator for which version you should choose while simultaneously saying that both are great games. This more accurately reflects our position while also being easier for everyone to understand.

We feel that the scores of the old reviews mesh well with the new system, and that's a big part of why we stuck with a scale that stops at 10 and not some ridiculous star system or something. Also, this change definitely doesn't mean that our scores are going to start creeping higher, unless games start getting a lot better. Is it possible for a game to earn a 10.0? Yeah, it always has been, but it's going to continue to be exceedingly rare.

Similarly, our Editors' Choice award is a very big deal. We don't give it out too frequently, and it's going to stay that way. Only the best of the best are going to hit that 9.0 mark.

"I understand that you want to give the graphics score a different weight for a brain training game than for a FPS, but why don't you just do that? I don't see why you can't use a different scoring scheme for different types of games if you want to adjust the graphics score."
A different weighting system for each type of game would be a billion times more confusing than any review system we've ever had. Remember, the main idea is that our take on a game should be easy to understand, regardless of how interested you are in games. That's not about "appealing to the mainstream" or anything like that, because the people on our own forums, people that use this site every day, people who use the word "casual" in a negative sense, have arguments based on their own personal misinterpretation of our reviews.

"The text in The Good and The Bad just rehashes the same thing that the medals say."
Part of that is by design, but we're still experimenting with the best way to say what needs to be said in The Good and The Bad. I've been comparing the whole system to the inverted pyramid style of writing, which news stories use to convey the most important information at the top, and then get more specific as you go. At the top of our inverted pyramid, you have the review score. Some people will see that and stop right there. We think there's more to gain by reading the full review, but hey, some people aren't going to get past the number. That's fine, thanks for coming by, come back soon. But if you see the number and desire more information, there's the medals. Maybe you stop there, or perhaps you're still curious and proceed to The Good and The Bad, where things get more specific. At that point, you should have a fairly good base-level picture of the game in question. If you're left with questions at that point, then proceed to our full review and get all the specific details.

"How could you roll out this change without running it by us first?"
This gets right back to that whole "we fear change" thing I was talking about earlier. Additionally, it'd be hard to put out test cases of the new reviews without reviewing the games under both scoring systems, which wouldn't make any sense at all. Rest assured that we did put a lot of effort into this new system and bounced it off of various trusted individuals with a variety of interest levels in video games before setting it in stone.

If you got this far, thanks for reading! We're happy that you care about our reviews just as much as we do, and we definitely don't take these sort of changes lightly. Keep that feedback coming--we do read everything even though we don't have time to personally respond to all of it--and have a rad weekend. Oh, and if you're reading this on an iPhone, let me know how it looks. Thanks.

0 comments
Sort: Newest | Oldest
Conversation powered by Livefyre