User Review/Member Score Issues

I, probably more than anyone else, appreciate that the new site has a way to search by Member Score. However, I have certain concerns with the feature:

1) Many games don't seem to be displayed in the proper order. For example, if I do a search for all Atari 2600 games by Members Score, "Frogger" is the first game listed, which has an 8.1 Average Player Score for the 2600. There are many 2600 games that have a higher Average Player Score than 8.1.

2) Many multi-platform games have no breakdown at all. For example, if you try to see what the platform-specific Average Player Score is for "Resident Evil 4," you get a message "We have no player reviews for Resident Evil 4 yet."

3) For many other multi-platform games, the aggregate number of platform-specific user ratings falls far short of the total userratings, and it's not clear where those user ratings went. For example, "SoulCalibur" has 3,364 user ratings, but if you try to break it down by platform, you get 766 ratings for the Dreamcast version and 1 rating for the IP version. There's also an arcade version, but you can't see the breakdown... and I find it hard to believe that the arcade version would account for the missing 2,597 user ratings.

4) For many, many multi-platform releases, the vast majority of platform-specific user ratings appear to have been assigned to one platform. This may be related to the fact that the new site doesn't permit you to rate a game by platform unless you write a review (the old site, from my understanding, had no such restrictions). As an example, "Batman: Arkham Asylum" has 18,748 user ratings for the X360 version, but only 1,854 user ratings for the PS3 version. I find it hard to believe that 10 times as many people rated the game for X360 than for PS3.

I know at some point, edgework and others at Gamespot were working on this, but the last I heard on it was that Gamespot felt that the user review breakdowns were 90% more accurate now than on the previous site. I simply don't think that's true, and I can find hundreds of examples like the ones I site above. I suspect that the site change may have wiped out some data about which ratings were assigned to which platform, and that this is the best Gamespot can do now. I don't have a problem with that (well, it's sad, but it is what it is), but I'd like to at least know that this is as good as we're going to get. Thanks in advance if anyone from the site addresses this topic!

I just looked up the user ratings for Lord of the Rings: Return of the King.

PS2: 1008 ratings

XBOX: 422 ratings

PC: 8,023 ratings.

C'mon. 8 times as many people rated the PC game as the PS2 game??? These numbers are obviously not correct. There's a systemic problem with the way the ratings for these multi-platform games are being assigned among the platforms. Given how many people have added comments to my post, maybe I'm the only one this bothers, but I'd still appreciate someone addressing it. Shrugging. Thanks.

I agree with you, please gamespot listen to us

X-Men Legends II: Rise of Apocalypse:

PC version: 319 ratings

PS2 version: 554 ratings

GC version: 213 ratings

XBOX version... wait for it... 7,746 ratings.

Sigh.

They just should have let the user scores like they were before the re-design of the site.

Here's a new issue - when I try to look at the user ratings breakdown for "Tom Clancy's Ghost Recon," all I can call up are the user reviews for the WII version. Strange.

"NBA 2k8":

Number of X360 ratings: 2,982

Number of PS3 ratings: 270

To some, maybe it seems I'm making a big deal out of nothing. But Gamespot is presenting us with purported facts. "This many people rated this game this way, and this many people rated this game that way", etc. I'm sure if I accused Gamespot of manipulating the user ratings intentionally to increase the average rating for one platform over another in return for kickbacks, Gamespot would deny it. But how much different is that from what we're being presented with? These numbers are clearly, obviously wrong. If Gamespot can't give us accurate information, then they should be up-front about it and shouldn't be giving us any information at all, not presenting their "best guesses" as if they were fact. It's sad that whatever Gamespot did in their site revision wiped out accurate information about what thousands of users did over the last decade-and-a-half, but it is what it is.

This is a big deal to me as well. I often used gamespot to see user reviews of the same game on different platforms. Let you decide which version to buy, as not every version of the same game is the same. Some have significant flaws vs another platform.

Well, look at that! I knew if I commented enough times about the problems with the filtering by Member Score and the off numbers of User Ratings, something would be done! Gamespot is listening! Because they changed the Member Score filtering! Woooo! And they....

... moved the filters to the top.

... and it looks like, when you filter by Member Score now, only games that have gotten 100 User Ratings or more are shown.

I guess that makes sense, though if 50 people are rating a game, that seems like a decent enough number for which to take note. They certainly could have done it the way Gamefaqs does it - by letting the user decide whether to filter only game that got 10 or more ratings, 100 or more ratings, etc.

Yet, as far as I can tell, my complaints have yet to be addressed.

For the ratings only reviews, in the past we had what I call a "worst-guess" algorithm to pick the specific release/platform to rate. For some dumb reason, in many instances it chose a Japanese release. If it didn't chose that it would make the worst choice possible so for example GTA-III was rated on Macintosh more times than on PS2.

We updated the algorithm and now its a lot more accurate. Its not perfect but its a whole lot better. We ran that algorithm on all the past ratings only reviews so they make more sense now. In the end though, we really have little to go by when determining the platform the user was on so the "most likely" platform may not be one that makes sense immediately but if more units were sold for PC than XBOX or if more users who rate a given genre are on PC that's the one we're going to pick.

Some time early next year we'll redo the ratings UI so that you can chose the platform when you rate.

Platforms associated to full reviews were not changed.

Didn't I tell you to put Excel away?

I understand that you're having to do the best you can with what you have now. My problem is that I don't understand it when you say that the problems originated before the site reboot. My recollection was that you could make a ratings-only review for a specific platform. And the numbers made sense before the reboot.

I'll give you one example (though I promise I could give literally dozens more):

Batman: Arkham Asylum - here are the numbers from August of this year:

PC: 5,899 user ratings average 9.2

PS3: 7,329 user ratings average 9.3

X360: 7,994 user ratings average 9.3

Here are the numbers now:

PC: 1,878 user ratings average 9.1

PS3: 1,854 user ratings average 9.1

X360: 18,751 user ratings average 9.2

I admit, I haven't looked up the sales figures for Arkham Asylum, but I'm pretty sure they didn't sell ten times as many X360 games as PS3 games. And the pre-reboot numbers, which were approximately the same between the two systems, seem much more reasonable to me. And that's my problem - if you're doing the best you can now because information was lost, okay, it is what it is. But I just don't buy that the numbers were inaccurate pre-reboot. The numbers looked accurate before the pre-boot, and now they look arbitrary.

A final note - you mentioned GTA III. I don't know what the Mac user review number was before the reboot... all I've got is the PC and PS2 numbers - 7,590 and 16,575 user ratings, respectively, as of September of this year. So the PS2 version got something over twice as many numbers as the PC version. That sounds about right to me.

Now? PC 1,903, PS2 22,679. I mean, tell me honestly - which set of numbers feels more accurate to you?

Shrugging. For more recent games, there's not a whole lot of difference between versions, so whether the majority of ratings get arbitrarily allocated to one over another probably won't affect the average anyway. This is really only a problem for people like me, who are compulsively trying to find a way to judge older games against one-another. Gamespot was once my best resource for doing so - I can't say that it still is. Still, thanks, as always, for your input and your efforts.

I just logged in after a few months and WOW. I don't know where to start but most feedback already has got me covered.

Yeah the ratings are screwed. Especially regarding multiplatform releases. Some very weird numbers - and I have been tracking down the number of ratings for games that I own in their respective platforms.

I'm extremely annoyed that I now have to create a stack from scratch with the games I've played (hello? they're the same ones that I've rated...). And when we add games to stacks, we cannot even choose platform... If I own a game for two different platforms how can I add them both to my library/stack/whatever... where to start... where to start... (a little off-topic, but a simple 'add to another stack' button next to a stack's game would be handy, just like your old libraries system).

Any update on this? Ratings are still being added to the "leading" platform. Just take a look at COD: Ghosts vote breakdown for christ's sake. And the problem is that most casual users don't even know/care that they're rating a game in the wrong platform. How hard is it to add an option for the user to choose WHICH platform release is he rating? Just like you seem to be doing when choosing which platform release to "follow". My "Winning Eleven 9" (Pro Evolution Soccer 5... jesus, where can I see the different regional releases names now...) rating is now attributed to PSP and my "Tekken Tag Tournament" one to ARCADE while I had clearly voted them on PS2 and I'm not sure how to edit that now... And even if you do address this issue now, it will already be too late for all those months votes gone wrong. Ratings have been skewed, how can they be restored to a reliable state again?

I test-rated COD: Ghosts and my rating was included in the PC release. I re-rated (re-clicked on the same number) and voila! Now I have 2 RATING ENTRIES with the same rating in my user reviews list (ofc I deleted them afterwards out of fairness). So would that mean that someone can double and triple-click the rating button and his votes will be counted multiple times to the aggregate average? (also it now says on my main bar "Ratings & Reviews (1)" instead of (0), even though I deleted the new ratings)

This is getting ridiculous, I'm checking wayback machine (gods bless it) to see the proper scores, lol.

Random example, Prince of Persia Classic (PS3 version):

http://web.archive.org/web/20130804062713/http://www.gamespot.com/prince-of-persia-classic/platform/ps3/

(98 votes, 7.4 user score)

while now

http://www.gamespot.com/prince-of-persia-classic/reviews/?sortBy=mostHelpful&platform=ps3

(18 votes, 7.1 user score)