Sorting user reviews

I was browsing the Watch Dogs user reviews, and I found the order in which they are displayed odd. First of all the default order seems to be by date, oldest first, which means that two pre-release non-reviews show up in the first spots. Newest first, best first, or some combination thereof would make sense.

Second, when I ask the filter to show "Most helpful" the order seems intuitively wrong. 5 upvotes and 27 downvotes would seem like a pretty bad score, yet it somehow ranks above 2 upvotes and 0 downvotes. It only gets worse in the bottom tier, every review with one upvote and at least one downvote rank above the reviews with only one upvote. Somehow the first downvote actually promote a review's position.

And just to add insult to injury the 5 up 27 down review is promoted as "Most useful positive review". What happened to most useful negative review by the way?

For some reason the 'most helpful' appears to be determined solely by the up votes, which is obviously a somewhat flawed mechanism, especially for reviews that get a lot of votes, as even if they're mostly down votes the amount of up votes still ends up quite high.

As for the default order - it does appear to be set to oldest first, which is also strange.

There's some other known issues I noticed such as deleted reviews still appearing in the list.

Not sure why the spotlight is for most useful positive rather than just most useful or having one for most useful positive and one for negative either, that may be something that just came with the GB base code that was used or it may be a design choice, I'm sure a staff member would be able to get an appropriate answer there though.

The sorting of reviews has been one of my major issues arising from the revamp of GameSpot in late 2013.

Displaying/sorting reviews using a thumbs up/down ranking potentially means newer reviews may never be seen as having no thumbs they go to the bottom of the review pile. I think a default setting of sorting user reviews from most recent to oldest works best as it allows new reviews a moment in the sun for people to give thumbs to. Keeping the thumbs up/down sorting as a default is to my mind a major disincentive to write reviews.

I have written about 100 reviews (pre-revamp) and many are for older games rather than latest releases ... I just take a while to get around to playing the games ... however when you open a specific games review page you see the "most useful" (or popular based on [thumbs up divided by total thumbs]) and then all the others sorted in order of thumbs.

For this reason I have not written a single review since the revamp as a newly written review, due to the default thumbs sorting, may never be seen except by users who may have me in their tracking lists. I wrote about this previously http://www.gamespot.com/forums/bug-reporting-feedback-1000006/why-am-i-the-only-one-seriously-upset-with-this-30861200/?page=1#js-message-341698232 Since then I have noticed it is now possible to sort reviews by date, score, platform (finally!) and "most useful" which is something new since the revamp and a welcome addition. A thumb up for that!

My suggestion is to revert to sorting reviews from newest to oldest for the default setting.

Finally, I am unsure about the term "most useful" as it is unclear to me whether giving a "thumbs up" necessarily means (1) a well written review irrespective of whether it is pro or anti the game, or (2) whether you agree with the review (i.e. you like the game too) and the contra for "thumbs down". Perhaps there should be thumbs up for (1) "good review" and (2) "agree" ... though this will be cumbersome. Best would be to base thumbs on review quality and leave the agreeing part to ones personal ranking of the game.

Update: The 5 up 27 down review has been removed, presumably because it wasn't a review. It however somehow manages to remain being the "MOST HELPFUL POSITIVE PLAYER REVIEW".

That is not how it works, it is not a simple division formula, I haven't been able to reverse engineer it, but it seemingly matters only marginally how many downvotes a review gets. And the default ordering is oldest first, the selection menu on the left is by default set to "Most Helpful", but that doesn't take effect until you click the "Apply filters" button.

It ends up meaning a bit of both, and I think that sort of works, the result is that well-written reviews that reflect the game well get voted to the top (or they would be if the algorithm for deciding what goes on top actually worked).

Not exactly true, there is some weight given to downvotes, for instance 3u-0d is listed ahead of 4u-5d, which is however itself ahead of the intuitively better rating 3u-1d. Somewhere in the GS code there is a somewhat complicated formula that ranks reviews. Someone should find it and replace it with something sane, for instance ([upvotes]+2)/([total votes]+4).

I have seen the "MOST HELPFUL CRITICAL PLAYER REVIEW" feature on some games, Dear Esther for instance has it (though the choice of review put on the spot doesn't make a lot of sense).

@Perth2008: A change to the default setting certainly would be ideal imo, and newest would probably be best for the time being at least as well. Oldest 1st is a bit silly for the default to say the least.

However, when it comes to thumbs it's largely down to how users treat it - and there's always going to be a way to abuse the system in the end. I reckon the best thing that can really be done here without a complete overhaul would be factoring in the thumbs down to some extent, as at the moment it seems to be based primarily on the thumbs up count. Incidentally, there was a lot of down-vote abuse on a GS site feature at one point in the past, forcing the removal of the feature, so having more weight on up-votes than down-votes does make sense with that in mind, even if not to the current extent.

@eBusiness: I've not looked into it much tbh, but it appears that upvotes have exponentially higher weight than downvotes - an extremely rough&simple approximation (that isn't what's actually used, but mostly lines up nonetheless) would be upvotes^3/total votes. So there's a huge amount of extra weight to upvotes such that with a significant number of upvotes they become all that really matter. The date is also factored into the ordering, oldest to newest, but that appears to simply be as a 2nd filter to order those with equal vote counts. The game score also factors in to the specific 'most helpful positive' one that is highlighted at the top of course. I'm not really bothered by precisely what the formula is, but it's pretty clear the current one isn't working well and needs some changing.

Interesting to see that some games have a most helpful critical review, though the selection for it is somewhat illogical - it's the same one as the most helpful positive review is despite there being an equally voted (+4/-0) review with a lower game rating, which would as such generally be considered as more 'critical' of the game. This is not paying attention to the review content itself however, assuming the entire thing is automated. With that only appearing on some games it's possible that it's not actually automated (or it could just be a feature in live-testing phases), though if that is the case the word choice could be better to point out that it's been spotlighted.