Steam Updates Reviews System, Will Ban Developers Who Inflate Review Scores

Users who activated their game via a Steam key will no longer be able to submit a review score on Steam.

73 Comments

Valve is issuing a number of updates pertaining to the customer reviews system on its digital distribution platform Steam. As detailed on the official Steam store, the update will be affecting reviews written by users who activated the game via a Steam key.

Users who have activated a game using a Steam key (as opposed to buying it directly from the Steam store) will still be able to submit a written review, but will no longer be able to enter a score for the game. According to Valve, this change is being implemented in response to the system being abused by developers for inflated review scores, whom Valve found "duplicated and/or generated reviews in large batches."

No Caption Provided

Developers will still be able to do what they wish with the Steam keys for their games--whether that be giving them away, or selling the keys in other online or retail stores. However, Valve has reported that "an analysis of games across Steam shows that at least 160 titles have a substantially greater percentage of positive reviews by users that activated the product with a CD key, compared to customers that purchased the game directly on Steam."

While Valve acknowledges that there are legitimate reasons for the discrepancy such as Kickstarter titles and strong external audiences, the "abuse" of the system is "clear and obvious." The company will be ending its business relationships with developers who have violated the rules.

Valve has stated its intention to continue updating the Steam reviews system to address other issues, including reviews being marked as "helpful" without actually contributing informative content, or cases where a game's community has had highly divergent opinions in its reviews.

Got a news tip or want to contact us directly? Email news@gamespot.com

Join the conversation
There are 73 comments about this story
73 Comments  RefreshSorted By 
  • 73 results
  • 1
  • 2
GameSpot has a zero tolerance policy when it comes to toxic conduct in comments. Any abusive, racist, sexist, threatening, bullying, vulgar, and otherwise objectionable behavior will result in moderation and/or account termination. Please keep your discussion civil.

Avatar image for goodbye77
goodbye77

Show em who's boss, Logan.

Avatar image for gts-r288
gts-r288

Wait, steam games have a score? Never noticed it.

Avatar image for punksterdaddy
punksterdaddy

This is good.

It is just a way to determine for yourself if the game is actually good or there was some kind of shenanigans behind the reviews scores. There is nothing to complain about here.

Your reviews will still be there, (if you bought the game with an activation key) along with the Steam purchased ones. It is sort of a way of TAGGING them, allowing the consumer to see for themselves, that there may/may not be a discrepancy between Steam and Third Party purchases in the reviews.

Anything that can aid the consumer in their decision on whether or not the title being a good game, before a purchase is made, is a great thing indeed. Especially on Steam.

I don't know what all the fuss is about.

Avatar image for xanatos357
Xanatos357

Guess I'm not writing reviews for all the HumbleBundle games I just got.

Avatar image for ECH71
ECH71

Valve has reported that "an analysis of games across Steam shows that at least 160 titles have a substantially greater percentage of positive reviews by users that activated the product with a CD key, compared to customers that purchased the game directly on Steam."

Could that be because they got those keys for less from legitimate sources like HumbleBundle? A person who paid less for a game is likely to be less critical of it than one who paid full price...

Also, this new system stinks -- I've been a user since its launch in 2003 and the bulk of my games were purchased outside of Steam.

Avatar image for saturatedbutter
SaturatedButter

@ECH71: Humble Bundle is legitimate. The illegitimate ones are places like G2A.

But I don't think this has anything to do with other marketplaces. It's about certain developers that are either inflating the score of their game themselves, giving away free keys in exchange for good reviews, or outright paying for good reviews.

As long as you have no ties to the developers and the developers aren't incentivizing you to review their game, I don't think they care what score you give it.

Avatar image for metallinatus
Metallinatus

@ECH71: And you think that it's fair for you to leave a score for people who is buying directly in Steam when, as you said yourself, you are likely to be less critical for paying less somewhere else?

You will still be able to voice your opinion on the game, just not while changing the game's score with your "less critical" critique....

Avatar image for R4gn4r0k
R4gn4r0k

@ECH71: Same here, I buy most games at other stores than steam because they are cheaper there.

In fact the only real advantage of buying games on steam as compared to a boxed copy/Cd key/other online store is that you can ask for a refund on steam.

Avatar image for R4gn4r0k
R4gn4r0k

"an analysis of games across Steam shows that at least 160 titles have a substantially greater percentage of positive reviews by users that activated the product with a CD key, compared to customers that purchased the game directly on Steam."

Maybe because they got the CD keys cheaper than buying on steam ? :P

Avatar image for saturatedbutter
SaturatedButter

@R4gn4r0k: I really don't think this has anything to do with cheap keys. It's about developers giving away keys for good reviews, using keys themselves to post their own good reviews of their own games, paying for good reviews, or otherwise incentivizing consumers to give a good review.

As long as you have no ties to the devs and the devs aren't incentivizing your review, I don't they actually care what score you give, nor do they care how much money you paid for your key.

Avatar image for wbknapp
wbknapp

@R4gn4r0k: Yet, you really can't begrudge Steam restricting their review system to only those customers who purchased the games through Steam, right?

How many reviews like this do you see currently on Steam?

Me: wake up on an island. Immediately attacked by a lion. Pick up rock to defend myself. Head ripped off by lion claws. Hooked for life.

Anything that gets rid of this is an improvement. Too often, Steam scores are skewed by these fake reviews and consumers pay the price.

Avatar image for arkhenon
arkhenon

@wbknapp: Aside from the fact that those reviews will continue to be there. Don't think of the community as a whole too optimistically and all the horrible reviews being fake :) To fix that stupid review problem, they need to put a minimum character limit (with systems that do not allow repetition and so on, for a coherent review) and maybe put up a review format with fields to fill. There are a looooot of trolls online, and they will continue to be there. Steam purchasers are not an exception, I'm afraid :)

Avatar image for wbknapp
wbknapp

@arkhenon: Excellent points, all. It's clear that this change does not solve the issue and further improvements are necessary. Still, it is a start.

Avatar image for ECH71
ECH71

@R4gn4r0k: Damn, you beat me to it.

Avatar image for koospetoors
koospetoors

So when they finally decide to issue quality control on something for once, its neither for Early Access or Greenlight which are both an absolute cesspool, but the user review system.

Guys got their priorities straight!

Avatar image for Desmonic
Desmonic

Aww man, I guess Valve isn't bae anymore! What a shame!

Moderator
Avatar image for kadaverhagga
kadaverhagga

@Desmonic: I'd argue this makes them bae to more people. to us consumers. Or do you mean they are not bae of the devs anymore.

Not a perfect solution, but better than what's there now.

Avatar image for Desmonic
Desmonic

@kadaverhagga: Hey man, I'm not here to argue bae logics.

If that makes them more bae to you, better!

The only thing that matters is something being bae.

Moderator
Avatar image for saturatedbutter
SaturatedButter

I can understand why this change was put into place. But I kinda wish Steam could implement a system that tracks keys bought through legitimate re-sellers, and allow those buyers to still post scored reviews.

Avatar image for deviltaz35
DEVILTAZ35

@saturatedbutter: Everybody lies, this will do nothing to ensure people tell the truth about a game they review.

Avatar image for saturatedbutter
SaturatedButter

@deviltaz35: This update isn't really about weeding out customers who are lying about the game they bought. It's about weeding out developers who are using shady tricks to get loads of good reviews.

As long as they bought the game and aren't being incentivized by the developer, I nor Valve cares what they say in that review.

Avatar image for jonnybrownieboy
jonnybrownieboy

Valve should integrate their achievement and reviews. Only players who have earned an achievement after the credits roll can review the game.

Avatar image for turbojugend16
Turbojugend16

@jonnybrownieboy: If a game is so bugged that playing it is a torture, how can we, the customers, notify other potential customers that the product is bad? Your system is utterly flawed.

Avatar image for RedWave247
RedWave247

@jonnybrownieboy: See my response below to Steamaddict.

Avatar image for jonnybrownieboy
jonnybrownieboy

@RedWave247:

Alright what about this? Steam divides reviews into 2 sets. 1 for reviews regardless of playtime and 1 for reviews made by players that have completed the game.

Avatar image for RedWave247
RedWave247

@jonnybrownieboy: Seems like an unnecessary divide, honestly. Gameplay time shouldn't matter if the game is a huge pile of crap.

Avatar image for steamaddict
Steamaddict

If steam's gonna fix anything it should be reviews with less than 2 hours of playtime are not allowed

Avatar image for RedWave247
RedWave247

@steamaddict: Ordinarily, I'd agree with the sentiment, but there are two problems with that:

1) What about games that take less than 2 hours to complete?

2) What about games, like ones by Digital Homicide, that are completely terrible and obviously so within only 5 minutes of playing them? You want to punish gamers to play those games for 2 hours only to let the rest of the world know that those games are rubbish?

Avatar image for voljin1987
voljin1987

@RedWave247: and most importantly what about those people for whom the game never worked? They should have the right to leave their review regardless of play time.

Avatar image for RedWave247
RedWave247

@voljin1987: Bingo. Warner Bros wouldn't have pulled Arkham Knight if all those negative reviews and subsequent refunds hadn't forced their hand.

Avatar image for arkhenon
arkhenon

They could have easily implemented a system that monitors the discrepancy between the key reviews and Steam-purchased reviews for each game and removes the key versions' points if it exceeded a certain threshold. It's not rocket science either (seeing that they already did something like cutting ties with devs who do this). It's a rather simplistic data analytics application. Would not have required even slightly significant computational power as well, looking at the number of reviews submitted for even the most popular titles... Is there something between the lines here (Valve trying to "lead" people to buy from them only, or maybe something else) or is it just laziness/overreaction? I have no idea.

Avatar image for RedWave247
RedWave247

@arkhenon: Is it that easy, though? Places like Humble Bundle and Bundle Stars just give you a Steam code to copy and paste. How do you differentiate between those and ones handed out privately?

Avatar image for arkhenon
arkhenon

@RedWave247: It's not easy to do THAT part of course. But they are not doing it here anyway. They are shutting all the review scoring for Steam codes; dev given and Humble bought alike. And they are literally cutting business ties with devs who "likely" inflate their scores based on data, according to this news piece. Since they are already doing it based on data, why not automate it? That way most of the people who use KS or HS wouldn't be unfairly acted upon either.

Avatar image for RedWave247
RedWave247

@arkhenon: Well, they're not shutting down the reviews. You can still see them. They just won't affect the game's overall rating. Still sucks, I know, but at least they're not saying "Didn't buy it from us? Screw you, pal. No review at all for you."

Avatar image for arkhenon
arkhenon

@RedWave247: Oh they couldn't have gone that far anyway :) There would be a HUGE backlash. This is as far as they could have safely gone, from their point of view; if they wanted to retain their "consumer friendly" recognition. I still do not think this is a consumer friendly move though, sadly. People who would have read the reviews wouldn't be fooled by Steam averages.

Anyway, sorry for the rant. I just feel bad that they are doing something that could easily have been avoided (or better implemented). And I don't understand why, because I'm sure Valve has smarter analysts compared to me. Makes me suspect exploitative intentions, and it's giving me a bad taste.

Avatar image for iandizion713
iandizion713

@arkhenon: Of course they want you to buy from them mainly, its best way for them to control the reviews from getting out of hand. Its like killing two birds with one stone.

Avatar image for arkhenon
arkhenon

@iandizion713: Yes, but at the same time they are unilaterally mistreating people who kickstart games, or buy them through Humble Bundles or legitimate third-party sellers and etc. My point is, why not do this if and only if when it actually happens for a game. If and only if a game's score seems to be unfairly inflated, based on data (since they already found this problem through data anyway). That way they can - to an extent - differentiate between legitimate third-party consumers from malpracticing devs.

Avatar image for saturatedbutter
SaturatedButter

@arkhenon:

"why not do this if and only if when it actually happens for a game. If and only if a game's score seems to be unfairly inflated..."

I think you underestimate how much work that is. How much micromanagement that requires. Do you know how many games get released on Steam on a daily basis? You're talking about tracking every individual game's reviews. Then sending an employee in to investigate every time it seems like a game's review score is significantly changed by people who activated the game through a steam key. That is no small task when you are dealing with the volume of software that goes up on Steam. Or would you rather a system that retroactively auto-nullifies a large amount of reviews after they start significantly affecting the overall score? That is only going to cause more problems than it fixes.

Avatar image for arkhenon
arkhenon

@saturatedbutter: I believe I was mentioning "automating" it using data analytics, which is fair since they already decided that these certain devs were inflating the score of their games by comparing the average scores coming from the Steam codes with the copies bought from Steam itself. I never meant for them to manually do all that. That would have been crazy.

A threshold "discrepancy" value could have been used to check if the game/dev should be investigated manually (e.g., "if the Steam code score average is 10% more than the Steam copy score average, then let's look at this more closely"; or "the code average is 20% above, this is ridiculous, let's automatically remove the scores of the people who scored with a Steam code"). If you took some grad statistics/data analytics courses, you would realize how "streamlined" it is to say that some values are inconsistent with the actual set of values with a certain probability (like 95%, or so on; based on the confidence intervals that you are using). My point is, I'm sure Valve has people working on this with much, MUCH more expertise than most of the grad students. They should have easily been able to automate this (at least the process that would decide if there should be a manual investigation or not). The fact that they are blocking ALL of the scores of Steam codes instead of doing this in a smart way makes me think that it is not a "consumer" decision, but rather a business decision. That's all.

Avatar image for saturatedbutter
SaturatedButter

@arkhenon: Any auto-flagging system that requires a person to manually come in and investigate would be flooded due to the sheer volume Steam deals with. The manual investigations would be a level of micromanagement that Valve really doesn't want to get into.

I agree it sucks that all scores that comes from key users is broadly denied the ability to score their review. Up above I posted a suggestion that maybe they should have set up a system that verifies key sales from legit third-party marketplaces. So that places like Humble Bundle can let Steam know which keys they've sold. Then Steam can know that key belongs to a legit customer who deserves to score the game. I guess the issue with this is that it requires Valve forge business relationships with other digital game marketplaces and cooperate with them to provide a better experience for consumers. Valve is the big dog in the realm of online game marketplaces. They have no incentive to play nice with other marketplaces like the Humble Store. Those are deals that benefit third-party key sellers way more than it benefits Valve. So it sucks, but it's business I guess.

Avatar image for arkhenon
arkhenon

@saturatedbutter: A person will not investigate a specific score though, if you were missing it. They would investigate the whole game. It would not be too much of micromanagement I believe, especially if they first let an algorithm decide if a game has a misconduct in scoring or not. How many games do this kind of misconduct in a scale that affects the total score significantly, think about it like that.

When it comes to the last part: I agree. It is business as usual, and as the leader in the market Valve is not obligated to "play nice" as you say. I understand how business and ruthless monopolies work, thanks to the nature of the market that I'm working in. However, Valve and Steam constantly state that they are "consumer friendly/first" and they are also thought of like that by gamers (mainly because of the sales though I think, rather than any actual consumer friendly acts). If they want to play it like a regular monopoly, I'm totally fine with that. But then they shouldn't lead the public on and call it as it is. We have enough Lord Gaben memes anyway :)

By the way, the biggest problem here is Kickstarter I believe, not Humble Store. People who kickstart a game literally do not have the option to buy the game on Steam, because the game is actually being made with their purchase money (unless they repurchase the game on Steam after it comes out, which wouldn't make any sense).

Avatar image for saturatedbutter
SaturatedButter

@arkhenon:

"It would not be too much of micromanagement I believe"

It doesn't matter how much or little work the manual investigations are. Steam deals with so much software that it would get flooded. Sure, misconduct on a large scale probably isn't something that actually happens with high frequency. But the algorithm that flags these games isn't going to be anywhere near 100% correct. Inevitably they would be sending investigators to take a look into games where there's been no foul play.

Valve never likes to micromanage Steam. They automate everything, and they probably save a ton of money doing that. They don't even like micromanaging customer service. Steam has a high reputation among people who never need to interact with a human being working at Steam. Steam has so much stuff going on every day that any amount of manual work would cost a ton, require a lot of manpower, and probably still be inefficient.

Avatar image for arkhenon
arkhenon

@saturatedbutter: I would love to hear your opinion on the other points that I brought up, not just the one. Let's kick this discussion into gear :)

Also, for a platform that makes as much capital as Steam, that would be just a simple excuse not to invest just a bit more (it is pretty easy for a good data scientist to ensure the false positives would stay in pretty low percentages - it is just a question of what the value of "discrepancy threshold" should be; which is easily discernable using the allegedly huge data that Steam holds). It is still customer unfriendly in favor of earning more money. Which, as I said, is totally fine by me. I just don't like that they have a reputation of being "customer friendly" while they are taking the easy path on many things like this. It is unjust for truly customer friendly services.

Avatar image for iandizion713
iandizion713

@arkhenon: I think cause they are having a hard time catching people like they stated. Steam doesnt have to let third party sells codes, Steam can cut them off no problem. But Steam enjoys them, so most likely trying their best to work something out. Until thy figure something else out, this sounds like their only move.

Avatar image for chiefwiggum16
chiefwiggum16

They should ban them now

Avatar image for iandizion713
iandizion713

I didnt know Steam scored games? O yeah, the thumbs thing. Dang, one of the best reviewing systems just got better. Very impressive Steam. Pay attention Nintendo, your reviewing system is awesome, but not this awesome i might say.

  • 73 results
  • 1
  • 2