Gamers swayed by review scores - Study

EEDAR study finds gamers more likely to like, buy, recommend a high-quality title if first exposed to critics' thumbs-up.

Gaming executives often emphasize the importance of a game's critical reception, and a new study by Electronic Entertainment Design and Research and the Southern Methodist University offers one reason why. The study, which measured how game reviews affect perception of quality, purchase intent, and likelihood to recommend to others, found that positive scores can dramatically elevate a high-quality game in the mind of consumers.

Plants vs. Zombies won the hearts, and ate the minds, of critics.

Conducted in conjunction with SMU during the week of March 29, the study involved 188 participants (23 of which were disqualified as inappropriate for the research). The study made use of Plants vs. Zombies, a highly regarded tower defense-style strategy game that participants had no prior knowledge of. Participants were divided up into three groups, with the first initially exposed to high-scoring reviews before participating in a 20-minute play session. The process was repeated with a group who were shown low review scores and a control group who was shown no review scores before its play sessions.

Following the play session, participants were tasked with providing a numerical score to the game on a 100-point scale, as well as selecting whether they'd prefer to receive $10 in cash or a copy of the game as recompense for their time. The study's results found that those who were shown a high review score on average rated the game six points higher than the control group and 14 points higher than the group shown negative scores.

"Even with first-hand experience of time spent playing the game, those exposed to high review scores give the game a higher review and those exposed to low review scores gave the game a lower score," the study noted. "Clearly, this initial review exposure influenced their opinion, even after they played the game themselves."

The data also indicated that 38 percent of those shown high review scores took a copy of the game over the $10 payment. That figure dropped to 21 percent who took the game from the control group and 17 percent from the negative-score group.

"The findings of this study indicate that critic reviews, independent of product quality, significantly influenced participants' willingness to purchase the product," the study found. "As painful as it may be for developers to consider, even with the creation of a high quality game, a game is likely to achieve greater commercial success if reviewed highly, than if reviewed poorly or not at all."

Continuing the trend, those shown a high review score also said they would be more likely to recommend Plants vs. Zombies to a friend. Ninety-one percent of those shown high review scores said they'd recommend the game, whereas only 65 percent of the negative-score group and 79 percent of the control group said they would.

"The most surprising result is Group B (low review exposure), where the majority (65 percent) indicated they would still recommend the game to a friend even though 73 percent of the group would not purchase it themselves…Group B's review score was higher than the anchored review score of 61, indicating that, while reviews influenced their behavior, they still felt that the game was much better than what the 'experts' suggested. This suggests that if a company releases an inherently good game that receives lower than 'deserved' review scores, the company may be able to increase sales through a widely played demo."

As for other real-world applications, EEDAR's study noted that publishers could be well served to offer select media outlets a period to review a game prior to release, for the purpose of selecting choice critique for use on box art. However, the study notes that this method would only work if the prerelease review process remained untainted and that the game itself is of a high degree of quality. These blurbs could also be used in game demos, as well as other marketing campaigns.

EEDAR also said that its conclusions only apply for those games regarded by professional critics as high quality. The market research firm noted that a variety of other factors could contribute to a game's success, or lack thereof, including "marketing, pricing, release timing, and brand awareness."

Written By

Want the latest news about Plants vs. Zombies?

Plants vs. Zombies

Plants vs. Zombies

Discussion

335 comments
Philly1UPer
Philly1UPer

@GBPackersrule Yes, I did play Kane and Lynch and it was terrible, and it was soposse to have a pretty low score on here originally, Then Gamespot, fired said well respected reviewer for simply being honest, and instead wrote a new review for it and gave it a medicore.decent review.

GBPackersrule
GBPackersrule

@Philly1UPer Did you even play Kane and Lynch?

golgo28
golgo28

all the study proved is nothing new.its the same with movies,music and books.most ppl are sheep and will go along with what they're told.is there a possibility of devs paying reviewers to give a good review?yes.big deal,if a person is to stupid to make his/her own opinion of a game.without being influenced by reviewers.then he/she is an idiot and shouldn't breed probaly.

Null7
Null7

I know reviews are important to some. But personally, I read them and if I felt that the review showed weakness in the game (like many errors or glitches or bad game play or story... etc), and I agreed with the review; I would not get the game. But if I felt that the review was harsh and made no sense... I would get the game. Ex: Kingdom Hearts Re:Chain of Memories on PS2 I played it many times and I loved it, but reviewers gave it 5 to 6/10! Also the PSP, many people or experts say it's not that good, I still play it, and surprisingly, the most enjoyable game for me this year was on the PSP! (MGS: Peace Walker). It's really up to the person and his taste in things. While reviews may give you a piece of what to expect from the game, you should not build your taste according to them!

pizzamuncher
pizzamuncher

So this would mean the sonic series will be killed by IGN and gamespot. Sigh.

krytorii
krytorii

This only really affects those who read reviews tho, so the "casual" market is gonna be less affected by this than the "hardcore" group. In other words games like wii music and such will still sell even if given -3/10

KokutenNagaki
KokutenNagaki

@mrzero1982pt2 I don't know when it suddenly became weird to know your getting a good game before it. I don't know when $60 suddenly became money that could be thrown around. Yeah, I do look at reviews, but I don't do it in the way you suggested. I listen to a lot of gaming podcasts and I find someone with similar taste as I do. Take Gamespot, Kevin VanOrd is my favorite reviewer because the games he gives good scores are mostly ones that I really enjoy, like Assassins Creed. Lets take Gamespot again, me and Brendan Sinclair do not have the same interests. So when he says he didn't like Heavy Rain, I know that I will. So no, I don't look at player reviews, why? Because most people hate being wrong and most people already have a bias when they spend $60 on a game they were hyped to be good. No one wants to be told what they did was stupid and that's were I value a professional, more the likely it was given to them for free so what do they care if its bad?

lazycomplife
lazycomplife

[This message was deleted at the request of the original poster]

Philly1UPer
Philly1UPer

@Mirha77 Please. Gamespot actually fired one of their employees, one of their top guys actually, for giving Kane and Lynch a terrible review (which it deserved) But the money was more important then a honets opinion. So the editors on this sites opinion obviously doesn't mean anything to me, because theya re paid off by Microsoft, Sony and Nintendo to give crappy, over hyped games a great review.

Toysoldier34
Toysoldier34

@bowlingotter Exactly, like I personally don't like Halo. I loved the first, then the second was okay and I don't like the third. ODST was fun for the atmosphere but it also strayed from the others a bit. And whilst I may not like it and personally to my it would score in the 5-7 range. Professionally I would never give it something like that because I know it deserves better in the 8-9 range. If companies devoted to reviewing games or any media can't get past the blindness, then they truly shouldn't be allowed to review games, especially when it counts and peoples lives, jobs, and incomes are on the line. Because there is no arguing that reviews do affect a games sales. Movie reviews are so far across the board they don't matter at all, you know there is something wrong when the best movies out there are swinging a solid 60-70, which is equivalent to a 90 for games because movies suffer heavily from this plague.

ssh83
ssh83

lol @ some comments. Just because you "know" something to be true via your opinion/observation, that carries 0 weight in the real world because nobody gives a damn about a joe-nobody's opinion. A scientific research however, holds some weight and it actually matters. The experiment settings and analysis in this research are pretty well written, too. I'd totally support this research team in hope of them making more competent research on other topics in gaming. Would be better if we have link to the actual paper though...

DrunkenStone
DrunkenStone

This is a thinly veiled ploy by GameSpot to increase ad revenue and secure further sponsorship from the game studios translation: "If you help GameSpot make money, we'll help you make money"

Raykuza
Raykuza

I honestly think that numerical scores are a bit useless. In order to assign a score on a scale you need points of reference off of which to judge. Also, by that logic no game can ever achieve a 'perfect score' because there is no such thing as a perfect game, but then again what is the point of having a scoring system if nothing can ever achieve the top score? And who has the right to assign these scores? Etc etc. I do trust well-written in-depth review content, however. Most of that is objective information about the game that is useful when considering a purchase. Even if the reviewer asserts his or her own opinion on a certain aspect of the game, the reader can decide how valuable that criticism is to them. For example, if a reviewer says, "this game has a lot of glitches; I didn't like that," the reader can say, "I care deeply about the quality of my experience; I hate glitches; I won't purchase this game," or they can say, "I don't care about glitches; this game still seems fun to play; I'm buying it." Either way, the review has helped them come to a decision. And reading multiple reviews increases the effectiveness of the research.

Glade_Gnarr
Glade_Gnarr

I still played dante's inferno even after reviews and it wasn't so bad.

Neo_Gamer81
Neo_Gamer81

...and in other news, the sun came up today.

kvan33
kvan33

There are plenty of studies out there that show the power of suggestion. No surprise here. But let's not make excuses for bad games. They usually get the reviews they deserve. I think the key is pay attention to the content in the review than just the number the reviewer gave the game.

Uwanq
Uwanq

I agree with so many here regarding consumers as the 'sheep' to gaming reviews. Although gaming sites and other platforms such as magazines or tv shows are thought to be neutrally objective. There has been many occasions in the past where the reviews have been influenced by the amount of money a sponsor throws at the media to 'boost' the profile of the title they are selling. Another point being there has been titles in the past that have slipped under the radar of a lot of review sites and the general public have not heard a thing. Due to the general consumer market being too busy or in fact too lazy to research available titles, these have unfortunately slipped through the cracks.

Sgtspud05
Sgtspud05

Great reviews though, think i'll buy it.............. :)

Gezquester
Gezquester

Ahh what a shame, personally I don't give a damn what reviewers or other people say about games. If it looks interesting to me and/or I had fun with the demo, then I'm getting it! I remember when I first got my PS3, the demo I played the most was Uncharted: Drake's Fortune, I must have run through that demo 40 times. That is the sole reason I bought the game in the end, and subsequently UC2. Thank god I did, two of the best game purchases I have ever made. If you're swayed by critical opinion, whatever way it leans, even after trying it out yourself, then, well, you don't really have a mind of you own.

gbrading
gbrading moderator

This is the sad reality of gaming criticism; people believe wholesale whatever criticism might be thrown out, even if it is ill-informed and badly researched. Above all, you must maintain your own independent and non-partisan opinion.

DrEvilKoala
DrEvilKoala

This entire article is an understatement. I think a large portion of gamers mimics their opinions to critics opinions. I'm sure that if you would let people voice their opinions on a unreviewed game the opinions would be less conforming if they were talking about a reviewed game. I personally don't give a damn about scores. I only read reviews for the information they give me. I don't care what one person thinks about the game. I just want to know if the game mechanics suit me. I bought the 360 version of Divinity II even though IGN gave it a 4.7 because it was "unplayable due too bugs". It was one of my favorite games of 2009, and I encountered about 4 minor bugs. I'm sure a lot of gamers miss out on great games because they follow critic reviews way too much.

AlphaCommando
AlphaCommando

This is a sad reality for hidden gems given average scores.

ebonyflame
ebonyflame

Wow, they needed a study to realize game reviews and their accompanying scores influence game purchases.

dmfreak
dmfreak

Well I too listen to some critics if I find a game and even a movie, in which I'm not so familiar with it, interesting. But not always especially if the game or movie they review is overrated.

Sta1k4r
Sta1k4r

Isn't that the point for review scores? To tell people if it is a good or bad or unplayably buggy game?

Blade8Aus
Blade8Aus

like, no way man... you're crazy....

Sirbobislost
Sirbobislost

@ Gelugon_baat Thing about the last CnC the moment I found out basebuilding had been removed I knew it was gonna suck hard EA have steadily destroyed CnC with each release generals was the last good one Cnc3 sucked and RA3 was a joke I only trust reviews to a point I check what a few gaming sites see what they say and then check actual peoples reviews I mean look at ff13 I wondered how an ff game didn't get it's usual 9 out 10 I'd like to know how it got an 8 really never been so disappointed by a game

wahyudil
wahyudil

cool research ... and it also proof that people tend to believe what they hear and read rather than what they see ...

acer_062
acer_062

How many years did it take to get a degree to fiqure this one out !!!!!! uuuuh !!!

Aletunda
Aletunda

pretty obvious, the problem is, is that people are sheep and are easily influenced by these reviews when the people writing them are just regular people, i find it odd that people are so easily influenced in their choices just by listening to other people they don't know who have different gaming preferences and opinions

jamyskis
jamyskis

I'm influenced, but not convinced by review scores. Firstly, I know how much influence the industry has on the media, and I'm always a little sceptical about how impartial the review scores are. Secondly, some reviews, even those from major publications, have been reviewed under such poor circumstances, that the review cannot be classed as reliable - why, for example, would a magazine place an RPG fan and FPS hater to review Modern Warfare 2? Happened here in Germany though.

LorenaLarue
LorenaLarue

Someone paid a bunch of people to do a study with such an obvious conclusion. What a way of waste money.

Saphire_Kitsune
Saphire_Kitsune

A lot of people are not getting the meaning of this study. They're not just saying that good reviews mean more buys, they're saying that people's post-play view/opinion of the game was affected BY the review. i.e. Good game shown with bad reviews=okay/bad game to them. Good game shown with neutral/no reviews=good game to them. Good game shown with great reviews=great game to them.

stakex007
stakex007

@Mirha77 - Ever heard of a preview article? A preview of a game is not a review, its a general overview of what the game is going to offer with little opinion and often no score... I kinda thought it was easy to assume what I ment when I wrote that. Guess some people need things drawn out for them, so sorry about that. Also, reading what other players have said about a game is also not really a review, and even if it is a player-review... they tend to be far more honest and less skewed by outside factors such as hype or pressure. Sure you have some idiots out there that will trash any game, but its normally pretty easy to weed them out because they often go way over the top.Sure, this study might be small... but Im pretty sure its safe to say that anyone who thinks a larger study would show anything different is an idiot. A whole ton of people wait for reviews before they go out and buy a game, and its not un-common to hear "Im waiting for the review before I buy" on forums or even from customers in game stores. I'm not saying its bad to consider reviews from paid sites... just that if thats all you follow, you're going to play a lot of the same and miss out on a crap load of original games.

JDWolfie
JDWolfie

You don't need expensive research to find this out or be a rocket scientist. Yeah, like the whole time, we were just buying games randomnly.

Mirha77
Mirha77

@Philly1UPer And people who are paid to do reviews typically have a much broader experience with games because their mere profession exposes them to far more games than your average gamer will ever even consider. Plus, the reviewers I read are gamers too. Why are their opinions less valid than your average 17 year old who wants to tell the world just how cool his favorite new game is? I was just making the point that it's a contradiction to tell people that it's fine to make your decision based on other people's opinions or on articles, but it's not okay to make your decision based on a reviewer (who is just a person putting their opinion in an article). Really... they're not the puppets of big game companies. They're just journalists.

Petch1984
Petch1984

With the internet and downloadable demos it's very easy to only buy good-great games. I have on the rare occasion gone for a game with mixed reviews probably because I'm either a fan of the series, know what to expect, or the game has a really interesting feature.

thetravman
thetravman

Why do people research on obvious facts? I buy games based on reviews and game companies all the time. This is actually a good thing because it'll teach the lazy developers to get some quality work done.

dancingsmurfs01
dancingsmurfs01

188 - 23 = 160 people not enough an actual experiment and get a good answer you need to test it multiple times with different people and/or games

Amnesiac23
Amnesiac23 moderator

Personally, I know the type of games I like and don't really care about review scores much anymore. I like to know the game is playable before buying it, but that's about it. But I can see how it's easy to be swayed. If a game gets a low score, you will be more inclined to sit there and try and find out why it got a low score. Thus, you will be only thinking of the negative aspects. This doesn't just go for game reviews though. It can be the same for all forms of media.

gamebreakerz__
gamebreakerz__

Someone actually did a study on this?? Seemed pretty obvious to me.

Philly1UPer
Philly1UPer

@ Mirha77 That may be true, But you're reading a review from someone who isn't being paid and are simply doing it because they love video games and want to voice their opinion.

Mirha77
Mirha77

@stakex007 You say "Read what players have said about it, rent it, read articles about it, or even play demos... " Wait, isn't reading what other players say about a game in an article called... a review? Besides, this is a fluff piece on a story with a 165 person study that essentially says "expectations influence perception... by about 15%" It also points out that "marketing, pricing, release timing, and brand awareness" were not taken into account, meaning these people had NO idea about the game overall before the study. In other words, this isn't the uber-hyped Modern Warfare, Grand Theft Auto, or Halo. So... you know... Don't base your world perspective on the difference of ten random people in a survey. It's really not worth it.

thingta42
thingta42

Well ive bought a few games in the past that got low reviews and i enjoyed them. Normaly for me i look at several gaming websites. IGN, Gamespot, Gametrailers and then i round up all the good points and bad points they have made. Starship Troopers, AVP3 gamespot gave a 3.8 and a 5.5. Did i agree with the points made on the reviews? yes some but i enjoyed the game dispite the mediocre reviews both games got. And i for the most part only spend my money on trusted names. I.E i'll always buy a soul calibur game or a Battlefield series game dispite reviews. Some games however are underated, AVP3 fantastic game in my opinion, yet it got bad reviews, and some good ones. So to me it's a love hate kind of game. Star wars force unleashed got ok reviews, the game had quite a few issues but that didn't stop it from being one of the best selling star wars games ever. My point is, that if theres a game someone is ones, I.E a star wars game where you are a Jedi/Sith hacking stuff up, using the force in cool ways, your gonna end up buying it anyway dispite reviews, because it's what you have been looking for.

jerim007
jerim007

I've bought games that got garbage reviews and loved them, and I've played games that got high scores that I thought were utter crap. The fact that I even read the reviews amazes me, but I still read 'em. I think I just convince myself that they're idiots. It is a semi-interesting study, but loses credibility w/o further knowledge of the participants. Had they ever played games before or did they just round up a couple hundred students at the cafeteria after saturday lunch?

The-Techspert
The-Techspert

I usually dont buy 90% of the games that I see reviewed/in the store anyway, usually I only buy games that I knew I was going to buy before the reviews came out (Mass Effect series, Ace Combat series, Assassin's Creed series) However, I have noticed that review scores can sometimes influence my opinion of the game after the fact, though not incredibly.