The Real Problem With Contemporary Video Game Reviews

This topic is locked from further discussion.

#1 Posted by juradai (2783 posts) -

I just read an article that I thought I might share with you guys.

Source: http://freedomcgc.com/real-problem-contemporary-video-game-reviews/

The author basically covers some of the inconsistencies regarding contemporary video game reviews and even goes into depth with specific reviewers. Some of the most recent reviews of Infamous: Second Son falls victim of some these inconsistencies.

Here are some excerpts below that I thought stood out.

Yet I have also, in reading hundreds of reviews per year, discovered that the biggest problem with game criticism isn’t so much about individual critics and their pervading philosophies and styles but rather a more simple issue, which is that critics are often wildly inconsistent and illogical, even within the subjective context of their own personal metrics.

Opinions, however subjective, must still be predicated on something, even if that rationale exists only within the mind and philosophies of the critic. If a reviewer derides a game for being short and that same reviewer praises another game two months later that is of equal or lesser length with nary a harsh word, this demonstrates something incongruous and indicates the criticism of the reviewer is arbitrary and nebulous.

It’s not that critics must be consistent with one another or even necessarily agree entirely within the parameters of a collective but at the very least any critic, professional or otherwise, should be beholden to their own ideology as to what defines quality.

Ultimately, any criticism is the dismantling of somebody else’s considerable efforts and while my own philosophy of humility as it pertains to criticism clearly isn’t shared by many, I don’t think asking critics to be consistent is unreasonable.

If anything, consistency would imbue their opinions, however subjective, with a much-needed measure of credibility.

I feel that he has some valid points, however, I also think that it is very difficult to remain consistent with the amount of reviews these writers have to go through. Anyway, it's worth a read.

What say you? Do you feel the inconsistency has grown over time in video game coverage? Do you feel that reviewers need to be held accountable for their inconsistencies or simply ignored?

#2 Edited by platinumking320 (663 posts) -

Maybe continued clarification of what game elements make the experience awesome or horrible or redeemable. Some out there are very good about it.

I feel Kevin Vanord, Jeff Gerstmann Jim Sterling and Totalbiscuit are pretty consistent. You know what they often look for, and even though we have our set opinions about what we'll buy, their common pet peeves, help pick out elements of a game that can be objectively measured by you when you read more than one review.

#3 Posted by LoG-Sacrament (20397 posts) -

i feel like i might have read something from that author before :P

anyway, i thought a few of his examples were good (the expectation of second son to have god of war-like combat and the skyrim reviews had nothing to do with consistency based on the information given). still, i'm not sure i'm convinced that reviewers keeping to their own general set of standards is a media wide problem despite the existence of a few individual examples. i still think "general consistency" is key as some games should be held to different standards. here are siskel and ebert on the same subject and i think it winds up being a good example.

there are still bad game reviews being written as i'm sure there will always be. i think the biggest issues facing games criticism are homogeneity and a fixation on surface level critiques and i think they're both related. however, i think reviews in general are improving on that front even though there have been individual reviewers giving good criticism for a while now.

#4 Posted by Jag85 (4279 posts) -

Interesting that the article starts off by focusing on David Jenkins, because he's actually one of the better critics around. I've been reading his reviews for over a decade (since the Teletext GameCentral days), and he was one of the few gaming critics to actually use the full 10-point scale (long before McShea and eventually GameSpot decided to use it). He's usually a good reviewer, but I can see some inconsistency in the way he gives COD a free pass for things he'd heavily criticize in other games.

#5 Edited by The_Last_Ride (69555 posts) -

@juradai: i got flack in System Wars for saying that reviewers on this site aren't consistent and often contradict themselves. I totally agree with this man/woman

#6 Edited by juradai (2783 posts) -

Maybe continued clarification of what game elements make the experience awesome or horrible or redeemable. Some out there are very good about it.

I feel Kevin Vanord, Jeff Gerstmann Jim Sterling and Totalbiscuit are pretty consistent. You know what they often look for, and even though we have our set opinions about what we'll buy, their common pet peeves, help pick out elements of a game that can be objectively measured by you when you read more than one review.

I think that comes down to simply getting comfortable with the reviewer over a period of time. It's as if you already know what to ignore and what to look for when you are so used to what they have reviewed in the past. I am like that with Gerstmann. Since I have read so much of his reviews I can anticipate what he is going to like and dislike. The cool thing about knowing your review thoroughly is if there is something he/she doesn't normally prefer in a game but found a particular game did a component they might otherwise would have disliked then it was something I was more prone to listen to, if that makes any sense.

I also thought one of the most consistent reviewers ever was Kasavin back when he was reviewing games. I really think there are very few who are able to balance their reviews the way he was able to.

i feel like i might have read something from that author before :P

anyway, i thought a few of his examples were good (the expectation of second son to have god of war-like combat and the skyrim reviews had nothing to do with consistency based on the information given). still, i'm not sure i'm convinced that reviewers keeping to their own general set of standards is a media wide problem despite the existence of a few individual examples. i still think "general consistency" is key as some games should be held to different standards. here are siskel and ebert on the same subject and i think it winds up being a good example.

there are still bad game reviews being written as i'm sure there will always be. i think the biggest issues facing games criticism are homogeneity and a fixation on surface level critiques and i think they're both related. however, i think reviews in general are improving on that front even though there have been individual reviewers giving good criticism for a while now.

You may have indeed read something from this author before.

I think you bring up a good point about homogeneity and a fixation on surface level critiques. It could be a case of playing it safe and maybe a fear of angering their fanbase which they risk losing their readership. I think it's almost an art of riding that fine line of being upfront with your preferences but at the same time being open to other people's as well. As I mentioned above there were, in my opinion, very few who can balance their reviews for a wide-spread demographic and show consistency in reviewing games like Kasavin.

@Jag85 said:

Interesting that the article starts off by focusing on David Jenkins, because he's actually one of the better critics around. I've been reading his reviews for over a decade (since the Teletext GameCentral days), and he was one of the few gaming critics to actually use the full 10-point scale (long before McShea and eventually GameSpot decided to use it). He's usually a good reviewer, but I can see some inconsistency in the way he gives COD a free pass for things he'd heavily criticize in other games.

Yeah, I am confused about the heavy lean towards Call of Duty as well but I think it's circumstances like this where we, as readers, should hold them accountable and delicately point that out to the author. Sometimes they give good responses but other times it allows them to reflect on their position and focus on staying consistent with future pieces.

@juradai: i got flack in System Wars for saying that reviewers on this site aren't consistent and often contradict themselves. I totally agree with this man/woman

Why you go into System Wars is beyond me, man. You're too good for that place. However, I do recall you mentioning some inconsistencies with reviewers on this site from time to time in these forums. Honestly, I haven't taken time to read reviews here recently in a while but when I did back in the day it did seem to be a little more consistent then what I feel it is now. However, it might just be me that is talking from an old man stance. :p

#7 Edited by udUbdaWgz1 (570 posts) -

I think the problem with reviewers is that they don't stick to the NON-opinion aspects of a game and concentrate too much on the opinionated fluff. I don't know how many times I've read a review and not known BASIC info about the game. it's a fundamental lack of "professionalism" from these yokels who quickly get into subjective issues like story and characters and graphics.

even when they talk about the gameplay I find that they are wrong or give an incomplete representation about 75% of the time.

it's why I love reading reader reviews because you get to see what they criticize SPECIFICALLY and what they like SPECIFICALLY.

most website reviewers talk about fluff and crap throughout their "reviews."

#8 Edited by Bigboi500 (29060 posts) -

Very inconsistent. Take graphics, for example. Last gen, Wii games got slammed for having poor graphics on a regular basis, and rightly so. Move on to this gen and games like Titanfall and Dark Souls 2 get absolute free passes here on Gamespot.

#9 Posted by Archangel3371 (15236 posts) -

Well in defense of reviewers I don't think that the time it takes to complete a game is an aspect that is or should be equally reviewable in all games. Take 3 different games that each take 5 hours to complete and they could each fall into 3 different categories. One could be to short, one too long, and one just right. It's all about content, pacing, presentation, and the overall experience.

#10 Edited by shellcase86 (1934 posts) -

Reviews are good for barometer checks, but reviewers are horrible inconsistent and illogical. It's good to use them, but don't depend on them. Contemporary reviewers contradict themselves day after day.

#11 Posted by wiouds (5014 posts) -

Reviews are inconsistent because games and reviewers are inconsistent.

#12 Posted by juradai (2783 posts) -

it's why I love reading reader reviews because you get to see what they criticize SPECIFICALLY and what they like SPECIFICALLY.

I'm going to agree with you here and say that I, too, have a preference for reading reviews from those that are not necessarily journalist or paid writers. I like to hear what the good 'ol gaming enthusiast have to say since there really is no pressure on them if they state whether a game is good, bad or ugly. Though not usually as polished as paid reviewers, most of the time they are speaking from the heart and love of the hobby which makes it genuine.

Well in defense of reviewers I don't think that the time it takes to complete a game is an aspect that is or should be equally reviewable in all games. Take 3 different games that each take 5 hours to complete and they could each fall into 3 different categories. One could be to short, one too long, and one just right. It's all about content, pacing, presentation, and the overall experience.

There is a Goldilocks analogy hidden in there somewhere isn't there?

@wiouds said:

Reviews are inconsistent because games and reviewers are inconsistent.

...what? Kinda lost me on this, Wiouds.

#13 Posted by loafofgame (421 posts) -

I'm going to take on a very personal perspective by stating I've never had any problems with inconsistency in reviews (although it is undoubtedly present). No matter how inconsistent reviewers might be, I myself am a very consistent person and I'll take from reviews (and more importantly, other sources) what I need. A reviewer criticising one game for being short and then letting another game off the hook is irrelevant when it comes to my choice to buy the game. As long as I know how long the game will take, I don't care if the reviewer is inconsistent.

I have never bought a game I regretted playing afterwards. I stay away from any trailers and other solely promotional material, which might negatively affect my expectations. I also never preorder and hardly ever buy at launch and wait until I'm absolutely sure I'm making the right decision. This means I have undoubtedly missed out on some games, but there's so much around and price drops happen pretty quickly. Luckily I'm in the comfortable position of not experiencing any peer pressure (none of my friends play games), so I can wait with my purchases and I'm also not pressurised into buying certain games.

That said, consistency can be very valuable indeed. And I would like to add transparency to that. Being aware of the context in which a game was reviewed is important when judging arguments. The whole misunderstanding about all the server problems surrounding BF4 (which the reviewer hardly experienced when reviewing) is a case in point. As a reviewer, report on that context. As a reader, try to create that context by being aware of what is missing and by not relying on one source. It puts things in perspective. But again, I would have also found out about potential problems in user comments and user reviews, so from a personal standpoint all these issues aren't a very big deal.

#14 Posted by Archangel3371 (15236 posts) -

@juradai: Yes, yes there is and it was purposely put there. *shifty eyes*

#15 Edited by jasonredemption (205 posts) -

I think the main problem with gaming reviews is the affect personal opinion has on actual scoring. It's one thing to say that certain thematic elements bothered the reviewer or that a particular genre or IP is not the reviewer's cup of tea. I think stating things like, "We had some issues with BF4's online connectivity so that docked the score." or "If you liked the previous inFamous games, you'll love inFamous: SS even if it doesn't do much to innovate new advancements in gameplay." I prefer the kinds of comments that help illustrate who should or who should not buy the video game. I've never played a Thief game in my life so the constant comparing of the 2014 Thief to previous Thief games is irrelevant to me. The reviews on GameSpot or IGN tend (generally) to be based almost solely on the personal opinions of the reviewer. Classic Game Room does this to an even greater extent, often living in the nostalgic past. I feel like the best reviews tend to be like Angry Joe who tries to give a deeper feel for the game's strong and bad points.

#16 Edited by Lhomity (770 posts) -

Reviewers are inconsistent because people are inconsistent. How a reviewer feels about game X in June, and game Y in August, is entirely irrelevant. What works well in game X doesn't necessarily work well in game Y. What might be a minor flaw in game X, may seem like a major flaw in game Y, and there may be any number of reasons for that.

Reviewing video games is not a scientific process. Reviews are opinions. They are based on the writer's own experiences and feelings about a particular game. If you don't want to hear the writer's opinion, read something else. If you don't want to read the writer's personal agenda or politics, read something else. These people are not employed to use magical psychic powers to evaluate how EVERYBODY ELSE feels about a game, and then write a fantastical piece that makes everybody happy.

That article is stupid. He's basically over-analyzing people's opinions about different games and completely missing the point of reviews in the first place.

#17 Posted by udUbdaWgz1 (570 posts) -

@Lhomity:

how somebody feels about the game shouldn't be the emphasis in a review: the cold, hard, facts should be. the subjective stuff should be kept to a minimum in so-called "professional" reviews.

i can give a review and not constantly refer to my feelings about how unimportant and foolish and lame the story in tlou is. i can even describe it's lame and mediocre gameplay without being subjective in my analysis.

#18 Posted by bussinrounds (1989 posts) -

Haha.. gaming journalism is pretty much a running joke at this point.

#19 Posted by Jacanuk (3850 posts) -

Haha.. gaming journalism is pretty much a running joke at this point.

ROFL just calling it gaming journalism is a joke.

The gaming media is nothing but hacks trying to pass their opinion off as something valuable.

#20 Posted by juradai (2783 posts) -

@Lhomity said:

Reviewers are inconsistent because people are inconsistent. How a reviewer feels about game X in June, and game Y in August, is entirely irrelevant. What works well in game X doesn't necessarily work well in game Y. What might be a minor flaw in game X, may seem like a major flaw in game Y, and there may be any number of reasons for that.

Reviewing video games is not a scientific process. Reviews are opinions. They are based on the writer's own experiences and feelings about a particular game. If you don't want to hear the writer's opinion, read something else. If you don't want to read the writer's personal agenda or politics, read something else. These people are not employed to use magical psychic powers to evaluate how EVERYBODY ELSE feels about a game, and then write a fantastical piece that makes everybody happy.

That article is stupid. He's basically over-analyzing people's opinions about different games and completely missing the point of reviews in the first place.

I think it might help me understand your position better if I knew your definition of "the point of reviews". Also, if you could point out where specifically in the article you think he is over-analyzing the reviewers he has mentioned. I just want to fully understand where you are coming from is all.

#21 Posted by wiouds (5014 posts) -

@juradai: Games as a whole is complex. There is graphic which can be broken up even more. There is story which affect the game. Then there is game play, and how good that is can be change by how the player plays. I do not play the same game the same way twice so how can you expect a person to play two different games the same way.

Also, two games can have the same flaws and one they do not look at it because there are things they they notice more while the other did not deal with the flaw as well.

#22 Posted by c_rakestraw (14579 posts) -

Very inconsistent. Take graphics, for example. Last gen, Wii games got slammed for having poor graphics on a regular basis, and rightly so. Move on to this gen and games like Titanfall and Dark Souls 2 get absolute free passes here on Gamespot.

Maybe that's because Titanfall and Dark Souls 2 look great? Just a guess.

#23 Posted by Jacanuk (3850 posts) -

@Bigboi500 said:

Very inconsistent. Take graphics, for example. Last gen, Wii games got slammed for having poor graphics on a regular basis, and rightly so. Move on to this gen and games like Titanfall and Dark Souls 2 get absolute free passes here on Gamespot.

Maybe that's because Titanfall and Dark Souls 2 look great? Just a guess.

Ehhm,,, Dark Souls 2 doesnt look great.

#24 Posted by Bigboi500 (29060 posts) -

@Bigboi500 said:

Very inconsistent. Take graphics, for example. Last gen, Wii games got slammed for having poor graphics on a regular basis, and rightly so. Move on to this gen and games like Titanfall and Dark Souls 2 get absolute free passes here on Gamespot.

Maybe that's because Titanfall and Dark Souls 2 look great? Just a guess.

Some games on Wii looked great too, like MP3 and SMG.

#25 Edited by c_rakestraw (14579 posts) -
@Bigboi500 said:

Some games on Wii looked great too, like MP3 and SMG.

Indeed they did. And they were rightfully recognized as such, no?

#27 Posted by jasonredemption (205 posts) -

@Lhomity:

how somebody feels about the game shouldn't be the emphasis in a review: the cold, hard, facts should be. the subjective stuff should be kept to a minimum in so-called "professional" reviews.

I agree with this. Yes, games make you feel. Yes, whether you like a game is going to be based on opinion. But just try for a moment to give us some cold hard facts to help us make our decisions. Tell us what type of gamers will like a game or not like a game.

#28 Posted by Bigboi500 (29060 posts) -

@Bigboi500 said:

Some games on Wii looked great too, like MP3 and SMG.

Indeed they did. And they were rightfully recognized as such, no?

SMG, yes. MP3, not so much. DS2 received a major graphical downgrade and still received a 9.0 with no mention of it. Of course, Kevin V is a HUGE Souls games' fan, and has reviewed every one of them here.

http://www.forbes.com/sites/erikkain/2014/03/20/dark-souls-ii-was-unplayable-and-broken-on-consoles-before-graphics-downgrade/

Titanfall isn't a very good representation of what the Xbox One is capable of, graphically. It's been mistaken for a 360 game countless times. Most other games would have received a negative for something like that.

It's just funny that GS put so much emphasis on graphics last generation, and so far this gen it's been a non-factor. Just one of many inconsistencies in gaming "journalism", and that's from just one website.

Another would be docking points because of similarities to previous titles--games like Pokemon and Metroid Prime 3 are penalized for this, while games like Call of Duty and Assassin's Creed are given passes. Probably because they're "blockbusters" and generate more traffic.

Big devs like EA and Ubisoft have been known to offer gifts like vacations etc in exchange for better reviews. Devs like Nintendo are held to a harsher standard because they don't do things like that. Then there's the Jeff G incident here involving Eidos and add revenue which led to his firing for giving a low scoring review of Tomb Raider. The mysterious changing of GTA IV's score from 9.5 to a 10.

The only defense people have against that stuff is plausible denial. That's a cozy little blanket to wrap one's self in for protection because there will never be documented court evidence of such shifty activity, but all the stories and controversies do add up over time.

I have no idea why so many gamers continue to trust and depend on review scores as though they hold any scope of value or worth, or have any honest standards about them.