PressSpotting: What reviewers can say and when

Although this column usually tries to look at the broader issues and personalities in the game-journalism world, there are plenty of smaller topics swirling around that don't necessarily warrant a full column of their own. The PressSpotting Round-Up will take a quick look at some of these issues on...

Although this column usually tries to look at the broader issues and personalities in the game-journalism world, there are plenty of smaller topics swirling around that don't necessarily warrant a full column of their own. The PressSpotting Round-Up will take a quick look at some of these issues on a periodic basis.

What to Say When They Say You Can't Say That
The wider gaming world got a peek at the complex give-and-take between journalist and game publisher earlier this month. MTV's Multiplayer blog was the one that pulled back the curtain, reporting that Konami was prohibiting those with early review access to Metal Gear Solid 4 from discussing the game's install times and cutscene length in their reviews. The rumor was essentially confirmed later that day when IGN UK's review of the game admitted, "In return for letting us play Metal Gear Solid 4 before its release, Konami issued us with a list of things that we're not allowed to discuss." A few days later, a Kojima Productions team member revealed to Kotaku the exact contents of that list. A few days after that, 1UP's Jeremy Parish added the shocking revelation that Konami "presented [the nondisclosure agreements] to us at the literal last moment...and made it clear that we wouldn't be leaving until we signed them."

Nondisclosure agreements are a pretty common prerequisite for reviewers who get early access to games for review. These agreements usually preclude the journalist from talking about the game before a certain date, or, occasionally, from discussing certain story elements that might spoil the game for the reader. But Konami's NDA went even further, asking reviewers not to reveal technical details that were directly relevant to any fair review of the game. GameSpot's Kevin Van Ord probably put it best on this site's own Reviews Blog: "I believe it is flat-out wrong for any publisher to request that pertinent information be left out of a review. The journalist decides what information is most important for the review--not the developer or its publisher."

So what should a journalist do when confronted with demands like these from a game publisher? One option is to offer a slightly delayed review of a full retail copy of the game, which is exactly what outlets such as GameSpot and Electronic Gaming Monthly did in response to the Konami situation. Nevertheless, this is a less-than-satisfying solution because the delayed review is both less useful to the reader and less lucrative to the outlet publishing the review. The better option is probably to leak the unreasonable publisher's demands out to the press anonymously, allowing the court of public opinion to punish the company for their overreach. As long as the press continues to adamantly and vociferously call out these types of situations when they come to light, publishers will eventually learn that trying to restrain what a reviewer can say about a game is just not worth it.

Battle of the Associations
With all of the attention paid to the Konami NDA controversy, you may not have noticed this month's shouting match between the Entertainment Software Association, which organizes the annual E3 trade show, and popular news blog GamePolitics, which is owned by an ostensible ESA rival, the Entertainment Consumers Association. Here's the five-cent version of what happened: The Escapist pointed out that Texas Governor and E3 keynote speaker Rick Perry supports some controversial statements from controversial pastor John Hagee. The story then got picked up by GamePolitics. All fine and good, until ESA Director of Communications Dan Hewitt issued a statement to Joystiq saying that GamePolitics is "tainted with anti-ESA vitriol" and accusing the site of being more of an ECA recruitment tool than an impartial news site.

This is pretty clearly an overreaction on the ESA's part, in my opinion. Yes, there is an inherent conflict of interest in ECA-owned GamePolitics covering the ESA, and the site should probably do more to disclose this conflict when its writers discuss the E3 organizer. That said, claiming that GamePolitics has a history of "anti-ESA vitriol" just isn't supported by the facts. Yes, GamePolitics covered the ESA's recent troubles retaining members, but so have countless other sites that have nothing to do with the ECA. What's more, GamePolitics' coverage has been relatively moderate compared to the blistering portrayals of the organization in some corners of the gaming blogosphere. There's a difference between being owned by a company and being a paid shill for that company. GamePolitics is clearly the former but not the latter.

The Anticlimactic Entertainment Expo?
Is it me, or does the buzz about this year's E3 seem a little less buzzy than it has in years past? Maybe it's because a lot of the behind-the-scenes action for this year's show is taking place weeks before the show actually starts. Already, large swathes of the press have been out to Los Angeles to see pre-E3 presentations from the likes of EA, LucasArts, THQ, Sega, and Sony, just to name a few. A select few journalists even went back to check out more games as part of the judges panel for the E3 Game Critics Awards. Although much of the information revealed at these meetings is embargoed until the show officially starts, this stuff has ways of leaking out through the journalist grapevine and making brand-new games seem old even before they're officially unveiled. Beyond the preshow events, the leaked contents of a preshow marketing survey have stolen the thunder from a number of announcements that Microsoft and Activision seemingly had planned for the show.

Though these events seem to point to a rather pointless E3 this year, I wouldn't be so quick to assume that there won't be some blockbuster announcements. The fun of a major trade show like this is that you never know exactly what will be revealed, or what new game will catch the collective eye of the thousands of journalists assembled in Los Angeles. Remember, it's pretty easy for a weak buzz to become a deafening roar of news and excitement once the show actually starts.

Chartz-ing out the Wrong Course
Business journalism is based on having good, fast, reliable data available to analyze and work into stories. So it's easy to see why some in the game press fell in love with VGChartz, a site that organizes historical and current game and hardware sales data by week and region. The free site provides some rather robust graphing and comparison features, and even earned a glowing write-up from O'Reilly's popular Radar blog. Is VGChartz the answer to every game journalist's data-based dreams?

Not so fast. Simon Carless over at Game Set Watch has put together a pretty thorough expose of some of the shoddy practices behind the creation of the VGChartz "data." Carless' piece details a variety of statistical sins, but the most egregious is probably the way the site adjusts sales figures after the fact to line them up with more reliable (and fact-based) estimates from the likes of the NPD Group. In light of this revelation, it would be pretty irresponsible for a game journalist to treat VGChartz data as anything more than an educated guess. It's a shame, because a site with VGChartz' presentation and accurate data would be a must-visit for any journalist.

This Month's Must-Read: GameRankings Derangement Syndrome
Anyone interested in the interplay between reviewer and developer should check out this great piece from the Dallas Morning News on how publishers and developers pay way too much attention to game-review scores on aggregators such as GameRankings and Metacritic. The best quote comes from Matthew over at Magical Wasteland, who compares a nervous developer trying to please feature-hungry critics to "a food company performing a taste test to find out that people basically like the saltiest, greasiest variation of anything and adjusting its product lineup accordingly." It's nice to know that critics have the power to affect the course of game design, but I have to wonder sometime if we have too much power in constraining designers to the path of least resistance.

Quote of the Moment
"That looks like one of those new-fangled toys the kids play with, the why-eee"--Anderson Cooper mispronounces the name of Nintendo's nearly two-year-old system on CNN.

For more about PressSpotting, check out the introductory column.

Kyle Orland is a freelance journalist specializing in video games and based out of Laurel, MD. He writes for a variety of outlets, as detailed on his personal site, and he's also the co-author of The Videogame Style Guide and Reference Manual. Orland's views do not necessarily reflect those of GameSpot.

Questions? Comments? Story ideas? Bitter invective? Send it to Kyle.

Written By

Discussion

75 comments
gamesguru123
gamesguru123

There is nothing credible about vgcrapz. It's the least credible site out there. All of you vgcrapz mods keep those positive posts pouring in but they won't do any good at discrediting Gamespot or Kyle Orland's article. We know about the email from Brett Walton that declares he doesn't work with retailers and he uses sources like Amazon and Wal-Mart to compile his lists. The owner of vgcrapz actually confirmed that he's a liar? That's ALL we need to know fellas.

idontexist
idontexist

@miltown Yes, Anderson Cooper is one of the better journalists out there. So we should give him some slack when he mispronounces the Weeeeeeiiiiiiiiiiiyyyyy He should do a "Keeping them honest item" on the NDA agreements between publishers and journalists. But that's probably not gonna happen. :P

strategyking92
strategyking92

This is sad, Gamespot needs to focus on their reviews instead of trying to discredit a great, credible site like vgchartz.. Shame on you gamespot.

Pastehunter
Pastehunter

lol ... Gamespot is taking a cheap shot at VGC ... common guys be serious . At least try and get back on track after last years incident and only try and poke other sites ... Im a long time GS and VGC user , and I have to say that both sites are great , but neither is perfect , so please stop analyzing each other .

ps3thabest
ps3thabest

[This message was deleted at the request of a moderator or administrator]

Itsthetruth
Itsthetruth

1- Publishers trying to control reviewers? Is that "News"? I've heard of that years ago in the N64, PSone, Saturn Generation and even then was problably already happening before. 2- Battle of associations... meh, im sorry but they're just some guys who dont have a job and found a way to suck money from the gaming industry. They're outdated. "oh but someone needs to plan E3?" - some will say - There's other expos around the world working in different and less expensive ways.. They just aren't so famous because they don't represent a market as big as the US. 3- VGChartz - It's a free site with statistics on sales numbers. Some of them may not be fully accurate, so what it's statistics! It just serves as a guide, do journalists really need the exact number of copies a game sells? %#@! even wikipedia is wrong sometimes

Dr_Corndog
Dr_Corndog

Rats. I wanted to think I could trust VGChartz' data.

Gen-Gawl
Gen-Gawl

It's actually funny. I had an argument with some fanboy that used VGchartz to confirm that his system was really on top. I did a few charts and graphs, got some numbers then compared it to the NPD. VGchartz numbers were off by a LOT. I brought the info back to show him and he argued that it was a problem with how the NPD reports it's data. What most people are doing is using VGchartz to track how well their chosen console is doing. My point is, if they are consistently off by 50,000 units + per month, how in the world can they get an accurate lifetime sales figure for fanboys to argue over.

Kippergames
Kippergames

On a hunch, I went to the vgchartz site and sure enough, there was a forum post complaining about this story. No surprises there but some friendly words of advice are in order. Why do you think that the site you trolls so adamantly defend has lost so much credibility over the past several months with both print and online media? Surely it's not because of "competition" as you put it because Gamespot and other sites calling Walton's bluff have far more visitors than your site could ever hope to muster. It's because the media and industry alike have been doing their jobs and researching vgchartz, only to come to the same conclusion: vgchartz = garbage. An individual like O'Reilly who wrote that surprisingly positive story love the site because it's free. Financial institutions are notoriously cheap and notoriously shadey. Their behaviour matches that of vgchartz owner, Walton. They don't care where the data comes from as long as it's free, nor do they care about relative accuracy as long as it can be trended. This is fine for Walton's niche but the simple, inconvenient truth is that Walton is a rogue. He freely manipulates facts to satisfy his own ego, going so far as to falsify his methodoogy that he proudly advertises on his homepage, while taking information and insight from various other sources and refusing to cite and properly credit them, etcetera. Regardless of what Walton will have you believe, neither reporters nor video games industry professionals take this type of activity lightly. vgchartz is essentially nothing more than a black market dumping ground for misfits with no regard for facts. Who cares where this information comes from as long as percentage of it is somewhat accurate and you have someplace to hang your hat? Who or what loses in this fiasco? The truth, that's what.

Irve
Irve

~sigh~ all these people leaping to the defence of VGChartz ... the problem with them adjusting data is that it shows that the current data they use is WAY off. And in regions where data isn't issued , like europe .. their figures never get adjusted ... i'll still look at the site but i know to take it's figures with a pinch of salt !!

Krunkcity3000
Krunkcity3000

So, if Konami was strong-arming reviewers to get good scores for MGS4 ( which is still a great game even with the loading and cutscense) What was the excuse for everyone and their mom givinng GTA4 10/10s? GTA4 was quite lackluster and actually it was a step back from San Andreas. At least MGS4 did everything ,literally, bigger and better(except the uninspiring boss fights). BLow the whistle on the GTA conspiracy.

Nawras
Nawras

And that's the reason it got perfect scores all over!

ShadoweaverCode
ShadoweaverCode

"reporting that Konami was prohibiting those with early review access to Metal Gear Solid 4 from discussing the game's install times and cutscene length in their reviews" Wow, no wonder the reviews were so good. Restricting the early reviewers from saying things about the pretty significant downfall to MGS4. They might as well not even review it at all, and IMO that just ruined the credibility of most of the reviews for the game. What a down-handed approach. BOOOOO on Konami and Kojima!! Huge respect to Gamespot for putting out that info.

Zenfoldor
Zenfoldor

Because VGChartz adjusts its data, it is the only accurate representation of current hardware figures short of shipped data once a year from the big 3. So, as a journalist, I would use VGChartz, as it is never more than a month above and or behind actual figures for most games. It appears that some have a moral issue with VGChartz "stealing" NPD data. Why should we care though? VGChartz simply provides a service for free, while other companies are trying to charge for it. Journalists citing it, I have no problem with it, however for hardware accuracy worldwide, there is no place nearly as good as VGChartz, and even your VGChartz hate article doesn't contend that the hardware numbers are off, which is the only thing that matters, now isn't it? Whining and complaining over a free service that basically redefined fans knowledge of the big 3 hardware week by week, is being a tad ungreatful, imho.

pencilpusher69
pencilpusher69

Oh, and LOL at it being a "sin" that VGChartz should correct themselves. Being that they're in the biz of having stats and numbers that will be available for years to come, it'd be a SIN if they DIDN'T correct themselves. When was the last time GS posted a retraction about incorrect sales guestimates from Michael Pachter??? Oh, right...

pencilpusher69
pencilpusher69

Oh, and lastly, I find it HILARIOUS, that this site is saying "Journalists" shouldn't treat a site that puts out ESTIMATES before the NPD releases there numbers, when SAID SITE CONSTANTLY POSTS ESTIMATES FROM MICHAEL PACHTER AND OTHERS, WHO ARE NEVER ANYWHERE NEAR RIGHT. At least VGChartz is damn close. And hooray for hipocrisy!

pencilpusher69
pencilpusher69

"Chartz-ing out the Wrong Course Business journalism is based on having good, fast, reliable data available to analyze and work into stories. So it's easy to see why some in the game press fell in love with VGChartz, a site that organizes historical and current game and hardware sales data by week and region. The free site provides some rather robust graphing and comparison features, and even earned a glowing write-up from O'Reilly's popular Radar blog. Is VGChartz the answer to every game journalist's data-based dreams? Not so fast. Simon Carless over at Game Set Watch has put together a pretty thorough expose of some of the shoddy practices behind the creation of the VGChartz "data." Carless' piece details a variety of statistical sins, but the most egregious is probably the way the site adjusts sales figures after the fact to line them up with more reliable (and fact-based) estimates from the likes of the NPD Group. In light of this revelation, it would be pretty irresponsible for a game journalist to treat VGChartz data as anything more than an educated guess. It's a shame, because a site with VGChartz' presentation and accurate data would be a must-visit for any journalist." As somebody who frequents that site, I have to call b.s. on this claim. Do they make adjustments to numbers that are off? Yes. Weekly. And they make it known. Are their numbers off from the NPD? HARDLY. Seems like some sties are tired of hearing about how well other sites are doing. No?

necronaux
necronaux

So, the studios are holding the reviewers hostage? We'll only let you play / review the game if you don't say anything bad about it. All these reviews will be nothing but glorified commercials. It will be like late-night TV hosts interviewing actors with new movies out. They fawn over how great the actor is, how great the movie was, when it is / was a piece of garbage. Is this what we have to look forward to? Cheers to Gamespot for allowing users to rate the games and post our own comments. We're the real judges of how god these games are.

ElZilcho
ElZilcho

the why-eee?? dont these people use reading cards and monotrons for this very reason? how could anyone let him mispronounce the name of the most popular videogame console out there? lol still, it was a little funny.

pourya_ahmadian
pourya_ahmadian

you talk that they were in prison;they make game man,nor bomb;

viewtifuljon111
viewtifuljon111

High_flyer, it did not violate the first amendment, because it was an agreement, albeit one that was made at the last minute. In exchange for letting IGN review the game early, Konami made them promise not to say anything about the cutscene length and installs.

jknight5422
jknight5422

[This message was deleted at the request of a moderator or administrator]

acsam12304
acsam12304

@-HCMF- Kojima Productions spend 3 years making MGS4 even working over time and on there days off 7 days a week for 3 years a lot of them not seeing there families most of the 3 years, just making MGS4. They didnt rush just to make a quick buxk. get your facts right next time, before saying anything.

jdhill1982
jdhill1982

It's funny how Anderson Cooper kept walking away and acting like he would be stigmatized if he were associated with a video game.

mattcake
mattcake

If only they put as much effort into releasing the games without technical problems as they did in trying to hide them from the public, maybe there wouldnt be a need for such underhand tactics in the first place :)

pourya_ahmadian
pourya_ahmadian

i think so;i like this site so more;a very good place to be with your friends(GREAT XBOX360 owners)but i do not understand why they delete my reviewe;i gave the mgs4 8.8and write my own oponion about it;but then loss of poit and delete;why they do it for GREAT HALO3? BUT ALWAYS IT IS SO GOOD(GAMESPOT)

puablo
puablo

I think it's amusing that this didn't cover the Alone In The Dark controversy and instead covered MGS4, which, even if you don't agree with the 10.0 score, is still spectacular. Those are silly restrictions though - like we didn't know Metal Gear has long cutscenes? That's not shocking, let alone news. The fact that you install the game between each act was startling but it doesn't take much time and it's done at good dividing points, so it's alright, but that's rather lame that Konami wanted to hide that info.

high_flier429
high_flier429

I am not sure what may happen outside of the United States in regards to this. But wouldn't this type of stuff violate the 1st amendment of the US constitution? I know that my country has been founded on the whole "Freedom of speech and press" idea. And it seems to be working out pretty good for all who share this land with me.

-HCMF-
-HCMF-

This is just sad. I know the company wants to protect it's product, but the best way to ensure that the product gets a great review, is well, to release a great game - not a rushed for the share holders so we can stack up against 3rd quarter of last years game. Flooding the market with crap is bad for everyone, when will they ever learn. This article does one thing right, and that is it shows just how absurd the indrustry has become.

NRIAgent
NRIAgent

You know, it is really shady to make an NDA that withholds certain pertinent details effectively asking a reviewer to lie. Course I think it would hold some more weight if it wasn't something like MGS. Sure we all love it so we're a little disappointed but the truth is that despite all this it still managed to get incredible marks all around and everyone is going to play it anyway. Besides, I thought that installations were somewhat common on a PS3 (don't have one but dieing to play MGS4).

EnigManic
EnigManic

I agree with Never-Named. Gamespot has posted numerous shady reviews and shown a lot of bias. But let's face it, they have the best-built gaming site, so I'll continue using it until something better comes along.

DrCLos
DrCLos

Lucky for Konami and Kojima, MGS4 ended up being so good anyway. Otherwise this whole review fiasco could have tarnished their reputations even further. It's still tarnished though cause they did try to manipulate reviews and basically hurt the gaming mags and websites that didn't comply.

gamesguru123
gamesguru123

I like this article. It was enough to get me to finally sign up on Gamespot. Nice work. Some thoughts on the article > ESA versus ECA The ESA and ECA rivalry isn't anything new but I think you play down the ECA and GamePolitics.com relationship a little too much. ECA had an obvious hand in this and you are only fooling yourself if you think otherwise. vgchartz The vgchartz write up is fantastic. I missed last week's controversial articles but did enjoy reading them. Thanks again. I used to use vgchartz but had way too many issues with it. I read all of the reader posts on O'Reilly and on both of the sites that ran Carless' article and thought they were some of the best I've come across in months. There was one post that really grabbed my attention. It was a post that was supposedly sent to NPD by the vgchartz founder in an obvious fit of rage and tells of how vgchartz's numbers are essentially "guesstimates", generated by using Amazon charts, public bestsellers lists online and at stores like Walmart, Target, Gamecrazy etc. He also goes on to say that "At a push" he would agree vgchartz is a "little complimentary" to themselves when he says he collects data from retailers - in reality they use public chart positions / press releases and their own estimates to come up with figures. This threw me for a loop because I was led to believe vgchartz had relationships that in reality it never did. Now I know why my reports were always conflicting.

humblumi
humblumi

Sorry... the truth is... you can't trust any website or magazine... they're all biased for financial reasons. Buy the games you like and conclude yourself... impossible to believe anyone. reading this article made me laugh (since it came from gamespot)

slim-jim69
slim-jim69

Never named should look up the meaning of hypocritical. Fair dues to gamespot for always reviewing the finished game that will be available in stores.

hungfar
hungfar

@never-named If this site has a "tarnished reputation" then why are you using it? Seems a bit hypocrtical AIN'T IT. Awesome grammar as well dude. Well done.

never-named
never-named

Looks like GameSpot is trying to redeem its tarnished reputation by posting articles about the various "injustices" of game publishers. Seems a bit hypocritical ain't it?

SonicRaptor
SonicRaptor

I seriously hope every time a publisher tries a dirty tactic to control what a reviewer says get caught wide out in the public arena. First Sony bribing Lair reviewer with swag then Konami telling reviewers not to criticize parts of MGS4 that people may not like? I pray that every publication calls BS on things like that. Except GameSpot, they'll probably play along as long as the price is right.

BewilderedRonin
BewilderedRonin

What's with the tl;dr posts? If the topic's not something you care for, don't read it. Personally, I enjoy these "behind the scene" wrap ups of how the video game journalism industry responds to these stories. @ asimplerapper What about Gerstmann? He's still setting up Giant Bomb. Besides, Gerstmann was quoted last month. You fail, narutard.

combatsoldier
combatsoldier

hhhmmm. very interesting. thankyou for sharing. good read.

BewilderedRonin
BewilderedRonin

[This message was deleted at the request of the original poster]

NeoNavarro
NeoNavarro

Damn I feel asleep at my computer reading all that... :Yawn:

Gen-Gawl
Gen-Gawl

Good read on VGchartz. I've been saying that for a while now. Everytime some fanboy uses Vgchartz as proof that his system is best it makes me crazy.

felshs
felshs

i read 2/5 of this ........man...too much info. I think the Publisher have the right to ban some unnecessary information that might Spoil the game. Cuz most of the time, when i read the review, i felt like i know everything about the game

taj7575
taj7575

oh and LMAO on the MGS4 review thing. i bet that IGN reviewer was reviewing while kojima was sitting next to him with a revolver in his hand whispering to the reviewer "dont mess up...or else..." haha.

taj7575
taj7575

what the anderson cooper vid on youtube lol. it is pretty funny. besides when a guy spends half his time in reporting in war zones, he shouldnt be worried about the....the...why-ee?

Erebus
Erebus

Informative article.

VirtuaCast
VirtuaCast

gnbfd I can't think of any other Pub that put restrictions on reviews not to tell gamers the install size of a game. Sounds like Sony is running into problems with the PS3 HD filling up.