EEDAR dishes data on achievements, new IP

GDC 2010: Researchers find only four percent of players max out gamerscore for any given game, predict a surge in original intellectual properties sparked by Project Natal, PlayStation Move.

by

Who was there: Electronic Entertainment Design and Research was represented by president Geoffrey Zatkin (a member of the original EverQuest design team) and VP of analyst services Jesse Divnich, who spoke about gaming trends in intellectual property, as well as what they've learned from sifting through Xbox Live achievement data.

What they talked about: In one Thursday presentation at the Game Developers Conference, former EA/Maxis developer Chris Hecker grappled with reward systems like Xbox Live achievements and whether or not they might ultimately take the fun out of playing games. EEDAR's Zatkin and Divnich have been likewise preoccupied with achievements, but in a very different way.

EEDAR's Jesse Divnich.

But before tackling the achievement issue, Zatkin dove into new intellectual property trends in games. EEDAR data shows that the percentage of games establishing an original IP has actually been on the rise since 2006, up from 16 percent to 22 percent. However, that growth is due mostly to the Wii, which in 2009 had 27 percent of its releases count as new IPs, Zatkin said. For the Xbox 360 and PlayStation 3, the number in 2009 was only 17 percent.

Zatkin attributed that imbalance to the "game-changing technology" of the Wii's motion-sensing controller. He also told the audience to expect a surge in new IPs for the Xbox 360 and PS3 once Project Natal and the Move controller come out.

New IPs were particularly prevalent in the role-playing game, action, shooter, and general entertainment genres, Zatkin said. Since 2005, about 38 percent of role-playing games have been based on new IP, while only 8 percent of fighting games in that span were based on new IP.

Zatkin also broke down the disparities of new IP and old IP based on Mature titles. On the Xbox 360 and PS3, 47 percent of new IP were rated M for Mature. For the Wii, only 2 percent of new IP received an M rating.

Zatkin then showed the top five publishers of new IP on each platform. On the PS3, it was Sony, EA, Codemasters, Sega, and Ubisoft. On the Xbox 360, the top five were EA, Microsoft, SouthPeak, Codemasters, and Sega. For the Wii, it was Destineer, Ubisoft, Zoo Games, Konami, and Activision. Zatkin said Destineer had published more than 50 new IPs for the system on its own.

Then Zatkin turned to review scores and pointed out that new IP review scores were trending a little bit lower on all systems. There were some exceptions, as new IPs in action games were as successful or more with reviewers, while new IP in the sports and strategy genres did considerably worse than their established counterparts.

Divnich then took over the talk to present EEDAR's findings on achievement data. Working with mygamercard.net and Microsoft, Divnich said he was able to mine about 32 million data points on Xbox Live achievements relating to a random sample of 100 different Xbox 360 games.

EEDAR broke down achievements into 16 different categories, like completion (beat a level) or exploration (find the skulls in Halo 3). The group found that only 4 percent of gamers on any given game were getting all of the achievements on their games. With AAA titles, that number drops to 2 percent. On average, only 27 percent of players managed to get half of the available achievements in each game. Divnich suggested that with numbers that low, most games' achievements are probably a little too hard.

Are you motivated yet?

Divnich posed the question of whether or not achievements actually motivate gamers to play the game. He said they did, but only at the very beginning and the very end of gameplay. He likened it to a runner trying to finish a marathon. Even if runners get off to a bad start and know right away they aren't going to finish the race, they're unlikely to quit so soon after starting. And when they're in the home stretch (80 percent or more of the achievements completed), seeing the finish line motivates them to tough it out, no matter how much of a struggle it is.

Achievements aren't just a motivator, Divnich said. They're also a monitor, as demonstrated by a Microsoft presentation earlier this week. Zatkin emphasized that achievements could be used as a feedback mechanism. For example, if there are achievements given for exploration, combat, and driving in a certain game, the developers can use the data to infer what players enjoyed doing most and allow that to guide their decisions on a sequel.

When asked specifically about Chris Hecker's talk from the previous day, Zatkin suggested achievements were unlikely to be a de-motivator in most cases, with one caveat. He said that role-playing gamers had been trained for years that once they'd finished the story, maxed out their levels, and filled every progress bar they could, the game was over. So if a game had achievements that were too easy to obtain, he acknowledged that could signal to players that the game was effectively done and serve as a de-motivator.

Quote: "Achievements are a reward. People use reward mechanisms in games to get players to do what you want them to do. If there were no rewards for collecting coins in Super Mario, you wouldn't do it. Achievements can be a very powerful tool that has a very low impact on a development budget."--Zatkin, offering advice to game makers.

Takeaway: Sales aren't the only data for publishers and developers to consider when it's time to make their next game. They can get a different perspective on key decisions by mining data to unearth trends across a wide spectrum of categories, including achievements and intellectual property trends.

Discussion

32 comments
Giorgio879
Giorgio879

I like achievements, a nice reward for completing a challenge, and expanding them to have rewards outside of the gamerscore is a good thing, like avatar awards and such. That being said I dislike multiplayer achievements, I prefer them to stay on single player or co-op.

Nintendo_Man
Nintendo_Man moderator

Another reason for low completion % is that many games now have 50 achievements per a game which is just far too much if you play plenty of games, most people don't have time to get all 50 of them in so many games. I guarantee that games with 40 or even 30 achievements will have better completion %.

Polybren
Polybren

@nedrith I'm curious. What do trophies do better than achievements? Is the lack of an overall gamerscore really an improvement? I think of them basically identical systems except for that and the platinum trophy. But you can check the game-specific achievements of anyone to see if they would have earned a platinum.

Blaketoberfest
Blaketoberfest

I don't own an X360 or a PS3 (but played a few iPod touch games with achievements featured), but it is agreeable that most of these achievements involving a bunch of repetitive tasks like "beat this guy fifty times" is tiring and hardly involves any skill whatsoever. I admit I am a completist gamer, but there is no point in giving me tasks that is as tiring as doing house chores. I play games to have fun, not to live a boring life just trying to complete a certain task just for the sake of completing it. Also, that use of the "YOU'RE WINNER !" image is hysterical. xD

mrzero1982pt2
mrzero1982pt2

achievements/trophies are good for games. think about it. there are some folks on xbl who have sf4 and have tons of achievements i haven't gotten. maybe i never will. i like how they make the games fun when you compare games with someone. look at something like halo 3.. there are a few people i know who have EVERY achievement. and i like how microsoft(i dont know if psn) does it, but downloadable games top off at 200 points, retail games have 1000 and through updates go up. so i say keep achievements and trophies. better in my book.

nedrith
nedrith

I think Achievements aren't bad(trophies are better though) and I think that most atleast in the RPG category aren't too hard. Hopefully it will motivate some more people to try to destroy the final optional bosses in games. But many achievements just aren't achievements at all. and many of the true achievements are given very, very low gamerscore. the platinum trophy a 360/ps3 multi-plat game is worth 0 gamerscore in many games. You get absolutely nothing for completing all achievements. I also find that many times the hardest bosses on the hardest difficulty either gives super low gamerscore or none at all. many of the true achievements have no achievement at all. There are many flaws with achievements. luckily many were fixed in achievements 2.0 aka PSN trophies but even then many are the failure of game companies to properly balance achievements and what they give it for. Completing the first level or using a easy to hit button doesn't usually warrant a achievement, even more so it doesn't warrant the same amount of score as beating the second level or the third or the fourth

Poodger
Poodger

I think the reason gamerscores are not maxed out for any particular game are because there are often a couple of VERY hard achievements to get. Which is perfectly fine, since they are called "achievements" for a reason. I hate the achievements which are gained for simply playing the game through (Unlocked: Beat Chapter 1). I also think there needs to be more in-game unlockables for gaining achievements.

Gamesmasterx
Gamesmasterx

I like it, though it's pretty nasty how they release DLC, and then the next time you play that game the 100% completion is taken away, I mean, it should at least check to see if you have the DLC, I had 7 on my list, now it's only 3, ;( That and achievements involving playing MP or co-op, I hate them so much.

KeArNiCe16
KeArNiCe16

achievements arent taking the fun out of nothing there allowing the game to be played for longer i love grabbing achievements

abbnormmal
abbnormmal

"New IPs were mostly on the Wii" Most of those "New IPs" were games like pretty pretty ballerina, various cooking type games, and heaps of other shovelware. 3 cheers for the wii expanding the wonderful world of gaming.

Marciano315
Marciano315

70% of the games i buy are because of achievements. i wouldnt even have an xbox 360 if there were no achievements.

Pumpkinhead3
Pumpkinhead3

the achievement system is okay. its fun to get them on a second playthrough for some replay value, or when you are required to do something fun and challenging ("Gaurdin Gnome"-L4D2), but i cant stand achievements that feel like a chore or are simply thrown at you without doing anything (AC2)

Katmando4Life
Katmando4Life

I like getting achievements, but I think they need to have some sort of reward beyond a number. "Halo 3: ODST" and "Left For Dead 2" have avatar gear tied to specific achievements, and "Assassin's Creed II" has uPlay rewards tied to certain achievements. While avatar gear is certainly better than nothing, Ubisoft has the right idea. Give us small rewards that can be utilized in the game and I think the achievement system would be a much larger focus for gamers.

ISuPrEmAcY32I
ISuPrEmAcY32I

i love to collect achievements. they definitely motivate me to come back to a game after I've already finished it. It's actually quite interesting, for the sake of achievements I've even played games contrary to how I think I should (for example, Dragon Age, where I normally would make one decision, but since I already have that achievement I make a decision that is completely contrary to how i would otherwise play). Achievements that are too difficult or just plain obnoxious to get should be done away with. For example, Mass effect 1's achievements of playing through the majority of the game with a certain character in your party the entire time. I want to swap around my team depending on what situation we're in and so that i can experience all the characters. don't pigeon hole me into playing with 1 guy the entire game so i can get that achievement. Plus for subsequent play through you then feel obligated to play with a different character which you may not even enjoy to get the separate achievements. On a final note, I'm not a huge fan of online achievements either, which force you to play game modes you're not good at or don't even enjoy to try and round out your score. Anyways, love achievements in general and I'm proud to say that i get 80 percent or more on every game I own, with 100% on several of my favorites.

-HCMF-
-HCMF-

good info and a new way to look at those darn achievements...

Trogeton
Trogeton

In every game I play I try to get at least 80% of the achievements.. and to be honest if achievements would be taken away all the sudden.. I would not play as many games

Wings_008
Wings_008

tbh in most games i play, i get only or just over half of the total GS

Carpe_Noctum
Carpe_Noctum

Achievements can be a cool way of getting you to do something you might not normally do and it turns out to be something you enjoyed but otherwise wouldn't have done. I am of the thought, much like Infinity Ward, that Achievements should stay on the single-player side, or have single-player/Co-Op but having Achievements for multiplayer adversarial is just a bad idea.

tyco_ex
tyco_ex

Lol, look at the picture of that dude. Such a fruitcake.

cpup
cpup

Achievements are ok but ultimately unrewarding. Getting 100 coins with Mario rewards you with a 1up. Achievements need to give me something in hand that I can use. I don't care about my gamer score because it doesn't really add to my experience besides bragging rights. As for multiplayer, achievments do change the dynamic of the game or match. 4 vs 4 quickly becomes 4 vs 3 if one person is trying for a specific, self serving goal rather than the goal that the rest of his/her team is trying to reach. Give me an online gun for doing something in the offline campaign and I'll play the campaign until I get the gun. Then I will mow you down with it in multiplayer. You will then want to get that gun to mow me down and go back to the campaign to get it. That's rewarding. That's an acievement.

DAMSOG
DAMSOG

Some of them are just wrong, host a game with 16 players? then the server tells you you can only host a max of 8 players....exactly how is this reflective of any gameplay? I must admit I have played crappy games just for the easy GS points....the shame

Knutte79
Knutte79

[This message was deleted at the request of a moderator or administrator]

Lionoversusking
Lionoversusking

It comes down to what game you are playing though some achievements rule but most of them have become a staple kill 100 enemies etc Matt Hazard was the worst for it but I just got 1000g on dead space and that was fun with the achievements working in a good range of challenges and staple achievements. Also some are great way to pick up the game again for a little challenge.

Vegeta-sama
Vegeta-sama

@JCDenton you need that Level Beaten achievment as a thumbs up motivator... you're probably pretty old school(original Deus Ex fan) but they need some kind of balance... you shouldn't have more than 1/4 of your GS per game be held to online... though it would be cool if they seperated them for the next system

JCDenton
JCDenton

I honestly don't even look at the achievements. I did it once just to see what kind of stuff was on there. To me, achievements are fine, but they should enhance the gameplay experience and be fun, and not just be tacked on. They should not be too easy, and they should not require excessive repetition (oh wow you killed 1000 zombies!!). Beating all of the levels' times in GoldenEye 64 was fun, because it challenged me to think many times over how I could shave off time (perhaps including not killing certain enemies). I really dislike achievements that are gotten for silly things like playing the game for 5 minutes, or even beating a level.

jeffv541
jeffv541

[This message was deleted at the request of a moderator or administrator]

DeltaSpirit
DeltaSpirit

I like getting 1000 points or a platinum trophy but I think most would agree that some achievements aren't worth getting/ it's too much work for a little reward or going online to get an achievement (instead of having fun) not so much that their too difficult. Remember GRAWs Get a total of 10,000 kills in multiplayer 24G, Play for 8 hours straight in multiplayer 28G?

drknessfeeds
drknessfeeds

I enjoy achievements/trophies too, I just dont like it when they are something ridiculous. Also I dont think that achievements chould be given in any type of multiplayer part of the game, it just kinda takes the fun out of getting them considering most games have just a tacked on multiplayer when there are so many other games with great multiplayer. COD MW does not have achievements for multiplayer, look at how many users are on their server constantly.

Tsuchikage
Tsuchikage

Firstly, I love the picture of the trophy screen from Big Rigs on PC. Nice touch. Secondly, I hate it when Achievements are tied into doing odd/repetitive things in a game's multiplayer (like getting multiple simultaneous kills with the Spartan Laser in Halo 3). These achievements tend to make players play differently than they normally would, throwing off the game's balance. This is more acceptable in single player, when it's only one gamer's self-imposed consequences to doing strange things in a game, but when it affects other players, it's a problem.

YEPEE00
YEPEE00

i think they ought to give an achievement if you buy the game rather then rent it.

XenoLair
XenoLair

All true! Achievements at least for me give every game a bigger replay value (if done properly). Fable II had some excellent achievements and so did MW2, where when I beat the game on normal I got achievements for beating the levels and the game + other "mini" achievements and when beating it on Veteran I got achievements for beating the levels on veteran and for beating the game on veteran. They do light up my day!

monson21502
monson21502

i like achievments. but i dont search for a list of each game i play just to try to unlock em. i just like to play normal and when one pops up. i like that feeling.