The Science of Playtesting

We go behind the scenes at Valve, Bungie, and Epic Games to see how user feedback helps shape the game development process.

by

Inside a dark building high above Seattle, a group of gamers wait patiently to be strapped to a voltage battery. Thin wires peek out from under their sleeves as tiny round contacts are stuck carefully to each of their hands. Somewhere down the hall, a PC registers the first electrical pulse, rippling across the skin of one of the candidates. Whoever he is, his nerves have just given him away.

The playtesting rooms inside Valve's headquarters resemble a large, untidy science lab. PC monitors line the wall, spewing out steady streams of graphs, patterns, and charts. Some keep track of candidates' heart rates; others record their eye movements. A pile of resistors and voltage batteries lie discarded in one corner of the room; candidate information reports, video recordings, audio transcripts, written questionnaires, and the results of thousands of hours of direct observation lie in the other. Valve takes its playtesting seriously.

While playing video games for a living may sound like the dream ticket for any avid gamer, for those involved in the process, it represents much more than just another routine part of game development. Testing games before they are released gives game developers a rare insight into the end user experience, helping them validate the quality of the game and isolate potential problems. We now see games as much more than simple products; we see subjective experiences that affect individuals in different ways. For this reason, video game testing not only has to be more rigorous and pervasive than product testing in other industries, but also more precise. Can feelings be measured? Why does one player enjoy a particular game, while another does not? How can user feedback be used to make games better?

Publishers and developers are constantly seeking answers to these questions, trialing a variety of playtesting methods to get the best results. In this feature, GameSpot will go behind the scenes of three major studios--Valve, Bungie, and Epic Games--to find out how tools like science and psychology are helping game developers better understand the nature of player behavior.

Playtesting room or science lab?

A Healthy Dose of Perspective

Three years ago, Valve hired experimental psychologist Mike Ambinder to head up the playtesting department of its development arm. With a B.A. in computer science and psychology from Yale University and a Ph.D. in psychology from the University of Illinois, Ambinder was the perfect man to lead Valve in a new direction in the field of user research. Using his background in visual recognition, Ambinder began treating Valve’s playtesting sessions as a series of scientific experiments designed to test various game design hypotheses. Since most psychological research revolves around trying to isolate mechanisms of behavior to find out why people are motivated to do the things they do, Ambinder thought game playtesting should strive to do much the same thing: use extractable data to make an altogether better user experience.

"For us, playtesting is the most important part of the game development process," Ambinder says. "It's not something we save for the end of the development, or use as a quality assessment (QA) or balancing tool. Instead, it is the dominant factor that shapes our decisions about what to release and when to release it."

This is a relatively new attitude for the games industry. While playtesting has always been a vital part of the game development process, the role it plays in shaping the final user experience has never been as important as it is now. Traditionally, playtesting methodologies focused on video games as products rather than as variable experiences that can affect players in different ways. Even as video games became more complex and publishers began to employ dedicated quality assessment teams rather than simply having the development teams playtest their own games, user feedback practices remained fixed on highlighting problems surrounding the more objective aspects of game design--coding errors and bugs--rather than exploring the experience of what it's like to actually play the game. Methodologies like functionality testing (looking for general problems in the game's overall design), compliance testing (checking that the game complies to publishers' technical and legal requirements), compatibility testing (testing the game on various configurations of hardware and software), localization testing, public beta testing (which lets users pick up any errors the developers may have missed), and regression testing (testing to make sure previously reported bugs have been eliminated) have become industry standards, with publishers and developers mixing and matching various methods to suit individual project needs.

But things are changing. The growth of gaming audiences and the subsequent push towards more diverse gaming experiences has led some publishers to rethink traditional playtesting methods in favor of something a little more relevant. According to Ambinder, the games industry is starting to move towards more innovative ways of gathering data, willing to spend more time, energy, and resources on its accumulation.

"I think more and more companies are starting to see the value in hiring folks with backgrounds in psychology or related fields that provide skilled training in extracting meaningful data from playtesters," Ambinder says. "For us, playtesting is crucial, as it is the most effective and honest means of validating our products. We would be foolish to release a game that went through minimal playtesting, as we could have little confidence in its quality had it not gone through rigorous testing. To that end, we start playtesting as soon as we have something playable, and we basically never stop as we are constantly updating our products after shipping."

When developers first became interested in user research some 10 or so years ago, the standard practice was not to waste too much time on it; therefore, only the first hour of the game would undergo outside playtesting. Things are a little more serious these days. Most publishers would consider it madness to release a game that hasn’t been subjected to hundreds of hours of rigorous playtesting, combing over every single part of the game right up to its release. A decade's worth of knowledge has come down to one thing: perspective.

After spending years as a playtester himself, Epic Games senior game test manager Prince Arrington saw the value of perspective in the work of those around him.

"I always got a kick out of giving feedback to developers and then seeing my suggestions come to life in-game. But sadly, it has the tendency to be one of those things that can easily be undervalued. It's very easy for us, as developers, to get too close to our projects and fall into the trap of not realizing that our baby isn't perfect. This often leads to poor design remaining poor. The value in having outside feedback is that it's always nice, if you're open to constructive criticism, to get those checks throughout the development cycle so that you can get the confirmation that you're making something kick-ass…or otherwise, the painful realization that you're not."

Bungie playtested its Halo games with people who had never even seen a shooter.

Just like Ambinder, Arrington's professional career is rooted in psychology. Earning his psychology degree from North Carolina State University, Arrington left academia to become a contract tester when his father told him that he played games so much that he should probably start making them. During his three years as Epic's QA manager, Arrington has come to see playtesting as a valuable tool in keeping developers in check.

"Playtests are as much a part of the development cycle as design meetings or code reviews," Arrington says. "While different in execution, it's something that should be done daily, with different combinations of participants with varying skill sets willing to give constructive criticism, with the sole purpose of making the game as good as humanly possible.

"Developers work on a project for so long, there's always the potential to lose perspective on what's working and what's not working. Without this form of genuine feedback, developers have a tendency to drink the Kool-Aid and become accustomed to the inefficiencies and flaws of a project, which leads to a failure to explore more suitable paths."

This perspective is often offered by people who have never played video games before. Both Valve and Epic aim for a wide demographic when deciding whom to bring in, from seasoned gamers, to people who have never played a shooter before, right down to non-gamers. With the gaming audience growing every day, developers have come to understand the value of reaching out to all skill levels, fighting to keep existing audiences while trying to snare new ones.

"We have a sign-up page for playtesters on Bungie, and we bring in a whole range of people: people who are experts, people who play casually, everyone," says John Hopson, the user research design lead at Bungie. (He too has a psychology degree). "For Halo testing we even bring in people who have never played a shooter before. It's painful to watch because they have a lot of trouble, especially with things that we don’t even think about anymore, like coordinating the use of two thumbsticks or knowing where the buttons are. But Halo is supposed to be a game that is fun for everyone, so it's necessary to see how people who have never played a shooter before react to it."

Arrington says if developers lean too far in one direction they risk skewed playtesting results, ones that don't take into account information from players who have the potential to account for a large portion of the game's user base.

"If you fail to account for players new to your game or genre, it's typically those players that will stop playing, or avoid your game, if the barrier to entry is too high. So this is very important for making the game easily accessible to the casual gamer as well as challenging enough for the veteran, hardcore gamer."

It's Not Rocket Science

Back inside Valve's playtesting rooms, Ambinder is busy measuring data. Surveys, question-and-answer sheets, verbal reports, direct observations, gameplay data, design experiments, and a handful of biofeedback data--Valve's latest foray into perfecting the process of playtesting--are painstakingly collected and categorized. It is this latter methodology that has made Valve a leader in the field of refining and improving the playtesting process. In the field of science, biometrics is a set of methodologies that can be used to identify individuals based on the measurement of intrinsic physical and behavioral traits; in the world of video games, the same set of methodologies can be applied to ascertain how much people are enjoying themselves while playing a game. Cue Ambinder, voltage battery in tow.

"We became interested in the use of biofeedback both as a playtesting methodology and as potential user input to gameplay because the idea of quantifying emotion or player sentiment seems to have utility," Ambinder says. "On the playtesting side, recording more objective measurements of player sentiment is always desired. People sometimes have a hard time explaining how they felt about various things, and memories of feelings and events can become conflated. Conversely, if you have a more objective measurement of arousal or engagement, you can get a clearer picture for how people are emotionally consuming your game."

Playtesting in Portal 2 helped to reveal that not all gels are good gels.

This is how it works: groups of playtesters are gathered and brought into Valve's headquarters where they are asked to sit down and play a specific game for an hour or two, depending on the title. While this is happening, a separate group of developers observe and record the behavior of the playtesters. No assistance is offered--the goal is to create as naturalistic an environment as possible. At the end of the playtesting sessions, the players may be asked to fill in surveys (so Valve can record long-term trends), before being ushered in to begin individual question-and-answer sessions (or group question-and-answer sessions for multiplayer games). From time to time, Valve brings out the big guns: eye-tracking (done remotely via a set of cameras built into the monitor while players are in the playtesting session), heart-rate monitoring, and skin conductance testing that measures the electrical pulses on the skin to determine how engaged someone is with the task he or she is performing. The point of all this is to gauge individual players' experience as they make their way through a game: Do they really like the game they're playing? Does a particular part arouse their interests more than others?

"At the moment, we're very curious about the measurement of skin conductance, a correlate of physiological arousal," Ambinder says. "By identifying the peaks and valleys in the arousal waveform, as well as the eliciting events that may cause them, one can start to gain insight into how various game mechanics and game events lead to both positive and negative changes in player state. In addition, one can attempt to predict subjective enjoyment and frustration metrics by looking at the quality of the arousal waveform produced during the playtesting session."

Valve says all the data collected in the playtesting process is useful in one form or another, be it to fix problems with the game itself or to make the game more appealing by including more of what players seem to respond to. For example, visual outlines in Left 4 Dead 2 were added after playtesting revealed that verbal communication among teammates was neither efficient nor accurate enough to convey relative location information. Several experimental paint types in Portal 2 were also done away with after playtesting groups revealed that the quality of the experience of playing the game was negatively affected by their inclusion.

Of course, not everyone in the industry is as open to biometrics as Valve is. Some publishers continue to rely on a process of trial and error to handpick playtesting methodologies best suited to the kinds of games they are producing. Take Bungie, for instance. It's hard to find a game franchise that has relied on the feedback of users as consistently as Halo. Hopson has been paying attention to how players unlock doors for seven years now.

"We watch how our playtesting groups play our game, and record them," Hopson says. "We notice every little detail, from how they pick up a weapon and use it to how they go about unlocking a door. Other times we bring in groups and have them play the game and we collect data about how hard a campaign mission is, or which map is easier to navigate, and so on. And for other questions, we go to data mining [collected on Xbox Live servers]. This gives us an idea of how players play the game, and things like what maps they are best on, and so on."

Is biometrics the best of the playtesting methodologies?

Hopson records everything: how players die during the game, where they die, and what weapons were used to kill them. From this, he moves on to individual question-and-answer sessions where he will ask players about their favourite missions in the game and their reaction to things like difficulty levels and how they solved problems during the set time period. Hopson and his team attend every development, design and tech meeting in the company, constantly looking at the best ways to apply user research. For example, it doesn't matter if only one or two people are stumped by a particular issue: if it's severe enough, Hopson highlights it as important. The aim, as with all playtesting methodologies, is to discover what players think about the game. But even applying basic psychological principles, such as paying attention to how much information a player can retain at any one time, or how much memory a player uses during the playtesting process, won't produce as accurate a result as the biometrics approach. So why isn't Hopson getting out the heart-rate monitors?

"Some publishers are going down this route, but I'm not sure biometrics is the way to go. We've had a lot of time to experience what works and what doesn't, and biometrics doesn't tend to add a lot to the techniques we're already using. You could learn this stuff by just asking people. I mean, what we want to know is whether people are having fun or not. And just asking them, or watching them play the game, can determine that. We don't need super-precise accuracy. For example, I can ask you right now to describe whether the room you're sitting in is warm or cold. And you could tell me. I don't need to stick a thermometer in the room to verify that."

The More the Merrier

In its purest form, playtesting is a simple process: have people play the game, and then ask them what they thought. The real trick is getting as many opinions as possible. By its very nature, playtesting makes this hard to do--it's not easy to consistently find people to physically bring in to the premises, watch them play the game, and then interview them, often individually, about it. While playtesting has become more sophisticated over the years, development teams still have a limited number of resources to dedicate to the game-making process; coordinating daily groups of gamers into the building and looking after them, down to the smallest details of where to sit them and what to feed them, continues to remain a logistical nightmare. While publishers are slowly finding ways to work around this (Bungie hosts regular playtesting "weekends," for example, with two straight days of user testing producing a substantial amount of data), more organic solutions are becoming increasingly popular.

Both the Halo and Gears of War franchises are known for their sizable and meticulous public beta testing (the Halo: Reach multiplayer beta saw 2.7 million players hit the servers, eclipsing the Xbox Live record for a beta previously set by Halo 3), which helps developers like Bungie and Epic gather a whole array of useful information, from bandwidth size to matchmaking system preferences in multiplayer. The popularity of these public betas also means developers are rewarded with a hefty amount of user feedback via forums, email, Twitter, Facebook, in-game chat, and so on. This is mostly where Arrington and the Epic team are treated to more subjective, what-is-it-like-to-play-this-game moments, which he says can often be more insightful than cold, hard data.

Epic uses the Gears of War public betas to gather all sorts of useful information.

"I'm kind of old school and think that there's something pure and genuine about letting players play, observing what they do and engaging with them," Arrington says. "I don’t know that ours is the most effective method out there, but it's a viable system, and it works really well here. Some swear by biometrics. Others isolate their testers in a room with a one-way mirror, record their actions, and give them an anonymous questionnaire to fill out. All methods have their benefits and drawbacks, but ultimately we have to choose what feels best for us."

It turns out public betas are a pretty useful tool: having 20 million multiplayer matches played by 1.3 million players around the world can tell developers a lot about the game they are about to release, things they might never have thought to check had it not been for a 10-year-old kid from San Paolo. For the Gears of War 3 public multiplayer beta, Arrington and his team tracked every single bullet fired (23 billion) while observing how players managed themselves in the game, from how weapons were being used to how maps were being played, exploits that were found, match durations, rule confusions, and so on. Based on the data received, Epic's development team went to work changing a large number of the matchmaking algorithms to account for the proper distribution of servers across the different regions, as well as improving gameplay aspects like opening up the spawn points to avoid spawn camping, reducing the goal score for King of the Hill to speed up matches, and tweaking the overall balance of the weapons to make them more fair during combat. But it didn't stop there--during every single day of the Gears of War 3 beta, someone from Epic jumped in the game to see how things were shaping up.

"It's one thing to see a bunch of charts and stats and to hear secondhand how the game feels, but until you're in there yourself, it's hard to really know what's really a big issue or not," Arrington says. "It just gives you a more knowledgeable perspective on how it's supposed to be so you can better compare that against how it actually is at that moment."

As much as developers can benefit from online public betas, this methodology does little to help in tracking the day-to-day changes that occur during the development cycle of a game. While it would be impossible to generate the same amount of data gathered in a public beta test in an internal playtest environment, the level of insight that developers are afforded by daily playtests cannot be achieved through something as large-scale as a public beta. In-house playtesting begins long before developers put out the call to eager gamers; it's part of a rigorous and intensive process that these days makes up a very important part of a development team's production schedule.

The Epic Games quality assessment team.

At Epic, Arrington says the goal is to make sure that everyone in the company gets multiple opportunities to give feedback on as many of the game's features as possible, including levels, AI, weapons, characters, modes, user interface, visuals, and so on. Arrington's team typically runs multiple playtests daily, usually lasting one hour per session, and tracks every developer's personal profile to monitor their progress and to check, among other things, who exactly shot them in the face.

During these playtesting sessions, Arrington has one of his team sit with the developers who are testing the game, recording comments and looking out for three basic types of data: feedback, bugs and glitches, and statistics and heat maps. The kinds of statistics tracked on the back end are things like number of downs, kills, deaths, and revives over the course of the project; heat maps offer a top-down image of the level, tracking activity on the map and providing level designers with a visual representation of high-combat areas and low-activity areas. Immediately after each playtest is over, the participants sit and discuss their experiences with the level designers while Arrington and his team take notes.

"A lot of the data collected in our playtests actually ends up being used. During the playtests, not only is my team documenting everything that occurs, but the designer of the level and the lead level designer are also there either shoulder surfing or playing with the participants. Once it's all over, they will go through all of the data that we've collected and incorporate any changes that they feel will have a positive impact on the quality of the game to the end user. Often, these changes are implemented quickly enough for the next playtest, which really allows us to iterate quickly."

Playtesting in the Future

A byproduct of games becoming more complex, mature experiences in the last decade is the exponential growth in the skill level of gaming audiences. Players have become more adaptive in their ability to respond to changes in the gaming environment, and the dominance of first-person shooters has contributed to an increase in the gaming mind: faster reflexes, better performance, and the ability to correctly judge and prepare for random in-game situations with relative ease. No one is in a better position to judge the evolution and growth of gamers than those who spend most of their time watching them play. Ambinder, Arrington, and Hopson have all noticed the same trends arising from their study of player behavior over the years: players are getting smarter, quicker, and more efficient.

"In terms of extracurricular activities, gaming has become as regular a pastime as playing outside (remember that?)," Arrington says. "As a result, players definitely have a higher game IQ than when we were playing Donkey Kong and Defender in arcades back in the day. As gamer IQs have increased, developers have increased the complexity of their games. Just compare the first Madden game to last year's version: while the game of football hasn't changed, the complexity in the execution has increased exponentially. The bottom line is that more intelligent gamers require more challenging games and vice versa."

There's also the hardware to consider: gaming consoles have become more complex. (Just compare the Atari 2600 with its joystick and one button to the Xbox 360 controller with its two thumbsticks, two shoulder buttons, two triggers, a D pad, and five face buttons). This has contributed to physical increases in the skill level of gamers, from hand-eye coordination to reflex times. The fact that most games these days use similar control schemes means gamers can quickly adapt from one genre to the next without having to relearn the entire process from the start, meaning that gaming proficiency is more widespread than it has ever been.

Gaming used to be so simple. What happened?

"We've been exposed to this familiarity in games for so long, we inherently, in whatever game we're playing, know that the blinking item is important, that there might be loot in the crate, that developers probably hid collectibles somewhere inconspicuous, and that shooting the red barrel is going to ruin someone's day," Arrington continues. "I think my son destroys me at fighting games to this day because I put a controller in his hand at a very early age. Had I known that he would grow to be that kid that crushes my self-esteem by letting dad win at Super Street Fighter IV, I would have made him take up chess."

The prevalence of online gaming has also changed the gaming mind: developers can no longer analyze games at the level of the individual player without taking into account online and social capabilities. So, for example, if Hopson and his team are looking at a group of 15 players in a multiplayer match and one of them is misbehaving, it becomes impossible to simply isolate that player to find out why he's behaving like that--the group has to be observed as a unit.

"We've certainly placed more emphasis on the fact that people have to experience the whole game," Hopson says. "Once upon a time, the game industry used to accept a failure rate that no other entertainment industry accepted--for example, everyone who walks into a cinema expects to see the whole film, not just part of it. I think we used to deliver games that were so difficult that not everyone could finish them, and there was no alternative to that. But now, we have the ability to offer a whole spectrum of difficulty levels for every kind of player."

The last few years have seen both publishers and developers branching out with playtesting methodologies, but always with the same goal in mind: more accurate data. Ambinder, Arrington, and Hopson all recognise the beginning of an exciting time in the field of playtesting, a time for taking risks and seeking even more novel ways of reaching further into the minds of players than ever before.

"The last few years have seen a broad expansion in the talent brought in to facilitate playtesting, and with the addition of these specialists, it is likely that the best breakthroughs are ahead of us," Ambinder says. "I think modeling of future gameplay behavior based off of prior behavior is an emerging field that has yet to reach maturity. It seems likely that our predictive tools will only improve in the years to come. I think we'll also get better at quantifying the subjective experience of playtesters and hopefully become more adept at isolating facilitators and inhibitors of enjoyment, engagement, fatigue, lack of challenge, and so forth. It is tough creating an objective environment for playtesting, and if we become more effective at removing sources of bias, we should be able to acquire richer and more meaningful data."

As games become more complex, so too will playtesting methods.

While Valve has clearly embraced a more scientific approach to playtesting, Arrington doesn't want the process to become too clinical.

"I'm sincerely hoping that it remains a good mix of both organic and clinical playtesting. My fear is that developers or publishers will start looking for shortcuts or that magic bullet, and so they'll stop taking risks and start going for the sure thing. I'm praying that playtesting remains a tool and doesn't become a weapon of mass destruction. The process as it is now [at Epic] keeps us honest and takes a lot of the ego out of the equation. Whether we're playtesting internally or bringing people in, there's nothing like listening and reacting to people that are passionate about what we're doing, and it helps us make some of the best games in the industry."

Discussion

129 comments
arroz
arroz

great article! it is pretty clear that Valve (quite typically and admirably) is going into more experimental testing strategies, but as a result they are likely missing the mark. Games like L4D would have benefitted from a much more human approach to playtesting--that being said, I still love their games!

UnivAdvTech
UnivAdvTech

Amazing article! I enjoy reading about this process very much, as it is something that is lost often in every day discussion of games, but is equally as important to most game studios and should be to most gamers as well.

kazumashadow
kazumashadow

@m4a5 i realize it is a job and not playing games, but in a certain manner i would still be doing what i like. Besides, you have to like what you do for a living and it would be great for me to help make good games, my dream is being a game developer.

janzx
janzx

[This message was deleted at the request of the original poster]

Nowayouthere
Nowayouthere

Portal 1, low buget, but sucess couse all types of players like it i guess :D

horosavinXX
horosavinXX

gamespot, for the love of god, less philosophizing and more on actual games.

WolfGrey
WolfGrey

@Morphine_OD I have done my homework enough to know that. But that really doesnt say much for the company eh? Why make a sequel to a GOTY and critically acclaimed game on a crappy old buggy engine.Just says how much they gave a crap about it.

feryl06
feryl06

yet crappy ass games with bad camera angles or poor framerates for example still get released.... to call it a science is a slap in the face of science...

marlobc
marlobc

too much money,no where to spend? idea!! PLAYTESTING! lol

stabby_mcgee
stabby_mcgee

They should do an article on the science of advertising since game companies spend much more on that than on playtesting. Heck, some companies spend more on advertising than they do on the making the game.

darrenecm
darrenecm

One thing's for sure. Given the blatent and obvious bugs and 'release week' and sometimes even 'release day' patches for the majority of retail game releases these days, this so-called science needs to get it's act together.

usermikey
usermikey

They should make a current gen game that played like StarWars Jedi Power Battles for PS1/etc. 1- player Co-op, with updated graphics, and of course maybe less linear.

MuShuuzzz
MuShuuzzz

@paradroid90 not all modern games need massive developing teams or funding, like the first Portal was a fairly small, low budget game and it was developed by by like a dozen people.

Morphine_OD
Morphine_OD

@WolfGrey Amnesia - completely in-house engine, a lot of freedom what to add and what to modify FONV gamebryo - licensed 6-8 yrs old technology with limited customization options and without first-hand support. Do your homework, bro.

Sargatanas13576
Sargatanas13576

This is one of the driest, most boring articles I've ever read. I tried. Really, I tried...but....zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

mr_gibberish
mr_gibberish

The thing is I bet on paper games testing sounds amazing but when it comes down to the crunch I don't think it would be. I think it would ruin an enjoyment of just playing a game normally when you get your hands on it. Image you walk in the office in the morning and told, "OK, I want you to spend the next couple of hours doing the last section of the last level in this game and walk backwards while jumping!". It just will show you the last level of that game plus you can never play it normally! I swear once understanding about how a certain game works some of the magic disappears! I studied animation and now I can not just watch an animation film without deeply studying all its parts. I can not just sit down and enjoy a piece anymore!

Nodashi
Nodashi

@Evtutchenk0 The fact is, the games industry is not exactly unexistant here, but it only hires programmers and system analists. Like most cheap labor countries (I have to be realistic), transnational companies only use brazilian workers to do the hard work, in the case of games, programming and coding. Other than that, they (and only the big ones) only keep small PR teams here, and sometimes a few people to deal with imports, logistics, etc. I am, on the other hand, a marketing professional, now taking law classes. I could only work on the gaming industry if the administrative decisions where made here, not on the heardquarters (usually in the US or Europe). If I'm lucky enought, maybe I could be hired by Blizzard to deal with taxes when I'm a lawyer... but then I'd not really be working with games close enought so it matters, right? :)

Evtutchenk0
Evtutchenk0

@Nodashi why don't yo start this Brazilian gaming industry then? i can say that you would make a lot of money out here, and maybe even revitalize the world of games.

hitmanxmk
hitmanxmk

@shadow551991 Yeah man, too bad, game industry took a "wrong turn" - following the wish of casual players and made a games which aren't requiring much intelligence, spent time, or player skill, they don't offer much choices...at the end they are dull and boring after an hour of play. And I agree, they are eye Candies...and that's all they offer.

umster
umster

all i can say is " one of my dream jobs, along with beer testing and girlfriend testing"

WolfGrey
WolfGrey

Lol at you all arguing if money has a say.Many Indie games that hit it big have a very small budget and almost no bugs. One cult hit can argue easily. Amnesia: The Dark Descent. One of the best survival horror games i can ever mention.Small budget, third company.Not even huge profit.But very highly acclaimed and all positive reviews.Not to mention almost entirely bug free AT RELEASE. New Vegas.Huge budget, tons of money involved.One of the most glitchy and buggy pieces of crap ever made.Even had to make a official apology THE DAY AFTER it released.Ya.... So yes hard work and actually caring about their games makes a big difference.Money really doesnt.

NTM23
NTM23

Actually, it's not in Seattle. It's in Bellevue and Redmond, WA. Though for those that don't live here in WA, I guess saying Seattle is the most recognizable way for people who don't live here in WA to understand where this takes place. Man, it feels kind of weird living only about 15 minutes away from these top game companies, and I hardly think much about it.

ShadowSaturn38
ShadowSaturn38

@UltraredM-I have to say you are right.If i remeber we have had what are called game drougths where many games are made but are poor,but when halo was made the attention and detail it got really shows what bungie is.Same with BioWare.BioWare always has a way of opening your mind.Valve makes you use your brain.Now i will say that yes Halo has flaws but what game doesnt?The only games i cant stand are COD games.They just dont have the story that halo,Mass Effect or Gears has.and thats why i really dont play them and for people who keep telling me "Halo sucks" well your wrong.If it wasnt for Halo,where would we be?

OmegaVader
OmegaVader

Valve is most definitely the epitome of the playtesting-design method, but Bungie? I refuse to believe they had peopel play through Library in the first Halo and choose not to make any tweaks. In fact, most of their game is just room-with-encounters followed by another-room-with-encounters, unlike more scripted titles. The only thing that needs playtesting is that you don't fall through the world, much less help with the actual design.

illagueril1
illagueril1

@paradroid90 I for sure see your point and agree. And jus so everybody knows, I'll play a game with tons of bugs, as long as it doesn't break a game, I jus perfer not too. It's jus like New Vegas, it sat on my table for months, than a few patches later, I give it another chance and put 150 hrs in it and really enjoyed it. I guess as gamers we all have the same thing in mind, we pay alot for these games and jus want a good product

Paradroid90
Paradroid90

@illagueril1 GTA IV was monstrously expensive, EA doesn't spend that much per game even though they are big. The point I was trying to make is that modern games are so complex that just having committed people developing them isn't enough, you need lots of money. Since there is a limited amount of money in the computer game market, there can only be so many AAA titles with lots of content and few bugs. Also I don't believe that Rockstar doesn't playtest their games and rely on seeing all the problems themselves. If you have crafted a piece of code you often become blind to its limitations. I do however agree that Rockstar couldn't be successful without people with talent and vision. But they need the cash to make a finished game out of that talent and vision. Just mad skillz isn't enough.

UltraredM
UltraredM

Bungie has enough pride to walk away from a series that, should they have continued to work on, would have continued to generate massive amounts of revenue for them.

UltraredM
UltraredM

These comments on Halo games being crap make me laugh. The gaming industry is flooded with so many poor or downright awful games that it has corrupted a vast amount of people's taste in gaming. Every Halo game, with perhaps the exception of ODST, was crafted with love and dedication that you just don't see often. For every AAA title video game you can name, there are 30 more that aren't worth the plastic they're shipped in. Also, before you rant on me being a Halo fanboy, this is coming from a person who won't be buying future Halo games. The Halo series has run its course, and I won't support it following in Call of Duty's steps.

illagueril1
illagueril1

@paradroid90 Money doesn't make games, people do. EA is one of the biggest game comapanies in the world and produce more games then any other company per year, there MONEY is unlimited, yet more than have thier games are crap. You can't jus throw money at a statue and expect it to come to life. Games are like movies, jus because a movie has a 150 million dollar budget doesn't mean it's goin be good, it depends on the director. The director style, commitment, dovotion and PRIDE. More money means longer dev cycle, that's it, it doesn't mean better game, ala Duke Forever!!!!!!

Paradroid90
Paradroid90

@illagueril1 You say that Rockstar makes bug free GTAs every 2 1/2 years. The lack of bugs in GTA IV depends on the massive amount of resources they put into developing it. And by "resources" I mean money. So "love and devotion" and "pride in your work" aren't the key factors. Earlier GTA games had major bugs. Remember the Purple Nines glitch in GTA III? Or the garage glitches in pretty much every game in series?

illagueril1
illagueril1

The main problems is who's developing these games, they rely on testers to tell them what is good to play, instead of playtesting the game themselves. And a game can be shipped without bugs, let's see here "every Metal Gear ever made!!! " Hideo makes his games bugs free with the best current graphics when his games come out. Rockstar makes bug free GTAs every 2 1/2 years, how do they did it, it's called love and devotion. It's having a lil something called pride in your work and never wanting to say, yeah skip it, we"ll jus leave it like that, it's a minor bug. What if car company's did that, or the people who fabricate airplanes said yeah, jus a minor bug. That would be insane. Do these fools not make enough money?? The dev cycles get smaller cause jus like any job you give a time frame, the sooner it's done the more money the company saves in the end. It's all about, well this is good enough cuz were reaching are " not gonna make 159% profit mark" Ive been gaming since before there was a Super Mario!! And the gaming world has changed time and time again, but that was innovation an such, now it's changing fir the worst, greed an such. Games can be bug free and innovative!!!! Trust me, I've seen em!!

Mondrath
Mondrath

@slavgunner Though there are several games that are exceptions, on the whole I agree with what you are saying.SP (at least in the FPS genre) seems to have been killed by its more popular brothers: MP and Laziness.

ptflea1
ptflea1

if bungie playtested with people who play games but never played halo (like me) then they would realize that having the left trigger as the frag button is not a good idea. i played halo 3 and 1st thing i did was blow myself up trying to aim down the sight. the weapons didnt feel powerful enough (and look like water pistols) and the vehicles are hard to control, i'm used to acelerating with trigger and getting my head round using the sticks with this strange contraption with a big wheel was harder than driving a tank, which also use stick controls.

slavgunner
slavgunner

Hope Codemasters (Dirt 2 and Dirt 3 freezing, no one noticed that for some reason. F1 2010 numerous obvious and ridiculous bugs), Konami (every current gen Pro Evo has some cheap goal scoring method) and Rockstar (GTA 4 doesn't work in 1080p on some machines for no apparent reason) took this as seriously I'd wait another month to have a game that worked. If the game sells well when it is broken a patch will not be coming. Half Life 2 is breathtaking, the industry moved on from there with jokes like the 25 minute Homefront, and continuously respawning enemies in CoD (until you move past some arbitrary marker) I keep getting piles of s##t that cost £40, having 'redeem' codes that I need to input on live to get half the game; and I greatly look forward to paying (fighters costumes Capcom? wtf!) for tiny pieces of extra content. If these idiots released games that had no bugs and lasted more than 15 minutes then you wouldn't find them second hand for £15 two months later. Playtesting my a###! In a couple of years we'll all have ps4 and xbox 720 consoles that developers know nothing about. Games will cost £70 because of development costs, be riddled with bugs and be 10 minutes long. Except Half Life 3....probably.

bigtruckseries
bigtruckseries

"Bungie playtested its Halo games with people who had never even seen a shooter." And it shows. Halo 1 was the best of the series, but, there were PC games long before Halo with better graphics and better gameplay. Halo did for XBOX what Goldeneye did for N64. It made an otherwise unnatractive system - attractive! Thing is, console gamers have no idea what they'd been missing and thought crappy games like Halo were the greatest things since sliced bread. And because all the game companies are working together with Gamestop to control release schedules, they are guarranteed to sell because each subsequent game offers less filler to keep us playing. I have a BETTER IDEA. How about they get REAL GAMERS to play test these games so the end product DOESN'T SUCK and isn't a REHASH of the game before it - only with a moderately different story? If I'd been on the Playtest team for Duke Nukem Forever, either that game woulda never been released, or it would have been released PERFECT.

Rocker6
Rocker6

@Kaine852 They can take their time with the game,but I want at least some news about it,I want to know if it even EXISTS,if they are making it!

_Judas_
_Judas_

@Kaine582: Some games take 12 years to make, and end up being one of the worst games of time...

squall_83
squall_83

@Kaine852 I know that had to be sarcasm... Please tell me that was just sarcasm.

squall_83
squall_83

@kwanzudood YES! I must now go and watch that movie again....

ultimateninja95
ultimateninja95

a great game must have a public beta (im talking to you infinty ward) and lots of playtesting the feedback counts towards making the game near perfect i mean look at the call of duty franchise its annual relaese every year in november has in my opinion ruined the seires. the only reason i play black ops anymore is because of zombies if the game had a beta then feedback can be given to make sure it doesn't end up like modded warfare 2 or glitchapalooza, sequels need time to be made great.

kwanzudood
kwanzudood

There's an awesome documentary out on DVD on the subject of videogame playtesting, it's called Grandma's Boy.

Kaine852
Kaine852

I don't see why people are rushing Episode 3 (And some wanting a team fortress 3), I mean good games take time to make, not a freaking sequel every 1 year or 6 months

Sk8rlink
Sk8rlink

No wonder effing Halo Reach sucked, if you give it to noobs, it'll end up being noob friendly

shadow551991
shadow551991

@DemannameD well i think by the time they release episode 3 i just have to go over all of the half lifes to remember wtf was the thing all about... @hitmanxmk too right mate, yeah i confess i played a GTA (vice city) game and that was the first and the last time i ever touch a gta or any gta kind game because at that time i was 12. today i can hardly find a game suitin me. only things like dead space 2 or assassins creed standards are appealing to me. i have a younger brother that "shadow of the damned" was great to him and his friend while i just played the first 5 minutes and i couldnt bare to play more. @No_Intelligence and how am i supposed to play something that isnt entertaining? just pay for it and throw it in the corner of the room? this isnt called support it makes us look dumb and they think they can sell us more of their craps.

chechak7
chechak7

"listening and reacting to people that are passionate about what we're doing"

illagueril1
illagueril1

I've always thought that being a playtester would be the best job ever!! Until I watched a show about it and saw that 30 people sit in a room and play a broken game for 16 hours a day!! Yes, most of the time, 16 hours a day playing the same death match after deathmatch, or level after level. F--k that. I give em mad props thou, cuz even though I've gone on many 10-15+ game runs, I couldn't play the same unfinished game for 6months straight, 16 hours a day

No_Intelligence
No_Intelligence

@Rocker6 You have no idea how hard the SEs work to get rid of the bugs. They have no choice but to ship the game with "shippable bugs." They don't need a bigger playtest team, they know that the bugs exist, but they simply couldn't fix them all in time. What they need is a longer deadline and a bigger budget and simply more time to develop the near perfect game. I feel that you are a concerned gamer, which is good :), but don't know the actual process of game development. Fixing a bug, might end up causing 2 bugs. Because the game is built upon an engine that they have probably used for 3 years, there are limitations as to what they can fix. Framerate issues especially, the memory allocation is limited, when more content is added, the more memory is required. If the producer suddenly wants a couple more characters in a cutscene, that alone, can potentially break the ENTIRE cutscene. So the cutscene now requires more memory, which means less memory for the next thing, and so on... Which is why there are endless bugs. Ultimately, if you want better games in the future, please buy the games and support the industry.