How the Fight Creates the Fighter

In the competitive, fast-paced world of fighting games, no choice is without consequence. Your attacks, and the skill with which you wield them, could mean the difference between victory and defeat. Each of these choices creates a bit of telemetry data, information that has another life after the dust settles. Whether it is recorded in a humble notebook or in a massive data farm, this data is being analyzed to help shape the future of your favorite fighting game series.

Katsuhiro Harada, producer at Namco Bandai Games (Tekken Tag Tournament 2); Adam Urbano, senior producer at NetherRealm Studios (Injustice: Gods Among Us); and the team at Namco Bandai Games' Project Soul (SoulCalibur V) explain how they make sense of your telemetry data.

What exactly is telemetry data? In the realm of video games, telemetry data refers to the highly granular player data generated the moment you start a game. The recorded information varies between development teams, but it can range from the order in which you play different game modes to the number of times an attack is used. For fighting games, statistics such as character popularity, frequency of throws, damage from combos, and more can be invaluable when designing future updates.

"I could tell you in all the online one-vs.-one matches how many people choose Nightwing, and when they do choose Nightwing, how many times they use a specific move and how many times that move is blocked by the opponent," Urbano explained. "The amount of depth we can go into to make the game perfect, in terms of the gameplay experience, is crazy!"

This data can come from a variety of sources. Focus groups and questionnaires used to be the go-to choices, but modern advances in connectivity have made it possible to expand these test audiences globally. For Urbano and Harada, that means developing in-depth data-mining programs that collect, catalog, and upload every input from every available player to a central database--anonymously, of course. In the case of online matches, telemetry data is stored locally until the match is finished so that no bandwidth is lost.

However, not everyone places their faith in raw numbers. As Project Soul explained, the data fails to capture what is most important: the human element. "We know that companies like to collect and analyze data through mining methods, but we chose not to do that for [SoulCalibur V]. We consider 'data' as nothing but a list of numbers that fails to represent players' passions and emotions during matches. We derive our conclusions from fighting game tournaments and match videos uploaded to online sites such as YouTube. There is no better reference for us than getting a close look at players actually fighting."

"We consider 'data' as nothing but a list of numbers that fails to represent players' passions and emotions during matches. We derive our conclusions from fighting game tournaments and online match videos."

That's not to say Harada and Urbano turn a blind eye to the videos fans post online. Seeing a game in action is just as important as seeing it on a spreadsheet, and as Harada explained, the two sources complement each other. "By comparing the claims that part of the users mention out loud on the Internet with objective data, we can determine if that claim is reasonable or not. We believe what's important is that telemetry data represents only part of the feedback we consider. Human beings analyze with logic but determine by emotion at the last call, so [Tekken] wouldn't be a satisfactory game using logic alone."

The uses for telemetry data extend to other aspects of development as well. For Urbano's team, the telemetry system in Mortal Kombat's also controls when the player receives certain unlocks and rewards, since the game can detect individual actions. "Another great example is story mode," he added. "We looked at how many people completed each chapter in MK's story mode, and we saw sharp curves at specific points. We investigated those points, and it was clear they were at those ridiculously hard fights. So with Injustice, we designed a fight curve that helps people complete story mode."

A lack of hard data hasn't stopped Project Soul from trying new ideas with game balance and special attacks. "We believe the most important data is players' reaction and if they are having fun. If there is an attack that deals a lot of damage but players never use it, we cannot consider it a strong attack. Conversely, players can view an attack that deals a small amount of damage as a strong attack if they really enjoy using it. Therefore, it is important to watch actual matches with our own eyes."

"For SoulCalibur V, the game designers attended the tournaments and watched the streaming videos to analyze and discuss in depth how players' emotion was up and down, and how well each attack can be combined to make combos. After that, we implemented the adjustment tentatively in-game and decided whether to implement it in the product after our sufficient examination."

While these developers enjoy the breadth and depth of information provided in the digital age, collecting that information hasn't always been so simple. "When Soul Edge was released in 1996, the creators could collect player feedback by going to arcades or giving out questionnaire postcards," said Project Soul. "The basics have not changed, but the Internet allows players from all over the world to upload or stream their matches. That increase in data has given the development team more information to draw from than ever before and helped achieve a solid balance for SCV."

"We believe it's important that telemetry data represents only part of the feedback we consider. Human beings analyze with logic but determine by emotion at the last call, so [Tekken] wouldn't be satisfactory using logic alone."

Harada is used to doing the legwork and visiting arcades, or at times requesting they fax him information. "Actually, we have been collecting simplified telemetry data from the first arcade Tekken," he said. "However, at that time, it was mostly just the character usage rates. We started taking a little more detailed data, including the win/loss data, starting in 1997 with the arcade version of Tekken 3."

"The arcades at that time were not connected to an online network, so the only way to look at this data was by flipping the test switch on each arcade board to view the collected information. Furthermore, and this information is not known yet, we knew a very difficult, hidden move that, when entered, let us secretly look at the more detailed data than the information collected from the test switch on the board. This information is something that I'm actually revealing for the very first time. We have been doing these things since 1997, but we did have the idea of collecting telemetry data as far back as 1995."

During the development of Mortal Kombat, Urbano was a one-man data-mining operation. However, the volumes of data were extremely difficult to sort through, and finding specific information could easily become an all-day affair. "We learned to have two separate buckets of data: the terabytes of detailed moves and inputs data that we can analyze to fix moves (but is harder to navigate through), and a second bucket that's just data we know we'll need on demand, such as infinities, exploits, and online desynchronization. That second bucket is full of quick data, just a hundred or so items, so that a report can be generated quickly."

Telemetry data has been a powerful tool for fighting game developers, and many believe the next step is sharing that data with the fans. "It'll be cool going forward to find ways to integrate that data so that the user can experience it," Urbano said. "In MK, we had a website that displayed a lot of player stats, but I don't think most people knew it existed. We need to figure out how to utilize this data to create new and exciting experiences for the user. There are a million ways to do it, and I think what Namco is doing with the World Tekken Federation is really cool."

The WTF gives players a virtual business card filled with stats on how they play. These cards can be shared with others to see how your fighting styles compare. "Since it was the first time in the world a fighting game tried this kind of service, it was perceived well," said Harada. "Until now, it was normal for a Tekken player not to know what kind of player their opponent is until they fight. Recent feedback from the players has requested we disclose even more detailed data."