Kinect speaks a global language

GDC 2011: Microsoft's Lief Thompson, Yumiko Murphy joined by Englobe's Kate Edwards to discuss the trials of bringing voice-, gesture-based gaming to worldwide audiences.

Who was there: With the introduction of the Kinect in November, Microsoft opened itself up to new ways to offend cultures around the globe. In a GDC 2011 session titled "New Technology and New Interfaces: Localizing Video Games for Kinect," Microsoft's Lief Thompson and Yumiko Murphy were joined by Englobe's Kate Edwards to discuss the many challenges of releasing gesture-based games across the globe.

This tiger speaks the international language of cute.

What they talked about: Thompson began the presentation by first offering a look at what Microsoft has accomplished with the launch of the Kinect. The hardware sold 8 million units through the end of 2010, and of the 14 launch titles, five went through Microsoft's internal localization department: Kinect Adventures, Kinect Joy Ride, Kinectimals, Dance Central, and Kinect Sports.

In Redmond, Washington, the localization team establishes the scope, engineering, and schedule. Microsoft also has localization teams in Dublin and Tokyo, which manage vendor costs and schedules, perform translations and testing, coordinate with regional operations, and arrange regional age-rating submissions. The goal of these global operations, he said, is to ensure simultaneous launches worldwide and to keep costs down.

There were a number of challenges that accompanied releasing the Kinect globally, Thompson said. Time was at the forefront of these, as there were really only two years between the Kinect's early design and prototype phase and its global launch. In this brief period, the team was challenged with catering to 12 languages.

On the software side, localization challenges started with security, as Microsoft was very sensitive about hardware and software leaks. Therefore, localization teams had to ensure that their testing was done in secure facilities and outside of the sight of consumers. This necessitated bringing testers to Microsoft's facilities. However, space requirements soon became a serious issue and were dealt with by testing 9 of the 12 languages in Dublin, with the remaining three occurring in Tokyo.

One other issue the team ran into was the sheer human logistics of testing a Kinect game. A single tester can't be expected to play a Kinect game for 8-10 hours per day, Thompson said, purely from a physical standpoint. Bug capture was also a problem, as players can't log a bug while interacting with the title. The solution here was to bring in two testers--one that plays and the other that logs. The two testers would then swap roles.

Thompson also noted that localization testing has a very specific purpose that is independent from functional testing. Therefore, the team was able to gain time by using a variety of debug hardware cheats, such as inputting codes to hop through or pass content.

Edwards then took the podium to speak about gestural issues and the accounting of cultural nuance. The goal is to not force players to use potentially negative gestures to control the unit or to play a game, she said. Hand and body gestures are particularly problematic for localization, because certain gestures, such as a peace sign or a thumb between the fingers can mean different things to different people.

Dance Central is the type of Kinect game that is particularly problematic, she said, before showing an image of an in-game character making what would appear to be a rock-and-roll hand gesture in the game. In the US, this gesture simply implies rock on, she said. However, in Texas, the gesture also has the connotation of hook 'em horns, thanks to its association with the University of Texas. Edwards also noted that showing someone the bottom of their foot can be quite offensive in some cultures.

Games such as this are given a gesture review, which involves examining all mo-cap and avatar moves and then making minor tweaks as necessary.

Murphy then takes the stage to talk about the speech functionality found in Kinectimals. In particular, this title supports voice commands and the Name Your Animal feature. Voice command, she said, is supported through a limited set of "grammars," which are sets of words or phrases that trigger an in-game response. Murphy pointed out that there are actually a number of ways to activate a certain action.

With the Name Your Animals feature, there were an infinite number of naming possibilities, and therefore there were no grammar files. Names would be broken down into phonemes that are then stored in memory.

Not every language was supported with the speech recognition and Name Your Animals feature, she said. The team focused in on US and UK English, Japanese, Mexican Spanish, and Japanese.

Translating for fluent speakers was of paramount importance, she said, and so it was very important that the translators had all of the appropriate context information while working. Translators needed to know how actions are used in a game, she said.

Data collection with native speakers was also important. Microsoft needed to have a wide range of age groups in the testing pool to ensure speech recognition worked for everyone. However, the game's targeted age group was given top priority in this regard.

Moving to the Name Your Animals feature, Murphy said they wanted this feature to be available in as many locations as possible. However, they ended up having to scale this back to 11 languages for 15 different countries. They compiled the top 100 pet names in each of these regions and then asked native speakers from these areas to test them. It took about four weeks to complete data collection, and they achieved 80-93 percent test accuracy.

She went on to note that scheduling was a major challenge for Kinectimals, as they had only about four and a half months to complete speech recognition localization. Due to security and confidentially issues, they could not use external vendors, as this was all occurring prelaunch. Therefore, they had to do all of their testing internally, involving Microsoft employees and their family members.

Because Kinectimals is targeted at young kids, it was important to collect voice data from many children. However, strict child labor laws complicated this process, she said.

Quote: "It's all very context-sensitive."--Kate Edwards

Takeaway: Between multilanguage voice support and gesture-based gaming, Microsoft's localization team needed to clear a variety of hurdles to bring its latest gaming device to market. These problems were compounded by the intense pressure to keep the device out of the public eye.

Written By

Want the latest news about Dance Central?

Dance Central

Dance Central

Follow

Discussion

49 comments
emerin76
emerin76

Dissapointed that I can't use the speech recognition in Australia...My wife speaks spanish, and I speak UK English... how is Australian English different to Southern UK English?

monson21502
monson21502

this is great because im starting to mix up the words in a sentence when i talk. no idea why???

Quietpower09
Quietpower09

Maybe I can start developing my Spanish language with Kinect now.

isshiah
isshiah

aussie users can't talk on kinect yet so i think i'll wait and see. great tech but not my cup of tea.

Witchsight
Witchsight

"With the introduction of Kinect in November, Microsoft opened itself up to new ways to offend cultures around the globe." Seems strange to start off an article with a negative sentence like that...

monson21502
monson21502

one thing you have to give ms credit for is. they always try to fix their mistakes..

Vorheez
Vorheez

LOL wow I just copied and pasted "Who was there: With the introduction of Kinect in November, Microsoft opened itself up to new ways to offend cultures around the globe." thats in the first sentence.

7rooper
7rooper

Very interesting news about all the Kinect testing team had to go through. I like this information since I care videogames from gaming and technology standpoint

monson21502
monson21502

who cares about this crap. wheres the news of the core games they teased us with at the last TGS????

IceJester45
IceJester45

I was pleasantly surprised by Kinect last year. I have had few problems with the how the hardware works. Of course, I have the recommended play space with plenty of room to move around. I sometimes wonder how people in places like Tokyo or Seoul could use Kinect in their apartments. I lived in apartments in two major Japanese cities, and neither was large enough to accommodate Kinect. Further, all that moving around would have probably bothered the neighbors. So, I'm not so sure that Kinect really speaks a global language.

ZICO11
ZICO11

ppl use kinect? i thought it was for touching women u heard about that game?

NColdhardt
NColdhardt

Kinect doesn't apparently understand sarcasm.

wexorian
wexorian

if it not speak Orcish i'l pass :D

pokecharm
pokecharm

I like this: 'Not every language was supported with the speech recognition and Name Your Animals feature, she said. The team focused in on US and UK English, Japanese, Mexican Spanish, and Japanese.' so does that mean they had to focus twice as hard on Japanese?

AuronAXE
AuronAXE

Personally, I LOVE the idea behind the Kinect, we really are paving the way for the future. Unfortunately, I decided it just wasn't the thing for me yet. The technology is awesome, the games...aren't. No offense to Microsoft or anyone, I just don't enjoy games like Kinectimals and Dance Central. Despite that, bravo Microsoft on the stellar product.

TheRealLisaAnn
TheRealLisaAnn

About time, Microsoft. If you want to beat $$$ony you will have to innovate.

TeTBCN
TeTBCN

I this waiting the spanish of spain lenguage. i have much accounts and one is of mexico, with Kinect I can speak good, i no understand by what no have the same in my account of spain. Is increible, first in Mexico with a old and bad spanish versus a spanish natal of spain... good for microsoft. After wait good sales of xbox 360 and Kinect in Europe. Spain is the four country in the top of users consumers of videogames.

DCUltrapro
DCUltrapro

Kinect is awesome, great little bit of kit and a lot of fun. But please for the love of jeeebus can people stop with this Political Correctness crap?? Its driving me insane, not to mention ruining society as a whole. People shouldn't have to look over there shoulders 24/7 for fear of offending some random person with their hard work in burka durka land or oommie goomie goom goom land... I mean seriously does no one on this planet have a sense of humour anymore? OR is everyone just out looking for the first chance to law suit a company for offensive material or behaviour?? Am I the only one that hates all this PC garbage?? Its worse now than ever before, and I was alive during the days of Thatcher god damn it!!

greyzhaze
greyzhaze

I like the idea of Kinect, its definitely paving the way into Jonny Quest style worlds and gaming. I just want more games that use this properly and to fix a lot of issues that I found.

KickinIt_ftw
KickinIt_ftw

mariokart64fan: "gamer =right , kenict is horrible , wii is alot more suitible for motion because 1 it is not restricted to 8 cm away from the tv ,2 it is every easy and accesable , where as kenict is not" I own both. Kinect is better. Where is your god now?

sicvita
sicvita

To me, the kinect is the only interesting thing happening in this part of the industry. Sure, the initial software line-up is weak, but stand out games like Dance Central really make this style of interface stand out in the crowd. If your on here hating just to hate, that probably means you're just a fan-boy, in denial or simply can't afford it. Honestly, in my opinion, the Kinect is one of the most impressive pieces of consumer hardware to come out in the last couple of years. If one of these new-fangled 'motion sensor' thingies is really D.O.A. it would be PS's Move. So far it just mimic's the wii albeit with better graphic's but with no real solid software support; and with that weak of an install base, saqly not much to come. All in all, I'll put it out there. If a PS move only game comes out in late 2013 or beyond come and find me; you owe me a free punch in the face. As for Kinect, so far you've got the best look, now please for some amount of substance ( outside of all the awesome hack's that I'm much to lazy to do myself) that will appeal to the rest of us 'non-casual' crowd.

ronnet
ronnet

Always nice to see a heap of self-richeous gamers. Kinect and Move aren't directly aimed at you. Look outside your own little circle and accept that although they'd like you to buy it, these things weren't designed for you. They are for a different audience that otherwise couldn't care less for gaming. This is new and it is fun, and for what it is worth it is rather succesful. You can hate it all you want but that doesn't take away the fact that things are changing. The gaming landscape is changing, if it isn't this motion generation then it will be something else. But gaming is evolving so it can have wide-spread mainstream appeal just like movies. Self-richeous critics can hate Scary Movie, Pirates of the Carribean and Twilight but these films sell and they make the industry what it is. Gaming could be even bigger then the movie industry. We can all dislike what these companies are trying to do. But Wii showed that motion controlling might be the future. Ignoring it now could mean that you're out of the market a decade from now. Microsoft, Sony and Nintendo don't want to become a niche within the market they now dominate.

DarthVeng_basic
DarthVeng_basic

@Kenshiro_Kenoh: I couldn't agree more. Jumping around the room like a hyperactive ape is not my idea of playing a game. Until they get to the stage in technology of Holodecks, count me out. I'll stick with my controller, thanx.

monson21502
monson21502

xbox. *waves at kinect* xbox! ahh f it, ill use the controller to watch a movie...

Oldboy08
Oldboy08

Oh yeahhhhhhhhhhhhhhh, kinect is still around?

Kenshiro_Kenoh
Kenshiro_Kenoh

kinect is trash so is move and all motion sensing BS i prefer the regular controller than this thats just my opinion

monson21502
monson21502

kinect doesnt need to speak.. its way more fun then those other 2 fake motion controls. remember..... actions speak louder then words:wink:

mariokart64fan
mariokart64fan

gamer =right , kenict is horrible , wii is alot more suitible for motion because 1 it is not restricted to 8 cm away from the tv ,2 it is every easy and accesable , where as kenict is not

TheArcade
TheArcade

Where's my Kinect sex game dammit! >=(

focus0894
focus0894

@inaka_rob Deleted, thanks for the heads up buddy. :)

edant79
edant79

Whats the global word for "Tower of Babel"?

Kulcan
Kulcan

Kinect is just plain fun . I hope to see better games supporting this device.

Generic_Dude
Generic_Dude

The global language: BODY language... yyyeeeaaahhh.

Gamerzmk
Gamerzmk

I am sorry inaka_rob, does it even look like im talking to you? NOOOOOOOOOO

inaka_rob
inaka_rob

@focus0894 and @Gamerzmk you guys just might want to delete your comments. I see 2 or 3 that will easily get you modded if someone wanted to report you. I HATE getting modded so just FYI. if you dont care then by all means. continue. I am enjoying it.

Gamerzmk
Gamerzmk

focus0894 You too bro ;) I love you more

focus0894
focus0894

[This message was deleted at the request of the original poster]

Gamerzmk
Gamerzmk

And besides, Kinect is fu*******ing terrible anyway.

Gamerzmk
Gamerzmk

[This message was deleted at the request of a moderator or administrator]

focus0894
focus0894

[This message was deleted at the request of the original poster]

SCV3
SCV3

@inaka_rob Finally some common sense.

inaka_rob
inaka_rob

@Gamerzmk i think you have no idea what you are talking about. they had "eye toy" technology since the 80`s. even had it in games shows on TV. so any body who grew up in the 80`s watching kids game shows on Nickelodeon could claim to have had the idea. its is 100% pointless to talk abot orignalty and who thought of what first. Watch this. Nintendo copied Atari and made a game system. Nintendo copied sony and used discs for the game cube. sony copies Nintendo by making the playstaion. playstation copied the wii by making the Move. I think the number 1 thing gamers forget is that video games are a business. If people "copied" ideas they would be sued. business may get inspiration. BUT I can 100% garuntee you that all 3 of the big console makers have a team. they have teams of teams with people who come up with idea of what to do next. What do gamers want? how far should be push the tech (because I can also 100% garnetee you that they hold back, they don't release the best of the best, the release the best they can make money off of). I am sure with the myriad of ideas that come up some are going to resemble other invention through out time and history. in the end it comes down to marketing and what the consumer wants at that time. Nintendo had 3D tech in their back pocket for OVER 10 years now. the consumer just didn't want it and wasn't ready for it. maybe the eye toy helped sales, IMO it hurt them. 1 useless product isnt going to promote another useless one.

Gamerzmk
Gamerzmk

Microsoft should really thank Sony for their success with Kinect, because come on, lets be real here. If Eyetoy never existed then Kinect would have never been thought of. Because,, after all, Microsoft are crap when it comes to originality. I am not a "fanboy" or anything, I am just speaking the truth... from experience. kk ppl :)

mdboomer
mdboomer

I'm sure enjoying my Kinect. Hats off to you MS