Kinect to recognize American Sign Language

According to a patent listing for Microsoft's gesture-recognition technology, the gear recognizes ASL, maps toes, and reads lips.

Microsoft's forthcoming $150 full-body motion-, facial-, and vocal-recognition technology, Kinect, will read lips, track toe movement, and recognize sign language for those who cannot speak.

According to a Kinect patent filed by Microsoft recently, those capable of communicating through American Sign Language standards will be able to input letters, words, and phrases using the imminent technology.

Kinect can now read your lips, map your toes, and watch you sleep.

"Where the user is unable to speak, he may be prevented from joining in the voice chat," reads the patent. "Even though he would be able to type input, this may be a laborious and slow process to someone fluent in ASL. Under the present system, he could make ASL gestures to convey his thoughts, which would then be transmitted to the other users for auditory display.

"In this situation, for example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED'. In another embodiment, a user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user's intended words and process them according to the context under which they were inputted to the system."

Further, the patent description reveals Kinect's "skeletal mapping" system, a line of code that enables the technology to track everything from a player's hands, to his or her smaller appendages like fingers and toes.

"Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes."

Microsoft's Kinect will be released on November 4 by itself for $150 in the United States and £129.99 in the United Kingdom. The new technology, which Microsoft is treating like a new system launch, will also be available in a $300 hardware/software bundle on launch day.

Written By

Eddie Makuch is a news editor at GameSpot, and would like to see the Whalers return to Hartford.

Want the latest news about Kinect Joy Ride?

Kinect Joy Ride

Kinect Joy Ride

Follow

Discussion

201 comments
Phatjam98
Phatjam98

Again everyone... this is not made currently for FPS or hardcore game controls. Its not that hard... MS isn't even marketing it toward that. Its a console/media/communications control device. Eventually I would imagine they will come out with guns of some type or whatever the item for the game may be that Kinect will recognize and will have buttons. In the mean time I get to talk to my electronics, control movie playback with gestures and walk around the room as I video conference. Again... NOT FOR USE WITH HALO: REACH AND GAMES OF THAT NATURE! Its so easy to see the sony faithfull that just want to bash MS even when MS in no way markets this device as a hardcore gamer controller replacement. LOL

jeffv547
jeffv547

so dumb I can't see straight is that a giraffe?

Dynamo11
Dynamo11

@bleach209 Cause deaf people aren't disabled, right...... right?

FlashCharge
FlashCharge

Its great to see that MS and PS are creating devices for the impaired to let them experience the joys of the video game era. Hats off to both companies.

DiscGuru101
DiscGuru101

Now drop it to $89, and throw in a killer app.

SDBusDriver1979
SDBusDriver1979

Eventually Microsoft will get around to combating Sony's marketing with the slogan "It only sees everything".

bleach209
bleach209

meh i am not sold and besides ps move can let disabled people play

Locutus_Picard
Locutus_Picard

The very first and probably last feature of the Kinect that's actually useful.

MacenKrace
MacenKrace

Why wasn't this made for the PC first? blind ppl surely needed this more than us.

COPMAN221
COPMAN221

Can someone explain to me how this will work with first person shooters like halo reach?

Dan21278
Dan21278

MS should make this sort of tech for PCS also.

gbrading
gbrading moderator

Recognising sign language is actually rather clever. I'd love to see it in action though rather than see clips from Kinectimals.

jupsto
jupsto

Well I used to think kinect was completely pointless for gaming, but hey if it actually helps disabled people play games then fair play maybe it isn't a complete gimmick.

frankj22
frankj22

For kinect haters or lovers. Even if the kinect doesn't deliver what we're expecting, It's a good step foward in video entertainment. The wii was the initial effort and kinect will be the first ittaration of the concept. We should look beyond the gaming context. Some poeple here in the comments post some good ideas and somehow someday those apps or features will be embeded in video game.

Senor_Kami
Senor_Kami

That's cool for those who need it but what impresses me is that it's sensitive enough to detect the movement and position of individual fingers.

blakeney
blakeney

I have seen ASL in action...... i have also seen all kinds a "recognition" things like voice recognition, face recognition etc. Anyone not caught up in a fluster over all this, will know that the ASL recognition will mess up what they are "saying?" and they will end up having to do it all real big and slow (aka just use the keyboard yeah??:P)

hitomo
hitomo

PcGamingRig Posted Aug 9, 2010 4:37 am PT maybe it's a mind controlling machine they're trying to get into every house so everyone buys their products... ->: you mean like the i-pad or google-phon ^^

hitomo
hitomo

what do you want .. is as useless as the wii ... but when nintendo brought this scam up you where all happy and in for the revolution, now that MS and Sony able to bring the tech that Nintendo was promising you are all upset ... if the tech really works like MS is stating, it would be so more useful everywhere else in the society except videogames ... it could help people unable to speak making everyday tasks eayser and more comftable ... if this tech is real its a total waste in the videogaming sector ...

kipohippo021
kipohippo021

drummerdickens is right, i want video games, not features.

drummerdickens
drummerdickens

[This message was deleted at the request of a moderator or administrator]

drummerdickens
drummerdickens

[This message was deleted at the request of the original poster]

SicklySunStorm
SicklySunStorm

"Even though he would be able to type input, this may be a laborious and slow process to someone fluent in ASL." gee and there was me thinking we had these old fashioned things called keyboards that made typing things real easy... don't get me wrong, it IS impressive that Kinect can do this, but when you already have something more simple and more reliable than something so flashy and un-necessary, you kinda have to ask - why bother? You could pay your $150 for kinect, and it will recognise your signs to a point (I guarantee it will struggle on occasion - it's new tech) or you could spend $20 on a USB keyboard that you could plug in to your 360 or PC and would do the job just as perfectly...

ParisSun
ParisSun

shani boy does have a point. I think this definitely needs to be a PC application. I could really use this on my work PC. I could also see a use for being used at information kiosk at airports, malls, office buildings, etc. Kinect has some real world applications that make more sense than use on poor casual games. That said, hardcore games are not just FPS. I could definitely see kinect being used in a games like Elder Schrolls, Knights of the Old Republic, Dead Rising and Tomb Raider.

SIDEFX1
SIDEFX1

Cool .. now they just need to make my game room a virtual parallel world for me to play in and then i can enjoy virtual reality without a headset..

PcGamingRig
PcGamingRig

maybe it's a mind controlling machine they're trying to get into every house so everyone buys their products, DON'T LOOK AT THE TIGER!!! :D

teknicz
teknicz

What game of Kinect's upcoming lineup ever requires one to say "PWNED" anyways? ...? Pointless to even suggest this.

shani_boy101
shani_boy101

The technology is impressive, but every single game i've seen from Microsoft using Kinect is horrible, casual, shovelware garbage and there is no hope of getting a hardcore game to use it (because of it's strain on the hardware), whereas with the Move it can quickly be patched in. Like I've always said, if Microsoft used this with PCs, it would be a killer interface. That's where it's potential lies.

amaan4ever
amaan4ever

[This message was deleted at the request of a moderator or administrator]

amaan4ever
amaan4ever

[This message was deleted at the request of a moderator or administrator]

Darth_Nater307
Darth_Nater307

To quote Darth Vader, "Impressive, most impressive."

zzamaro
zzamaro

Forgot to say, I'm not saying that the PS EYE can actually do what this article says because it can't.

zzamaro
zzamaro

@TTDog Well, Sony back in 2005 promised us a game that for the new camera and it was supposed to feature the voice recognition function but they never released it. The PS EYE DOES have that voice recognition function, as far as I know, EYEPET is one game(and possible the only game) that it uses it. I saw a video(not a pre-recorded video or something) where you canwhistle something and then your pet will imitate that sound. Eyedentify is the name of the game, here: http://www.youtube.com/watch?v=pt-Cah3Gz44 Kinect is more advanced than the PS EYE obviously because the PS EYE is very old now.

jmcmanj
jmcmanj

i'm sort of creeped out now (1984!!!)

bjmage16
bjmage16

I'll probably get the Kinect later, but it all depends on how well it is supported. I was all over Nintendo's pioneering into the motion control world, but that has proven to be far less than amazing. And while it looks promising, Microsoft needs to ensure Kinect is well supported not only by themselves, but by 3rd parties as well.

bluefire1360
bluefire1360

I'm not interested in the kinect, but it's defiantly an awesome piece of technology.

ParaBellum71
ParaBellum71

Basically my point below is, why are you creeped out by a device you will probably never buy, and why are you creeped out when you most likeley have a webcam plugged into your computer..... and probably have little to no idea on how to apporpriatly defend your comp outside a firewall.

ParaBellum71
ParaBellum71

Good news for the disabled. And quit bashing the Kinect. I never intend to buy it, but if this or the new WII touch app for the computer help one disabled person, then I support it 1000%. Imagine loosing some fingers or the ability to speak, as a gamer. And as far as the Kinect telling you are asleep, turn it off. Jesus H Christ, this is the same as a webcam.... and with a webcam people can hack into your computer and ACTUALLY watch you.

Omegaman007
Omegaman007

2 things: 1: The picture says Kinect can now watch you sleep. Anyone else creeped out by that? 2: American Sign Language? That's great, but I use Spanish Sign Language. What will I do?

Omegaman007
Omegaman007

2 things: 1: The picture says Kinect can now watch you sleep. Anyone else creeped out by that? 2: American Sign Language? That's great, but I use Spanish Sign Language. What will I do?

Link3301
Link3301

i dont get why people bash motion contol its awesome. The kinect will be awesome im sue of it. Oh and guess what i dont own a xbox.

lucium689
lucium689

duuude anyone else read 1984?

criinok
criinok

"In this situation, for example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED'." I pray that no one would ever do this, out of respect for the dignity of all mute people across the world.

Darkages04
Darkages04

Lol I posted a comment here about kinect body recognition, and someone reported it, LMAO, who is the baby then?

Maluigi
Maluigi

Soon, the Kinect will be able to read your thoughts...