Kinect to recognize American Sign Language

According to a patent listing for Microsoft's gesture-recognition technology, the gear recognizes ASL, maps toes, and reads lips.


Microsoft's forthcoming $150 full-body motion-, facial-, and vocal-recognition technology, Kinect, will read lips, track toe movement, and recognize sign language for those who cannot speak.

According to a Kinect patent filed by Microsoft recently, those capable of communicating through American Sign Language standards will be able to input letters, words, and phrases using the imminent technology.

Kinect can now read your lips, map your toes, and watch you sleep.
Kinect can now read your lips, map your toes, and watch you sleep.

"Where the user is unable to speak, he may be prevented from joining in the voice chat," reads the patent. "Even though he would be able to type input, this may be a laborious and slow process to someone fluent in ASL. Under the present system, he could make ASL gestures to convey his thoughts, which would then be transmitted to the other users for auditory display.

"In this situation, for example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED'. In another embodiment, a user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user's intended words and process them according to the context under which they were inputted to the system."

Further, the patent description reveals Kinect's "skeletal mapping" system, a line of code that enables the technology to track everything from a player's hands, to his or her smaller appendages like fingers and toes.

"Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes."

Microsoft's Kinect will be released on November 4 by itself for $150 in the United States and £129.99 in the United Kingdom. The new technology, which Microsoft is treating like a new system launch, will also be available in a $300 hardware/software bundle on launch day.

Please use a html5 video capable browser to watch videos.
This video has an invalid file format.
Sorry, but you can't access this content!
Please enter your date of birth to view this video

By clicking 'enter', you agree to GameSpot's
Terms of Use and Privacy Policy

Got a news tip or want to contact us directly? Email

Join the conversation
There are 199 comments about this story