Microsoft files patent for head-mounted display

Microsoft has filed a patent for a multiplayer voice- and gesture-driven head-mounted display.

A patent filed by Microsoft on January 30, 2012, and this week made available to the public, outlines a series of head-mounted display devices and their potential gaming applications.

The filing (spotted by a NeoGAF user), dubbed "Multiplayer gaming with head-mounted display," describes "a system and related methods for inviting a potential player to participate in a multiplayer game via a user head-mounted display device…In one example, a potential player invitation program receives user voice data and determines that the user voice data is an invitation to participate in a multiplayer game. The program receives eye-tracking information, depth information, facial recognition information, potential player head-mounted display device information, and/or potential player voice data."

The patent alludes to a reference design (subject to change) that may include transparent or partially transparent displays that sit in front of a user's eyes in the form of traditional glasses. The patent filing includes a range of potential device scenarios, including the use of Organic Light-Emitting Diode (OLED) displays, eye scanning, microphones, and gyroscopic and accelerometer sensors to determine user head and body positioning when worn.

"The HMD device may include an eye-tracking system that utilises at least one inward-facing sensor. The inward-facing sensor may be an image sensor that is configured to acquire image data in the form of eye-tracking information from a user's eyes. Provided the user has consented to the acquisition and use of this information, the eye-tracking system may use this information to track the position and/or movement of the user's eyes. The eye-tracking system may then determine where and/or at what person or object the user is looking. In another example, the inward-facing sensor may capture retinal scan information from a user's retina. Provided the user has consented to the acquisition and use of this information, such information may be used to identify the user wearing the HMD device."

Outward-facing sensors would also detect the environment of the wearer, locate users sharing the space, and track gestures performed by the wearer. Numerous mentions of "time-of-flight depth camera" systems also draw parallels with similar technology included in the updated Kinect 2.0 that will ship alongside the upcoming Xbox One console. These could be used to scan the space before determining which games may be the most appropriate for the given play space.

"The HMD device may also include an optical sensor system that utilises at least one outward-facing sensor, such as an optical sensor. Outward-facing sensor may detect movements within its field of view, such as gesture-based inputs or other movements performed by a user or by a person within the field of view. Outward-facing sensor may also capture image information, such as facial recognition information, and depth information from a physical environment and real-world objects within the environment. For example, outward-facing sensor may include a depth camera, a visible light camera, an infrared light camera, and/or a position-tracking camera."

The potential inclusion of three accelerometers and three gyroscopes would allow for head-positioning tracking, not unlike the technology found in the Oculus Rift.

"Motion sensors may also be employed as user input devices, such that a user may interact with the HMD device via gestures of the neck and head, or even of the body."

Voice controls, with a focus on natural speech, are mentioned throughout the filing. The HMD may include one or more microphones, and be used to input commands and requests, such as when instigating a multiplayer game with another user.

"In other examples, audio may be presented to the user via one or more speakers on the HMD device. Such audio may include, for example, music, instructions, and/or other communication from the multiplayer game program, the potential player invitation program, or other sources."

The filing also lists provisions for users to toggle the amount of personal information shared by the device with other users, from limited privacy modes showing user IDs and game achievements, all the way to exchanging personal information, such as game interests.

While only a single device appears to be wearable per user, the report includes information on multiple head-mounted displays being used in conjunction with each other, either to view other matches in progress or join games with multiple participants.

The technology and accompanying diagrams and descriptions in the patent share some striking similarities with images shown previously in a purportedly leaked document, prior to the Xbox One announcement. The information was later taken down at the request of Microsoft.

Written By

Discussion

193 comments
robertcain
robertcain

I'm guessing this is MS's answer to Google Glass, after all that did turn a lot of heads when people tried it out over the past few months of development. Still they'll probably just stick with Kinect for Xbox One though.

Monsterkillah
Monsterkillah

The only reason why I'm buying this is to scan people power level

I hope it's not over 9000

Andrew Richtmeyer
Andrew Richtmeyer

How about no? How about getting rid of the kinect must have for XBox one?

deathstream
deathstream

This patent is a year and a half old.  Perhaps the trolls should crawl back under their bridges until this is a real story.

Chuck Renninger
Chuck Renninger

Seems it will be LIKE oculus rift.. But with Kinect technology involved.. Which seems pretty cool if you think about it... (Notice the "users" hand articulation in the graphic above?)

Oozyrat
Oozyrat

I don't even think the OR is that big of a deal, so I won't be spending any money on this. The only thing I can see happening with something on someone's head is a HUD displayed on glasses or something (and I still wouldn't even buy that).

mingolo
mingolo

Hmm...as far as I can tell by all the mumbo jumbo spoken above, these are NOT VR glasses like the Oculus Rift, they track head movement like the rift, but the image on the glasses is only a transparent Head Mounted Display for minor elements like the HUD and stuff, you would still need to look at your TV for the actual game. 

More than anything, they're yet another motion control that can now track head and eye movement, along with displaying minor info on a screen. 

Brian Lee
Brian Lee

Better to listen to a stoned dude in a suit.

andrescapo_
andrescapo_

Oh no! Microsoft is trying to develop new hardware with motion controls! Everyone hide!

Bakercat
Bakercat

After watching Sword Art Online,I don't trust head-mounted 

video games.

warhawk-geeby
warhawk-geeby

I call bullshit.

I'd LOVE it to be true, but I think it's just an expensive PR stunt for attention.  If ain't though then awesome :D

ElDuderino1
ElDuderino1

An NSA powered Oculus Rift, get hyped baby.

pamant
pamant

I cant understand ppl saying that is like rift and ms should not make it. so with that thinking there where be only one mobile phone in the market only on pizza only one car etc. We dont know yet what is this and everyone taking sides. How can you tell if this will work or not? You know only what the original writer wrote. So I will wait if this is make the market and is good i will support it.

soulfulDAGGER
soulfulDAGGER

Uh oh..watch out Oculus Rift. Looks like you may have missed a couple patents to file. Here come's a juggernaut

snugglebear
snugglebear

So they want me to wear their glasses so they can watch me while I'm "sword fighting?"

Qmajinbuu
Qmajinbuu

@COPMAN221ISBACK More then likely.  MS is working with the government and since their "always online" plan failed, they need some other way to spy on people.

mike9876543210
mike9876543210

@pamant what this is is a device simmilar to the google glasses with motoin and eye sencors with xbox capabilities, nothing like the rift as the rift is a head mounted video display not a head mounted hud.

shadow6323
shadow6323

@pamant the simple response is bandwagon hate. A lot of people are going to be hating on MS for the next little while more than usual no matter what they do, because that is what's "cool" right now.

mike9876543210
mike9876543210

@vadagar1 where in this whole article does it say that this device is in any way shape or form like the rift, the rift is head mounted video diplay, in other words its two lcd screens inside a plastic case that you put on your head to give you a 3d experience, this device it a head mounted hud (heads up display) with xbox connectivity, nothing like the rift

Qmajinbuu
Qmajinbuu

@Oozyrat @COPMAN221ISBACK The NSA's illegal and wide ranged actions were revealed  alittle over two months ago and no one has even been charged for the crimes being committed.  So just how is this old news???

pamant
pamant

@shadow6323 You are reasonable here so I will ask you something: When you go to buy a mobile phone did you ever ask them to cut out the camera so the price go down? :P Or do you believe someone doing that?

Qmajinbuu
Qmajinbuu

@R2C25

@Qmajinbuu

 @COPMAN221ISBACK

I'm not a paranoid sociopath either, but that still doesn't give the NSA the right to spy on me or anyone else all the time.  And they don't need manpower to track every person, that is what they're new spying decives do.  If you read those two articles I sent, you would have read that the government is excited and ready to use any and all tech in everyone's house to spy on them.  These devices can watch, record, send, and store all forms of information, so manpower isn't really needed to gather or monitor anymore.  And then there are automatic license plate readers.  Ever hear of it? 

Well they're cameras that are placed on top of police cars, bridges, traffic lights, or anything else that is stationary.  Originally, the purpose of these devices were to just catch criminals.  But the government has admitted they're now using it to record the license plates of the entire population and track everyone wherever they go.  And they don't need manpower to do this either because the cameras are everywhere and over time, a profile of everywhere you go and everything you do is built.  In some cases this info is held for hours or days, but in most cases it's stored for months and years.  In some areas, however, it's stated that this info is never deleted.

Police Documents on License Plate Scanners Reveal Mass Tracking | American Civil Liberties Union

I know whenever I say anything it's being monitored.  But i'm not going to sit here and be quit like a good little sheep or puppet as the government repeatedly break the law with these programs.  If people don't speak up now, they might not be a chance to later, as history tells us.  History also tells us if we don't learn from it, we're doomed to repeat it.

R2C25
R2C25

@Qmajinbuu @R2C25 @COPMAN221ISBACK They can spy on me all they want. I'm not a paranoid sociopath to start with, then they don't have the manpower to track every single being on US and abroad. You're bitching about being spied on and you're leaving your print all over internet with your stupid comments.

Qmajinbuu
Qmajinbuu

@R2C25 @Qmajinbuu @COPMAN221ISBACK

How true, but that would still leave pr*cks like you unmonitored.  Wait, nevermind, the NSA has got that covered.  With their new line of cameras in fridges, washing machines, dryers, ovens, microwaves, clock radios, and game systems, they'll be able to watch everyone wherever they are.

Smart appliances could help CIA spy, says Petraeus.

The CIA wants to spy on you through your TV: Agency director says net-connected gadgets will 'transform' surveillance | Mail Online

Qmajinbuu
Qmajinbuu

@deathstream

@Qmajinbuu

 @COPMAN221ISBACK

Idiot???  Idiot?!?!

Ok, I usually try to not insult people, but for you I'll make an exception. 

Now then, just because the three branches of government say something is legal doesn't make it legal.  The constitution, THE SUPREME LAW OF THIS COUNTRY, says whether something is legal or not.  Our founding farthers made it so that laws and amendments could be added to the constitution, but only if they didn't erase or destroy any freedoms that were already guaranteed.  If any laws were somehow added that hurt freedoms, then our forefarthers say the constitution wins over them because it's THE SUPREME LAW OF THIS COUNTRY!  This fact is very easy to understand, so easy that most people learn this in elementary school, you moronic fool!

But for the sake of agrument, lets just forget the constitution.  Even then, the spying the NSA is doing on the American people is still illegal!  Why do you ask?  Well, according to the laws our government put on these programs, the NSA can only spy on people or groups if they're related to a foreign power or government.  But that's not what the NSA has been doing, the NSA has been spying on THE ENTIRE AMERICAN PUBLIC!  So unless every American citizen is planning to destroy America, or unless every one of us is working with a foreign government, what they're doing is illegal!!!  Maybe if you would put your XBox controller down once in awhile and PAY ATTENTION to what is actually happening in the world, you would know this!  So instead of insulting people on topics you know very little about, perhaps you should educate yourself!  Read a book!  Do some research!  Look at the facts on both sides of a situation instead of immediately agreeing with the side you THINK is right!

mike9876543210
mike9876543210

@deathstream @Qmajinbuu @COPMAN221ISBACK wow, you do know that google, yahoo, steam and many many other companies are working with the nsa, the biggest one being google as google chrome is the most popular web browser so i dont think the nsa will be pushing microsoft to make new ways to spy on people (not that they were really doing this in the first place) because their getting far more information from google. also the always-on drm was created to force people to buy live gold, nothing to do with the nsa.

mike9876543210
mike9876543210

@shadow6323 @pamant i completely agree with most of the things you said, I personally prefer the play station quite simply because I've been a user of the play station since the ps1, I'm very used to the console, the way they do things and the exclusives e.g. gran turismo, I have been an avid player of the series since the first one. 

There are many things that i don't like about Microsoft e.g. the fact the the xb1 it more expensive in the uk than in the us, the way they initially set up the xb1 ect, but i don't hate them, when you thing about it, 90% of the Microsoft's haters are using a computer that runs windows, without Microsoft windows the computer market would be nowhere near as advanced as it is today so for people to hate on them despite being the reason we all have computers that are so easy to use is beyond me.

I've used a kinect and yes the kinect games are suited more towards the younger audience but that doesn't mean its shit, its very useful, i use it all the time when I'm using netflix, you just tell it what program you want to watch and its there without going though a single menu, it's used in games, like telling the ai team mates to take cover, fifa 13 has kinect features that allows you to tell teammates to do shit (i could be wrong as i haven't played it as i hate fifa)etc, so it is changing the way we play games.

Getting the kinect with the xb1 also means developers can implement kinect features more often as everyone will have a kinect, unlike the 360 version where devs couldn't implement kinect features that change game play characteristics because few people had one which meant people who didnt have one will have a shitter gameplay experience

shadow6323
shadow6323

@pamant If I were to buy a phone that came with a camera, I would not even think of suggesting that my phone come without it to lower the cost.. at that point it would be clear that that isn't the phone for me.

I'm a fan of gaming in general. I'll be getting both Xbox and Ps4, I'm getting the xbox first though, as I've always enjoyed Microsoft and xbox a little more than sony personally.

Why is MSoft changing some of their policies and whatnot to please their customers and fans a bad thing anyway? This I don't understand. Normally people complain their voices go unheard, and now people complain when they are heard.

I have no problems with xbox coming with the Kinect, and I have no problems with how it (or the xbox) functions. People that do need to take off the tin foil hats and realise that if someone wanted to spy on them they could probably get much more information about their lives based on what they post on their social media sites, their mobile devices, their built in webcams, etc. Cameras are already all around you. In your homes, in your places of work etc.

I also have no problems with Microsoft (or any other company for that matter) taking something and trying to improve on it. Rift looks cool, but I'm sure with other minds trying to crack something like that there could be even more improvements. That's like complaining someone already did mobile phones, and trying to compare apple to Samsung. Two different companies working on a similar product, they come out and have pros and cons different from each other.

Just my thoughts on all the bandwagon hate MSoft is getting (which I don't understand)

ggregd
ggregd

@pamant I'm no fan of Microsoft and what they're trying to do next generation, and setting aside the spycam accusations against Kinect 2 on XB1, I don't get why some people are so opposed to having it included with the system.  Just because one thinks there were no good games for the 360 version and it's just a gimmick doesn't mean it will be be true forever. Does having a Kinect in your house demote you to casual gamer status? 

This technology connected to Kinect is potentially seriously cool VR/AR that goes beyond anything Occulus or Google Glass does now.