NASA controls robots with Kinect and Oculus Rift

Kinect and Oculus Rift aren't just for games. The Human Interfaces division of NASA's Jet Propulsion Lab in Pasadena, California has published a video showing off how Kinect and Oculus Rift could come together to control a robotic arm.

"Using the new Xbox One Kinect sensor, we are able to manipulate the JACO robot arm in real time. By combining position tracking from the Kinect and rotational tracking with the Oculus, we provide a first-person view for the operator," NASA said.

"Future work will include integrating sensor array data into the scene and translating our research to the Robonaut 2 humanoid on the International Space Station," NASA added.

In the video, a NASA engineer is seen directing a robotic arm to pick up a block, which he appears to be able to accomplish without issue.

NASA said robotic spacecraft will play a "critical role" in the future of space exploration. Efforts like the new Kinect/Oculus control system will increase the efficiency of such robotic endeavors, which in turn will allow NASA to explore the solar system faster and more extensively, the group said.

One issue NASA is facing right now is the time delay between the operator and the spacecraft. But with advances to robot-controlled systems, these issues could be remedied, NASA said.

"Traditional control systems become unpredictable and unresponsive in the presence of time delay," NASA said. "Our system decreases perceived time delay by allowing the user to control the spacecraft at a predicted future state. By predicting and displaying a range of expected future states, safe operation is possible within the modeled uncertainty of the spacecrafts' behavior. "