NASA Jet Propulsion Laboratory

UX Research & Development: VR for Telerobotics 

For around 6 months I was an intern at NASA Jet Propulsion Laboratory’s OpsLab, assisting with UX research around uses for consumer gaming and VR technologies like early Oculus DK1 and Xbox Kinect for telerobotics and space exploration. My role included shadowing and interviewing mission planners and scientists, helping design and run experiments, and prototyping different ways of visualizing and interacting with data from rover missions.



 

One of our major projects was about improving control systems for telerobotics at interplanetary distances. The speed of light prohibits direct control; it can take up to 24 minutes for a signal from Earth to reach rovers on Mars, meaning up to a 48-minute round trip for an operator to get any feedback on their actions.

We explored interfaces that displayed a ‘cone of uncertainty’ representing where a rover might end up given a certain command, and tested its effectiveness with a cheap robot and in-room obstacle course driven on a delay of just a few minutes.






We also explored using game like interfaces to control fine manipulation tasks at a distance, including under time delay. The flashiest of these was an experiment using a structured light sensor (xbox kinect) to translate gestures directly, VR goggles to help the user with depth perception, and a similar predictive interface as the rover exploration. A more conventional screen and mouse configuration offered more precision. We tested our ideas using a Kinova JACO robotic arm (an assistive device for mobility issues) to perform fine motor tasks like stacking lego blocks.







Finally, we used VR to visualize point cloud data from the Curiosity Rover, as an alternative to the panoramic views and conventional 3D viewer available in ArcGIS. We did user studies with a mix real mission planners and scientists as well as laypeople, asking them to draw maps of the martian surface based on what they saw in the headset, and again based on conventional panoramic views. The VR experience afforded significant improvement for laypeople, less so for the scientists who already used the panoramic software on a day-to-day basis.