Facebook has unveiled its mind-reading wrist device and augmented reality keyboard that will allow users to replace the mouse and keyboard in future hardware products.
The company’s Facebook Reality Labs division showed off the prototype technologies during a virtual conversation with members of the media this week. Division executives talked about the technologies while videos of the projects were played to the public.
The wrist device can read neurological signals sent from users’ brains to their hands. It could theoretically read these signals to get an idea of what a user wants to do and replicate the action in a virtual or augmented reality environment.
“You’ve actually devoted more of your brain to controlling your pulse than to any other part of your body, probably twice as many neurons controlling your pulse and the movement of your hands than is actually devoted to your mouth for nourishment and speech, ”said TR Reardon, director of research science at Facebook Reality Labs.
The Facebook researchers demonstrated “forced” actions where a user in real life could pinch their fingers to hold and control virtual, distant objects in augmented reality. The name of the action is a reference to the Star Wars franchise in which certain characters can use the Force to control and move people or objects far from them.
In addition, the company demonstrated electromyography wristbands that users can wear to type on any surface as if they were typing on a physical keyboard. Although there is no keyboard, the EMG wristbands record the intentions of a user’s finger movements and record the letters and words.
Facebook’s development of these technologies is taking place as the company prepares to release its first smart glasses later this year. That device will be Ray-Ban glasses and will be released in collaboration with Luxottica.
Unlike smartphones, which rely on touchscreens, or Oculus virtual reality headsets, which rely on handheld controllers, there is currently no clear input mechanism for smart glasses. That’s why Facebook is working on these projects.
Asked how quickly these technologies can make their way to the public, Facebook CTO Mike Schroepfer said they are in the early stages of development.
“It’s difficult to predict their timeline,” says Schroepfer. “How these things are going in the marketplace, when they appear – are things I don’t have clear answers to. What we’re focusing on is hardening these technologies.”