Imagine a future where the way you control virtual or augmented reality (VR/AR) isn't with a device in your hand, but by touching your own skin. This idea is becoming a reality with a new technology called EgoTouch, developed by researchers at Carnegie Mellon University. EgoTouch uses artificial intelligence (AI) to turn your skin into a touch-sensitive controller.
AI helps the AR/VR headset recognize when and how you touch your skin. Before EgoTouch, there was OmniTouch, which also aimed to let users control interfaces by touching their skin, but it needed a special camera that was big and cumbersome.
EgoTouch, however, uses the cameras that are already part of AR/VR headsets. Here's how it works: when you touch your skin, it creates little shadows and slight changes in how your skin looks. These changes are what the AI model, trained by the researchers, looks for to detect touch. They collected data using a special sensor on the finger and palm, teaching the AI to understand different touches, like light or hard pressing, without needing to label each touch manually.
Simple hand controls for VR/AR
This system can detect touch with over 96% accuracy, meaning it gets it right almost every time. It can tell if you're pressing down, lifting off, or dragging your finger across your skin, and it's smart enough to differentiate between a light touch and a firm one, which could be used like a right-click on a computer mouse.
EgoTouch is simple and broadly applicable. It works well across different skin types and conditions, though it's less effective over bony parts like knuckles because the skin there doesn't deform much when touched. The researchers are also looking into making EgoTouch work in the dark by using night vision technology.
With EgoTouch, the future of interacting with VR/AR might just be at our fingertips, quite literally, as we might soon control virtual environments simply by touching our own skin.