Researchers Demonstrate Body Tracking via Modded VR Controllers in Meta Quest 2
A research team has showcased an experimental body tracking technique that uses the cameras on the virtual reality controllers. How does this ‘modding’ work? Let’s have a look.
Conventional virtual reality systems capture head movements as well as the movement of a user’s arms and hands inside virtual reality via internal or external tracking. This is done by cameras and sensors which are either directly attached to the virtual reality headset (internal tracking) or utilize a laser beam or infrared beam network generated by external tracking stations (external tracking).
There are merits and demerits to both of these systems. However, both external and internal tracking have one commonality: the focus is mainly on the movements of the controllers and the Virtual Reality headset. There are, therefore, challenges when it comes to full-body tracking which is just barely realizable or only achievable with accessories.
The research team has developed an approach that allows for body tracking through the cameras located on the Virtual Reality controllers.
Virtual Reality Body Tracking via Controller Cameras
Each of the Virtual Reality controllers has two fisheye cameras with two wide-angle lenses. One of the lenses is situated on the tracking ring and the other sits at the bottom of the grip. It aims to capture as much of the user’s body as possible as they move.
Images captured by the right and left controllers are merged in real-time using the ControllerPose software. There are four images in total so it creates a 185-degree view of the user. The researchers say that the system is also capable of locating user poses with 3.38 inches (8.59cm) accuracy.
The researchers also employed two variants for processing functions. The images from the camera were streamed to a PC. Although this is more cost-effective, it results in higher latencies. The images were also processed directly on the controllers through a Raspberry Pi Compute Module along with Google Coral.
The researchers demonstrated this modded body tracking with virtual reality games that they themselves programmed like a movement game that was much like the VR fitness game OhShape. In this example, the player struck some poses using their arms and legs so as to math some approaching shapes. Another demo was of a field hockey app where a user deflects the puck using their arms and legs.
How Effective is the Fully Body Tracking?
The experimental full-body tracking described above can come in handy when you are training specific body parts. The researchers’ prototype is, however, not yet finetuned and does not run smoothly. The main shortcoming with this method is that virtual reality controllers do not have a sufficient view of the body about 32% of the time in any conventional VR app.
The researchers say that most of the time, the VR controllers are either too close or too far from the body. However, this handicap can be resolved by using a system with tracking cameras both on the controllers and on the virtual reality headset. The images from both gadgets can then be merged for reliable full-body tracking.
Will We See Full-Body Tracking in Meta Quest 2?
Meta has invested considerable resources in researching a suitable body tracking solution. Earlier this year, there were some signs of a possible full-body tracking solution for Meta Quest 2 in the official Quest developer documentation. However, these were dismissed by Meta’s head of VR Andrew Bosworth who explained that the camera positions in the current virtual reality headsets aren’t suited for full-body tracking which captures the legs and feet.
It is, however, likely that Meta’s upcoming high-end VR/AR headset, the Project Cambria, might have controllers with built-in tracking cameras. Meta has recently introduced camera-based body tracking in the Instagram AR effects.
Source: Future Interfaces Group
Related Posts
https://virtualrealitytimes.com/2022/05/05/researchers-demonstrate-body-tracking-via-modded-vr-controllers-in-meta-quest-2/https://www.xrcentral.com/wp-content/uploads/2022/05/researchers-demonstrate-body-tracking-via-modded-vr-controllers-in-meta-quest-2-4.jpghttps://virtualrealitytimes.com/wp-content/uploads/2022/05/Body-Tracking-with-Modded-VR-Controllers-150×90.jpgTechnologyA research team has showcased an experimental body tracking technique that uses the cameras on the virtual reality controllers. How does this ‘modding’ work? Let’s have a look. Conventional virtual reality systems capture head movements as well as the movement of a user’s arms and hands inside virtual reality via…Sam OchanjiSam Ochanjisochanji@yahoo.comAdministratorVirtual Reality Times
https://virtualrealitytimes.com/2022/05/05/researchers-demonstrate-body-tracking-via-modded-vr-controllers-in-meta-quest-2/?utm_source=rss&utm_medium=rss&utm_campaign=researchers-demonstrate-body-tracking-via-modded-vr-controllers-in-meta-quest-2
