Immersive Web Emulator "Point At" Feature

An experiment to provide feedback on the Immersive Web Emulator v1.3.0, released on June 2023.

If you are not familiar with it, the Immersive Web Emulator developed by Meta Reality Labs is a crucial and extremely useful tool for testing and iterating on WebXR experiences without a physical XR device.

After giving it a try, I could appreciate some of the great key features in v1.3.0, such as:

• Accurate simulation of controller input, including binary input (buttons) and analog input (trigger, grip, joysticks)
• Accurate simulation of hand input, including built-in hand poses and pinch gesture control
• Keyboard action mapping for added efficiency
• Keyboard input relay

And since the tool is still in development, it's reasonable to believe that it will get better and better.

With all this said and from my personal experience ATTOW (June 2023), the Immersive Web Emulator still suffers from some weak points that impact the UX, though.

Problem

Currently, developers face a somewhat complex task each time they need to test for intersection and interaction with the objects in the WebXR scenes they are building because they have to manually adjust the VR controller's position and orientation in the 3D viewport to point at the different objects.

Therefore, due to the inherent limitations of working on a 2D screen, transform manipulations in the interactive 3D viewport often require multiple adjustments, slowing down the process and potentially causing frustration.

This time-consuming process poses a UX challenge, deterring the intuitive and user-friendly interface Meta Reality Labs strives for with the Immersive Web Emulator.

Solution

There's an evident need for a more efficient way to simulate the VR controllers' transform manipulation, and this brought me to a possible solution to the problem: the "Point At" feature.

With this feature, developers can intersect and interact with the different objects in the WebXR scene in no time and with minimal effort: instead of manually adjusting the VR controller's position and orientation in the 3D viewport, developers can type the object's ID selector in a dedicated input field and hit a button for the software to apply the required transform manipulation automatically.

The Making Of

Rough Prototype

To create the "Point At" feature prototype I used:

• A combination of HTML, JavaScript, and CSS to build a simple web page that includes some info, a basic UI, and the required logic to interact with the embedded WebXR scene
• The A-Frame framework to create the WebXR scene and show how the VR controllers (both in the scene the developer is building and in the Immersive Web Emulator's 3D viewport) would behave when using the "Point At" feature to test for intersection and interaction

NOTE: WebXR frameworks/game engines like A-Frame, Three.js, Babylon.js, and PlayCanvas work differently. Therefore the primary goal of this prototype is to show the "Point At" feature, not to tell how to implement it.

Hi-Fi Mockup

To create the Hi-Fi mockup I simply edited in Photoshop a screenshot of the Immersive Web Emulator's UI v1.3.0.