Background

Augmented reality (AR) has the transformative potential to seamlessly integrate digital information into the physical world, creating new opportunities for visualizing and interpreting 3D spatial scientific data. However, optical see-through head-mounted displays (OST-HMDs) pose significant challenges for accurate visualization due to the blending of virtual content with the real-world environment. This blending can lead to inaccurate perception of rendered pixels, which is particularly problematic for scientific visualization. Traditional approaches have attempted to address these issues by focusing on separate objectives such as intuitive placement, visibility enhancement, and color correction of virtual objects. Volume rendering, which maps data attributes to optical properties like color and opacity using transfer functions (TFs), faces additional challenges in AR due to the difficulty in maintaining perceptual mapping and discernibility against real-world backgrounds. Existing solutions often fail to consider the coupled problem of placement and color constraints or do not adequately address the specific needs of volume visualization, resulting in suboptimal user experiences and potential inaccuracies in data interpretation.

Technology

VoxAR is a method designed to enhance the visualization of volume-rendered objects in optical see-through head-mounted displays (OST-HMDs). It addresses the challenge of rendered pixels blending with real-world colors, which can hinder accurate perception of augmented information. The system operates in two primary steps: first, it determines the optimal placement of virtual objects in the user's field of view by evaluating spatial and environmental objectives using a GPU shader language. This ensures that the placement is managed according to user preferences and predefined constraints. Second, VoxAR adjusts the colors of the transfer function (TF) used for volume rendering based on the real-world background. This adjustment is achieved through a novel optimization method that ensures the rendered pixels are distinguishable from the background while maintaining the perceptual mapping between colors and data intensity values. The system is implemented in Unity3D for the Microsoft Hololens2 and includes a real-time solution for evaluating and optimizing the placement and visualization of virtual objects. The effectiveness of VoxAR is validated through both objective evaluations and subjective user studies, demonstrating significant improvements in the user's ability to perceive and analyze volume-rendered visualizations in augmented reality.

Advantages

Optimal Placement of Virtual Objects - Color Discernibility Optimization - Real-Time Performance - Preservation of Perceptual Mapping - Support for Changing Field of View (FoV) - Enhanced User Performance - Compatibility with Existing AR Systems

Application

Scientific Visualization - Medical Visualization - Situated-AR Visualizations - Educational and Training Applications - Collaborative Data Analysis