Designing an Interactive AR Experience

Welcome to the behind-the-scenes journey of crafting an interactive AR interface for displaying exhibit information. In this post, we’ll explore the intricate process of interface design, object placement with QR code tracking, and intuitive hand gesture controls. Let’s dive in!

Interface Design

Wireframe of Initial Design

The journey began with sketching wireframes to conceptualize the AR interface. Figure 1 showcases the initial layout, featuring virtual components seamlessly integrated with the real-world environment. Interactive buttons and information panels were designed to enrich the user experience, providing detailed insights into exhibited artworks.

Final Interface

Evolving from the wireframe, the final interface introduced subtle yet impactful changes. Button designs were refined for clarity, with intuitive icons guiding users seamlessly through the experience. Additionally, audio control buttons were strategically relocated to enhance accessibility, contributing to a more polished interface design.

QR Code Tracking and Object Placement

QR code tracking emerged as a cornerstone of the AR experience, facilitating precise object placement within the virtual environment. Leveraging Microsoft HoloLens 2’s capabilities, the QR code SDK enabled seamless integration with Unity, empowering developers to harness the power of AR technology.

QR Code Manager

The QR code manager served as the backbone of the tracking system, orchestrating the detection and placement of virtual elements based on QR code coordinates. Through meticulous scripting, developers ensured seamless synchronization between the physical and virtual realms, fostering a truly immersive experience.

QR Code Panel

A user-friendly interface was paramount to guiding users through the QR code tracking process. The QR code panel, equipped with intuitive controls and real-time status updates, empowered users to effortlessly engage with the AR environment, enhancing overall usability.

AR Interactive Components

Buttons

Interactive components, featuring a blend of buttons and slates, formed the heart of the AR experience. MRTK’s versatile UI elements enabled developers to craft immersive information panels, seamlessly integrated with the exhibit environment. From sleek round buttons to dynamic square controls, every element was meticulously designed to elevate user engagement.

Slate

Slate prefab emerged as a versatile tool for crafting immersive information panels, offering unparalleled flexibility and interactivity. Equipped with bound controls, users could manipulate and resize panels with ease, fostering a sense of agency and immersion within the AR environment.

Hand Gesture Control

Hand gesture control added an extra layer of interactivity, enabling users to intuitively interact with virtual elements. Leveraging Microsoft HoloLens 2’s built-in hand tracking feature, developers implemented two distinct input models: point and commit, and direct manipulation.

Point and Commit

The point and commit model empowered users to interact with virtual objects seamlessly, utilizing hand gestures to navigate the AR environment effortlessly. With a simple air-tap gesture, users could engage with virtual elements, fostering a natural and intuitive user experience.

See here for more

Direct Manipulation

Direct manipulation introduced a tactile dimension to the AR experience, enabling users to touch and interact with holograms directly. From one-finger presses to two-finger pinches, users could manipulate virtual objects with precision, enhancing immersion and interactivity.