Jackson Barnes LogoJackson Barnes
AR IxD Prototyping Tool

Case study

AR IxD Prototyping Tool

uiarengineeringBy Jackson Barnes

This IxD prototyping tool enables designers to create and test interactions for AR devices with eye tracking and brain-computer interfaces.

This interaction-design (IxD) prototyping tool let designers create and test AR interactions using gaze, brain-computer interfaces, and mobile touch controls.

The focus was on gaze-based interaction, with phone-based controls providing a complementary input method. The tool supported rapid exploration of mixed-reality experiences that combined eye movements with familiar mobile-device interactions.

Tested Interaction Modalities

  • Eye-to-world interaction — TrueDepth camera: Tracked gaze direction and focal points for interaction with virtual elements in physical space.
  • Neural interaction — NextMind headset: Used neural signals as an input for hands-free control.
  • Mobile interaction — touch display: Provided touch-based controls to complement gaze and neural input when manipulating AR elements.

Interface and Prototype

Concept sketch showing a tablet-based gaze and controller demo, interaction targets, and progress indicators

The concept sketch brought the AR view, interaction modes, and controller feedback into one interface.

AR prototype with a virtual ball and a marked destination over a desk

An interaction task shown in the AR prototype.

Unity editor and C# code alongside onboarding and interaction-mode interface panels

Prototype interfaces and implementation in Unity.