2022 – 2023, HoloLens & Mixed Reality Design Team | Microsoft
Overview
The Mixed Reality Design Language (MRDL) is a visual and interaction system for spatial computing at Microsoft. It represents the evolved successor to the spatial UI patterns originally introduced with HoloLens 2, expanding the earlier Fluent-based design system into a more sophisticated, depth-aware spatial framework that can support complex productivity use cases such as virtual meetings, shared documents, multi-window spatial layouts, and collaboration scenarios.
MRDL was first showcased in Microsoft Mesh, where virtual presence, shared environments, and productivity tools required a higher information density and behavioral fidelity than previous MR applications. It was later formalized and made broadly available to developers through MRTK3 UX components, giving Unity developers access to the same spatial UI system used in Microsoft’s own products.
Context
Before transitioning into a manager role, I served as Principal Designer (IC) contributing to the MRTK UX building blocks, which were used to construct all UI inside Microsoft Mesh. Much of MRDL’s foundation was informed by learnings from HoloLens 2-era deployments, where the need for more structured spatial hierarchy, clearer affordances, and deeper material cues became clear — especially in enterprise workflows. When remote work and virtual meeting scenarios emerged as core product requirements, MRDL evolved intentionally to support productivity-grade spatial computing, rather than lightweight demo-style experiences.
Role
As Principal Designer (IC)
- Contributed to spatial UI component behaviors and interaction principles in MRTK
- Worked with Mesh and research teams on early MRDL explorations
- Supported translation of spatial UX rationale into reusable Unity components
As Principal Design Manager (5-month leadership window)
I led a 12-member Design Language team responsible for:
- Visual design system
- Common UI controls and UX patterns
- Material, shader, lighting, and depth systems
- Audio design
- End-to-end design integration across Mesh
Team composition included:
- Visual designer
- Interaction designer
- Technical artists and shader specialists
- Prototyping engineer
- Audio designer
- (plus partner engineers and PM collaborators across the org)
Leadership responsibilities included:
Establishing new collaboration rituals and alignment across:
- Interaction Design
- 3D Art and Avatars teams
- User Research
- Guiding design-to-engineering pipelines and reviews to ensure UI shipped with intended fidelity and adhered to the spatial design language
- Drove cross-team alignment between design, Mesh engineering, and MRTK engineering to deliver a scalable UI tech stack, unlocking theming, rapid iteration, and adaptable layouts for future spatial products.
- Externalizing the components for other Microsoft Mixed Reality products and public through MRTK
Design Goals
- Evolve the HoloLens 2 spatial design language into a more robust, depth-expressive system
- Support productivity-first mixed reality workflows such as virtual meetings, co-presence, shared screens, multi-window layouts, and tool palettes
- Expand the system to support 2D surface interaction and mouse-based input, enabling continuity with Microsoft Teams and desktop productivity use cases
- Improve spatial clarity and affordance through light, material, elevation, and contextual audio cues
- Ensure both internal product teams and third-party Unity developers can build full, production-grade MR applications rather than demo-only experiences
Improvements
Updated geometry with rounded corners for more approachable and friendly experiences
Improved visual feedback for enhanced multi-modal input such as eye-gaze combined with pinch gesture
Improved modular units that support various UI scenarios with permutations
Modular backplate system for building complex layouts that remain clear and usable
Redesigned bounding box visuals to reduce visual noise and enable fluid gaze-powered interactions
Integration with RectTransform and Unity Canvas that supports dynamic and flexible layout
Key Elements
Geometry & Form
Volumetric shapes with tactile thickness help users understand objects in space and perceive depth.
Color & Hierarchy
Colors tuned for additive displays ensure legibility in varying light conditions.
Material, Light & Audio
Materials and audio cues communicate state, selection, hover, and activation — critical in hands-free interaction.
Layering & Elevation
3D stacking creates hierarchy and expressiveness without clutter.
Iconography
Updated Fluent icons and tuned scales improve information density and clarity within spatial UIs.
Typography
Refined type ramp for better structure and hierarchy in the content.
Tooling & Integration
MRTK3-Unity Foundation
All UI components inside Microsoft Mesh were built on MRTK3-Unity, leveraging its established input and interaction system for spatial experiences.
Ensured that MRDL was not just a philosophy, but a tooling-based system:
- MRDL rules → translated into usable MRTK controls
- Visual language → converted into shaders, materials, audio events
- Patterns → delivered through prefabs, UX components, and Unity examples
This empowered both Microsoft Mesh product team and external developers to use MRDL by default rather than recreate UI from scratch.
Microsoft Mesh Integration
Mesh for HoloLens 2
Experience optimized for HoloLens 2 spatial input, including direct touch, hand ray interaction, and optional eye-gaze input.
Mesh for Microsoft Teams
Updated color and branding approach to ensure visual clarity and usability when surfaced on desktop through Microsoft Teams.
Outcome
MRDL became the bridge connecting:
Mesh (product) → MRTK3 (tooling) → developer ecosystem (Unity)
Through IC-level UX foundations and manager-level cross-team leadership, the system raised spatial UI consistency and quality across Microsoft Mesh and Mixed Reality stack.
Microsoft Mesh entered preview in early 2023 and officially launched in January 2024, introducing immersive avatar-based collaboration within Microsoft Teams. The platform enables hybrid-work participation from both desktop (via Teams) and Quest devices.
Figma Toolkit for MRTK3
Subset of MRDL components were published as Figma Toolkit for MRTK3 on Figma Community
