Mixed Reality Design Labs

May 2017 – Present, Ecosystem Experience Design | Analog | Microsoft

Mixed Reality Design Labs(MRDL) is an open-source project by the HoloLens design team. We share experimental samples, explorations, and learning from spatial interaction design.


To make the mixed reality platform succeed, it is crucial to bring developers and creators into our platform. To build great mixed reality experiences, developers need great tools and resources.

Windows Mixed Reality devices

Personally, I have some developer background, before going back to design school and change my career as a designer. Because of that, I have been creating many apps on iOS before joining Microsoft and also brought some apps to Windows using Windows Bridge technology.

When I joined the HoloLens design team, I was excited about the new canvas and wanted to learn how to build apps for HoloLens. I started learning Unity and created several HoloLens apps. As I go through the journey, I realized that there are a lot of developers and designers who are experiencing similar pain in creating common mixed reality interaction patterns and user interfaces.

For example, a lot of developers were having trouble making a simple button work with proper visual feedback. Everyone wanted to make a beautiful button that looks like the one in HoloLens shell. However, the only button UI control that they could get was a 2D button from Unity. For the developers and designers coming from the 2D world, new concepts and variables such as collider, rigid body, shader, material, and z-depth were big challenges. I realized that we really need common interaction and UI building blocks.

Common UI controls

We started researching crucial common UI controls that are mostly used in existing apps and requested by developers from the community.

The spectrum of the common UI controls
Foundational components and UI controls
Layers of the common controls

When we build the UI controls, we wanted to make sure to make them modular and extensible so that developers can easily customize and bring their own brand identity into their app experiences.

MRDL’s Holographic Button made with modular components

Sample apps

To demonstrate the use of MRDL’s common UI controls and interaction building blocks, we created sample apps with a complete end-to-end experience.

Periodic Table for HoloLens

Periodic Table

This app demonstrates the use of Interactable Objects and Object Collections. It shows a good example of Microsoft’s Fluent design elements such as spatial typographic layout using depth, motion, and materials.

Lunar Module

Example of using two-handed gesture and manipulation. The app shows how to provide UX flow for spatial mapping room scanning.

When we publish sample apps, we also wanted to inspire developers and designers by showing practical examples of what they can build with them. This example shows how the periodic table sample app could be used to build other experiences such as a stock market app or sports data visualization.

Application examples


Responses from the community

With published open-source UI controls and sample apps, developers and creators started sharing their own creations and blog posts. We found great innovative use of the sample apps being connected with Microsoft Cognitive Services.

Periodic Table app with Bing Speech API’s translation service

Migration to MRTK

With positive responses and strong requests from the community, we ported MRDL’s UX building blocks to MRTK v1 (Mixed Reality Toolkit).