Mixed Reality Design Labs

May 2017 – Present, Ecosystem Experience Design | Analog | Microsoft

Mixed Reality Design Labs(MRDL) is an open-source project by HoloLens design team. We share experimental samples, explorations and learning from spatial interaction design.

Background

To make the mixed reality platform succeed, it is crucial to bring developers and creators into our platform. To build great mixed reality experiences, developers need great tools and resources.

Windows Mixed Reality devices

Personally, I have some developer background, before going back to design school and change my career as a designer. Because of that I have been creating many apps on iOS before joining Microsoft and also brought some apps to Windows using Windows Bridge technology.

When I joined HoloLens design team, I was excited about the new canvas and wanted to learn how to build apps for HoloLens. I started learning Unity and created several HoloLens apps. As I go through the journey, I realized that there are a lot of the developers and designers who are experiencing similar pain points and looking for the solutions.

For example, a lot of developers were having trouble making a simple button work with proper visual feedback. Everyone wanted to make a beautiful button that looks like HoloLens shell. However, the only button UI control that they could get was a 2D button from Unity. For the developers and designers coming from the 2D world, new concepts and variables such as collider, rigid body, shader, material, and z-depth were big challenges. I realized that we didn’t have great common UI building blocks.

Common UI controls

We started researching crucial common UI controls that are mostly used in existing apps and requested by developers from the community.

Spectrum of the common UI controls
Foundational components and UI controls
Layers of the common controls

When we build the UI controls, we wanted to make sure to make it modular and extensible so that developers can easily customize and bring their own brand identity in their app experiences.

MRDL’s Holographic Button made with modular components

Sample apps

To demonstrate the use of MRDL’s common UI controls and interaction building blocks, we created sample apps with complete end-to-end experience.

Periodic Table for HoloLens

Periodic Table

This app demonstrates the use of Interactable Objects and Object Collections. It shows good example of Microsoft’s Fluent design elements such as spatial typographic layout using depth, motion, and materials.

Lunar Module

Example of using two-handed gesture and manipulation. The app shows how to provide UX flow for spatial mapping room scanning.

When we publish sample apps, we also wanted to inspire developers and designers by showing the practical examples of what they can build with them. This example shows how the periodic table sample app could be used to build other experiences such as a stock market app or sports data visualization.

Application examples

Responses from the community

With published open-source UI controls and sample apps, developers and creators started sharing their own creation and blog posts. We found great innovative use of the sample apps being connected with Microsoft Cognitive Services.

Periodic Table app with Bing Speech API’s translation service