Mixed Reality Toolkit (MRTK) v2 for HoloLens 2
Apr 2018 – Present, Mixed Reality | Microsoft
Mixed Reality Toolkit-Unity is an open-source project which provides a set of foundational components and features to accelerate Mixed Reality app development in Unity. MRTK provides building blocks for crucial spatial interactions and UI. The latest release of MRTK v2 supports HoloLens 2’s new Hand Tracking and Eye Tracking input.
I led the overall design of the MRTK’s UX building blocks. It includes the design of interaction behavior and UI controls, prefabs, example scenes, assets, and documentation. In this process, I closely worked with the engineering team to integrate visual and interaction details that match the HoloLens system’s UX so that customers can achieve the same user experience quality easily and quickly.
In addition to creating and providing shell-parity UX building blocks, I made sure the building blocks are extensible and customizable so that customers can infuse their own brand identity. I closely worked with the engineering team and program managers to prioritize UX items, implement building blocks, and triage bugs based on internal/external customer’s needs.
Through various types of projects with HoloLens partner companies including Philips, Bosch, Ford, Lowes, and NASA, I helped the team aggregate and extract common UX patterns and turn them into MRTK’s building blocks. This process involved identifying the common patterns for spatial UX design challenges, prototyping, working with user research studies, and publishing on GitHub.
UX Building Blocks & Example Scenes
- Led the design of MRTK’s UX building blocks
- Helped the team to validate hand tracking APIs by building prototypes
- Updated existing UI controls and interactions to support near interaction with hand tracking & far interaction with hand-ray
- Created various example scenes to accelerate the design and development of the HoloLens 2 applications
- Worked with the engineering team to ensure MRTK’s building blocks are extensible, modular and people can easily customize and infuse their own brand identity
- Created test scenes for the near interaction user research studies
- Polished the visual and interaction design of the controls
- Submitted & reviewed Pull Requests on Visual Studio Online and GitHub
- Aggregated common UX patterns from HoloLens projects in various types of industries
- Extract crucial common UX patterns that customers need, prototype, build, and publish based on the customers’ needs and requests
- Created technical documentation with visual diagrams
- Created design guidelines on Mixed Reality Docs
Branding & Promotion Content
- Created videos with Mixed Reality Capture to promote new releases
- Created MRTK brand guidelines and assets for coherent communication
- Engaged with the developer community through multiple channels including GitHub, Slack, and Mixed Reality Academy
- Prepared MRTK’s UX Building Blocks deep dive deck and presented at multiple AR/VR conferences including Microsoft Build, Unity Unite, and Global XR Bootcamp
- Worked with multiple partner companies and helped them to update HoloLens v1 apps to support new interactions
Created example scenes where developers and creators can experience, play, and learn about HoloLens 2’s new hand tracking input interactions. All building blocks still support HoloLens v1’s GGV(Gaze/Gesture/Voice) input and Windows Mixed Reality immersive headset’s motion controller input.
Technical writing for MRTK’s building blocks and functionalities. Writing on Design/Dev guidelines. Design of the document layout and modules. Visual graphical elements such as diagrams, Mixed Reality Capture images, videos, etc.
Created MRTK branding guidelines and assets for coherent communication.
MRTK branding guidelines PDF file
Created and presented HoloLens 2 & MRTK’s UX Building Blocks deep dive content in multiple conferences and AR/VR/MR events.
See Talks page for more information.
Wrote and published articles on medium to share Mixed Reality app design & development story using MRTK.
- Bringing the Periodic Table of the Elements app to HoloLens 2 with MRTK v2
- MRTK 101: How to use Mixed Reality Toolkit Unity for Crucial Spatial Interactions
- Designing Type In Space for HoloLens 2
- MRTK 유니티 사용법: 홀로렌즈, 혼합현실, 증강현실, 가상현실 앱 개발을 위한 필수 인터랙션 UX 컴포넌트
Shared MRTK updates on social media through visual content – Mixed Reality Capture videos, screens, examples, and sample apps.
MRTK v2 (Release Candidate 1) has been released on 4/5/2019.
MRTK v2 (Official Release) has been released on 8/21/2019.
MRTK Examples Hub
Created and released MRTK Examples Hub which allows developers to easily experience various example scenes in MRTK.