LibreHack 2022
In this case study, we explore how we could leverage Azure Spatial Anchors during a 48 hour hackathon in order to provide additional value to industrial workers through navigation and training.
Before starting the prototyping process, I brainstormed with the team members, all of whom were developers and was the first time we collaborated, to discuss how we could best utilize spatial anchors to create a navigation and training system that would benefit our clients' workforce. We discussed various ideas and potential solutions, and evaluated their feasibility and effectiveness in improving safety, productivity, and training.
Spatial Anchors for Navigation
The first challenge we decide to tackle was to provide workers in an industrial setting with a way to navigate through a large, complex facility. We needed to make sure that workers could quickly and easily find their way to the equipment they needed, while also help solve problems.
To quickly prototype our navigation and training system during the hackathon, we used Splice to create a 3D arrow model. Next, I used Adobe Aero on a mobile device to quickly prototype the system.
I imported the 3D arrow model into Adobe Aero, and then placed it in the physical environment using the device's camera. This allowed me to quickly test the placement of the arrow in the real world, and make adjustments as needed.
We added an interface using Adobe Premiere and utilized a voice to text service for quick instruction input. This allowed us to create a proof of concept and gather feedback for further development.
Navigation Demo
Since the Hackaton was remote, I created a concept video in my home that allowed us to show the functionality in simple steps. The helper character was meant to dynamically replace a loading screen by using animation and sounds. While this was shown on mobile, the functionality would also work on tablets and HoloLens.
I was also tasked with creating a prototype for a more efficient way for industrial workers to learn about specific components of a complex product, such as a carburator. Traditional training methods were time-consuming and expensive, and often failed to provide workers with a deep understanding of the product.
Training using holograms
To address this challenge, I decided to leverage the power of Azure Spatial Anchors to create a holographic training system. By using holograms, workers could interact with a virtual representation of the product, allowing them to gain a deeper understanding of its components and how they work togetherTake a minute to write an introduction that is short, sweet, and to the point.
Training Demo
Similarly to the previous demo, we used the existing mobile interface and generated the audio guide using text to speech. We also learned that in order to facilitate the creation of these assets our clients could use a photogrammetry app such as Polycam that scans objects into 3D items.
Outcome
We won the competition, and the development team also got a basic working version of Azure Spatial Anchors within the HoloLens. These concepts were going to continue to be explored as valuable avenues for the business.
Some of the things that I would have liked to add more of if I had additional time was the ability to turn captions on and off, as well as showing more instructions and options over the existing interface.