Hero shot of the HoloLens 2 sitting with the INVIVO logo on a screen out of focus behind it.

Sometimes the whole world shifts, and suddenly new shapes and possibilities emerge into our field of view—and the applications possible with the HoloLens 2 for education, training and remote collaboration are firmly in INVIVO’s sights.

As a member of Microsoft’s Mixed Reality Partner program, INVIVO was able to participate in HLS2 pre-launch dev jams hosted at Microsoft last year. Now having several of the HL2 devices in-house (or in-houses, with our WFH-ready team), we’re excited about the possibilities for new ways to communicate science and add value to your business.

Breaking down barriers

In the medical communications industry, delivering information-rich content is easy; the challenge is to tease out core messages, and engage the viewer or user in a way that excites them, and makes it memorable.

Augmented reality (AR) is less immersive than its cousin virtual reality (VR), but it has the potential to be far more practical in training and learning situations.
In AR, digital artifacts are overlaid on our view of reality, either via a tablet screen & camera, or projected into our field of vision by a small screen attached to a headset.
The HoloLens 2 is a mixed reality (MR) device. MR is commonly used as an umbrella term encompassing AR & VR, but more accurately describes the mixed interplay between digital and physical environment.

“I’m interested in utilizing MR’s potential for breaking down barriers,” says Gary Ng, Team Lead, Senior Interactive Developer, “the barriers between the physical world and the digital world, letting the average person just use it in an intuitive way.”

“As the world deals with the current COVID-19 pandemic, new barriers have emerged that impact the way medical and scientific information is shared.  This situation has created opportunities to change the ways we communicate and collaborate by delivering dynamic and information-rich visuals on interactive and immersive platforms” suggests Kevin Millar, SVP of Creative & Medical Science. “Imagine onboarding a team of Medical Science Liaisons on the mechanism of action for a new therapeutic compound, but instead of showing a traditional animation, you allow them to reach out and manipulate molecules to show their role in disease progression. We can do that now.”

Or consider working on the manufacturing floor of a pharmaceutical company, where a repair is needed on a large piece of machinery. Instead of flying in an expert technician, an MR tutorial could guide an in-house engineer through the trouble-shooting checkpoints. A visual overlay on the actual equipment using the new Dynamics 365 Guides  shifts from one piece to another, until the faulty component is fixed efficiently and remotely.

Kevin notes, “We’ve seen this type of guided tour in video games for decades now, but overlaying this type of information onto the real world to solve problems on-site is now within our capabilities. There’s real value for biopharmaceutical and medical device companies to visualize their work using MR in order to problem-solve.”

An INVIVO designer manipulates the coronavirus module in the air in front of him.

Mapping the way forward

So how would a futuristic headset help solve a real-world problem?

With the HoloLens 2 use of spatial anchors.

“The use of spatial anchors – whether with a phone, tablet, or the HoloLens – will allow us to map interactivity with virtual objects into the environment in really exciting new ways,” explains Dylan Ravka, an Interactive Media Developer, “This allows for seamless interaction with virtual objects, because they become grounded in spaces we’re already familiar with. I’m impressed with the improvements made from the first version.”

The HoloLens 2 can attach virtual spatial anchors on real objects in a room, so the interactive elements will interact with the room that way permanently. It maps where everything is located, which as you can imagine, could let you know when something is out of place.

The potential ROI goes far beyond mechanical repair.

Two people with HoloLens 1 manipulating molecules in the air in front of them.

Intuitive and surprising experiences add value

The use of high-quality 3D models by INVIVO’s medical animation team can be quite surprising the first time you try the HoloLens.

Developed for HoloLens 1, we’ve created a mechanism of disease scavenger hunt game that can be played in any room, teaching participants about the actions and morphology of different molecules. It’s like being able to see something that was always there but was hidden from you. You leave the room and return, put on the quite-comfortable HoloLens 2 and boom – there are the molecules again.

Perhaps the HoloLens 2’s most significant advance comes from the use of a wider array of gesture controls. The first version required either a hand held clicker device, or for the participant to make an awkward pinching gesture with their finger and thumb to grab virtual objects; the HoloLens 2 gives developers a wider vocabulary of hand gestures to make manipulating visual objects feel natural and intuitive.

“Instead of having to explain to new users about the specific ‘air tap’ gesture of the first HoloLens, now you can just literally reach out and grab the hologram, and interact with it,” says Dylan, “it makes onboarding so much easier than before, especially in a conference or remote training setting. It feels futuristic to someone new to the technology.”

Gary agrees, “Whether they only have 5 minutes or 20 minutes with it, HoloLens 2 experiences will have a lasting impact for the user, beyond a website or even an interactive app.”

Animated gif of someone wearing a HoloLens 1 and gesturing to bring a small hologram of a person to life.

For HCPs, for patients, for people

“AR is persuasive, and a more personal form of technology,” suggests Gary, “You can literally take it anywhere with you. The phenomenon of people going into parks and outdoor spaces they may not have visited otherwise was fuelled by Pokémon Go, and in a way that’s like a sneak peek of what will be possible with MR in the next 10 years. Imagine how your relationship with the environment will change, in a medical setting, in a patient experience.”

Since the beginning of the pandemic, “remote & telehealth treatment has jumped with over 90% treating patients remotely”. As telemedicine becomes the new norm for many, the HoloLens could certainly be a powerful new tool to facilitate and enhance remote medical training in order to meet this new demand.

“Mixed Reality is here to stay, and INVIVO is excited to continue to help our clients integrate this new technology into their initiatives to increase engagement and improve efficiency,” suggests James Hackett, VP of Creative Innovation.

“The addition of the HoloLens 2 to our toolbox means that we can continue to provide our Clients with the most innovative, effective solutions while realizing our own aspiration – to better the health of patients worldwide. Interactive mixed reality allows patients to participate in their care, and visualize their conditions and treatments in completely new ways – resulting in better understanding and better relationships between patients and caregivers. For medical practitioners, the increased understanding that MR storytelling provides gives new confidence in treatments as they’re able to understand the benefits more clearly.”

The potential benefits of using an MR model have even been estimated to have a 10-fold gain in efficiency when in an R&D setting, which could  save time and cost in the drug development process.  Using this technology any R&D team could manipulate a high resolution, accurate, 3-dimensional model of a biological molecule to fully explore potential binding sites for a targeted therapy as an additional step in the long path towards drug discovery.

Overhead image of the HL2 and case.The world has changed and we’re here to help

Our team – from medical animators and medical writers, to User Experience designers and Unity developers – are ready and bursting with ideas for the HoloLens 2. Indeed, we’ve been ramping up for years now.

Expect to see & experience the new reality of medical science communication; interactive training; guided repair; and remote collaboration soon, from INVIVO.

Interested in exploring a project with us? Reach out at info@invivo.com