Pharma brings complicated material to life with augmented and virtual reality.

Imagine you are standing in your exhibit booth on a very crowded scientific and educational meeting floor, armed with fascinating, yet difficult-to-understand medical content that needs to make its way to the many passing physicians in order to educate them on the science your company is working on. Amid the masses of other presenters, how do you catch and hold an HCP’s attention as he or she streams by?

Bayer Pharmaceuticals, for one, has found a solution, as have several other pharmaceutical companies by using augmented reality (AR) and virtual reality (VR) capabilities to provide experiential educational content.

According to Dr. Suneil Malhotra, director, head of scientific communications and medical education, oncology, at Bayer Pharmaceuticals, before AR and VR, exhibit booths were restricted by regulatory standards and passive, low-interaction medical content that hampered their ability to engage HCPs in the booth for educational and scientific exchange.

“We wanted to bring good, interesting scientific education to life," Dr. Malhotra says. “After all, we have a very compelling pipeline."

So Bayer partnered with Pixacore, a sales, marketing, and medical communication agency to invigorate the company’s scientific content through implementing AR and VR digital strategies for the purpose of educating on oncology pipeline pathways in an immersive, innovative environment.

The first time Bayer used AR at a medical meeting, Bayer saw its engagement per booth attendance soar from under two minutes to up to 10 minutes on average. This certainly solidifies the theory that cutting-edge AR and VR technology increases consumer buy-in and drives the educational value of the scientific content.

“We knew we were on to something; for physicians to spend eight or 10 minutes going through multiple pathways at a booth is unheard of," Dr. Malhotra says.
“Who spends eight minutes in a medical congress booth, especially when they’re not getting anything else but education — no tchotchkes, not even a cup of coffee?"
It is the content interactivity, relative ease of use, and the wealth of overlaid and/or integrated information of AR and VR that allows for more meaningful disease education, because the user’s own extraordinary personal experience is the focus, says Dan Chichester, chief experience officer, Ogilvy Health.

“For example, in our Innovation Lab, we’re using AR to display atopic dermatitis flare ups on a HCP’s own hand for a two-fold effect," he says. “One, AR increases empathy — your skin vs. another person’s; and two, the ability to visually peel away skin layers actualizes the need for treatments that go beyond topical creams and get to the problem source."

Bayer uses Magic Leap, a head-mounted virtual retinal display that superimposes 3D computer-generated imagery over real-world objects to bring information to life. However, this method only works in some situations.

“Augmented reality with the right content and with the right use case can be very engaging," says Sanjiv Mody, CEO and founder of Pixacore, which partnered with Bayer. “Augmented reality is not a replacement for a traditional presentation but AR certainly adds dimension and interactivity to current methods."

If it is not possible to present the material as dynamically in other mediums then it is time to explore VR/AR for its capabilities, he says.

Both VR and AR are compelling ways to provide an engaging experience, but AR is easier for people to access since it doesn’t require any special hardware like a VR headset.

“The experience with AR is so fantastic," say Tim Murphy, VP of commercial, Veeva. “Depending on the use case, it can really generate a type of empathy for the HCP because he or she can actually see what is happening in their own space in reality, and it’s really exciting to see."

It can be difficult to get someone to try out VR, however, and sometimes people can have an uncomfortable or negative reaction to the life-like motion. Augmented reality gives a less immersive but similar experience, and there is a much lower barrier to entry as a consumer.

Cognizant has been using AR technology at clinical congresses for several years, says Bryan Hill, chief technology officer, life sciences, Cognizant. One particular example involved an 80-foot artery that HCPs could access through an iPad and “walk" through the experience and literally were part of the mechanism of action as they hit different hotspots and different types of experiences.

As a follow up, directly after an AR or VR experience, it is easy to capture the knowledge assessment of the HCP through a quick questionnaire, which allows reps to follow up with that HCP with targeted information on what the HCP knows or doesn’t know about the product.

Mr. Hill quotes Ben Franklin when he sums up the value of using AR or VR: “Tell me and I forget, teach me and I remember, involve me and I learn."

“When we talk about the value proposition for this set of technology, the ‘involve me and I learn’ phrase is key," Mr. Hill says. “Done well, these immersive technologies can drive understanding and retention by giving HCPs or patients the opportunity to experience science or other concepts in a very complex field."

AR and VR can both easily employ adult learning principles to make the experience engaging and to improve retention.

“All of these aspects of AR/VR not only lead to higher retention, but a more fun and memorable experience," Mr. Mody says.

“AR is transformational in terms of learning, decision-making, and in how we interact within our physical environments," says Angelo Stracquatanio, CEO and co-founder of Apprentice. “AR can vastly improve how life-sciences organizations meet customer needs, train employees, record data, design, and test products. Pharma-specific AR removes the physical restrictions experienced in the lab and manufacturing suites caused by manual data recordation, referencing SOPs, or waiting for on-site vendor assistance, while improving audit readiness."

Use Cases of AR and VR in Pharma

It’s been a number of years since there’s been any innovation in e-detailing, but AR is hitting its stride in terms of being an effective and accessible tool.

AR will enhance customer engagement by creating highly engaging experiences for HCPs and patients. This is crucial today, Veeva experts say, as HCPs expectations increase for digital engagement. Finding new ways to interact with them can serve as a true differentiator and help foster more personal relationships. Using AR to spur emotional reactions such as wonder or surprise can also create a better connection with customers, and help ensure HCPs are fully engaged.

Specifically, AR enables life-sciences companies to innovate how they tell and deliver their product’s value, as well as demonstrate the outcomes. These demonstrations are extremely powerful, particularly when unpacking complex medical concepts because customers can visualize the product in their actual environment.

“Over the last 10 years, despite e-detailing providing greater personalization, insights, and integration with CRM platforms, both the number of physicians who are receptive to e-detailing — 44% as of 2018 — and the average length of e-detail — 90 seconds —— has been decreasing rapidly," says Ric Cavieres, executive VP for markets and consulting at OZ. “This is where AR-detailing comes in; AR-detailing applications transform the physician experience, providing a compelling interactive experience, that drives both awareness and education of conditions and treatments, driving a far longer and more impactful detail."

Sales representatives and medical science liaisons are expected to benefit from leveraging AR for detailing physicians. Research has shown AR is more effective in demonstrating spatial and temporal concepts, allowing people to engage in the learning process with multiple senses and producing better results. AR can be used to teach medical practitioners complex medical procedures — not just in theory, but also in practice — allowing them to virtually touch and manipulate objects to see the effect or practice the procedure. AR can also provide 3D on-body visualizations of how medicines or medical devices work for both HCPs and patients.

“AR-detailing enables pharma sales representatives and MSLs to have a much more immersive and educational detailing experience with physicians on treatments as well as educating them on the actual health conditions and diagnostics," Mr. Cavieres says. “This is particularly important for specialized medicine treatments where physicians may not have the latest background on the condition, indication, or treatment."

In addition, pharma should also be developing a patient AR application for a physician to provide during a consultation that supplies an educational interactive environment on conditions and treatments.

Apprentice’s pharma clients are using AR as part of their manufacturing process across a broad spectrum — from R&D laboratories to clinical trials and biologics manufacturing, to fill finish and even packaging line clearance/change over.
Pharma AR systems can bridge the paper-to-digital divide, which facilitates easy adoption and transforms unit operations from process development, through upstream and downstream operations, to commercial production.

Mr. Stracquatanio says when AR is deployed at a pharma facility, there needs to be a clear paper-to-digital path to encourage quicker adoption. Unlike paper, AR-enhanced systems equip users with hands-free data, 3D technique demonstrations and a network of recorded tribal knowledge, which reduces deviations and preemptively eliminates unnecessary cost and time expenditures. All data is fully traceable, improving audit readiness and also providing further insight into human processes.

“A clear paper to digital path provides for easier adoption resulting in streamlined operations," he says. “Learning and decision-making are dramatically improved, which is a real game changer for our complex industry when combined with an easy to adopt roadmap."

Operational AR is crucial in preventing human error and production delays that could cause million-dollar losses.

“AR is the most promising digital technology that is quickly transforming life-sciences manufacturing," Mr. Stracquatanio says. “It’s imperative that pharmaceutical manufacturers find innovative ways to increase reliability, quality, and output to keep up with manufacturing trends and market demand."

AR can also be used to illustrate drug MOA, as AR can bring the action to life in ways 3D models can’t. Demonstrating the progression of a disease state and how that disease reacts to treatment along a timeline is also a really effective use of AR, Mr. Murray says. These are two common uses of AR in the industry today.

Experts at Veeva say AR can provide an important point of differentiation for life-sciences companies. Marketers will be able to use AR to stand out in a world saturated with content.

With AR, companies can add value to the customer – helping them visualize the product, providing empathic experiences or creating context for the product.

Because AR blends the real world with the virtual world, it empowers brand teams and content partners to build rich, immersive new experiences. This helps the field force deliver their messaging using a new visual way to communicate product benefits, and offers a new avenue for marketers to engage in storytelling for their brand.

Beyond the complex science and immersive aspect of the pharmaceutical space, medical device companies also have a lot of interest in the augmented reality world.

“For example, a 200-pound gene sequencing or diagnostic piece of equipment can’t be lugged around for demonstration, but with AR that equipment can be placed virtually inside the doctor’s office and the model can be explored in real time, opening doors, tapping on buttons, doing many things that were practically impossible before augmented reality," Mr. Mody says. “What used to be shown through a brochure, video, or PowerPoint slide can now appear in the real world using AR."

Challenges to Adoption

Experts describe the industry’s uptake of AR and VR as “dipping toes," and the reason may be the additional cost, but also a lack of understanding of the technology’s capabilities.

According to Dr. Malhotra, the budget for implementing novel digital techniques can certainly be a barrier.

“Cost is why I think companies are still toe-dipping because they’re not completely investing,’ he says. “Bayer encourages a lot of innovation and risk taking and we had a couple of years where we committed a pretty substantial budget to the technology, and now we’re reaping the rewards."

According to Mr. Hill, another reason the industry has been relatively slow to adopt AR and VR is because companies don’t understand the many ways the technology can be used.

“People are only seeing AR being used in places like clinical congresses to draw attention to a booth, so one of the biggest challenges is a real understanding of how to apply the technology in a meaningful way that is not just about the next shiny thing," he says.

According to Dr. Malhotra, companies will need an incremental strategy laid out over a minimum of five years with a budget to match to have successful deployment. Bayer started out by introducing AR in exhibit booths at meetings, but the company has also implemented strategies to tie the booth in with its pipeline website to ensure a full end-to-end experience for our users.

“We set up AR first in the booth and then made sure we could deliver a similar experience on the website and now we’re expanding AR throughout the field," Dr. Malhotra says. “It’s a five-year strategy and commitment. Companies shouldn’t just think in a one-year increment; there has to be a long-term strategy and commitment to reap the rewards."

According to Mr. Cavieres, there are three key challenges with AR/VR for the pharma industry: innovation inertia, regulated content, and user training.

“I believe the slowness to adopt AR/VR is largely because people and companies do not understand the fast-moving AR/VR technology and still view it as a gaming environment, and conversely experts in AR/VR often do not understand the pharma business processes," he says. “To innovate in this area takes a deep understanding of both."

Secondly, developing content in 3D is more involved than 2D and ensuring the materials conform to all regulations is an added layer of complexity.

Finally, end users — whether they be sales reps or clinicians — need to be trained on these new platforms with an eye to take their work to the next level. This means a whole rethink of training content and the art of the possible.

“None of these challenges are in anyway insurmountable, and in some way explain the slow adoption of AR/VR in pharma compared with other industries, such as retail," Mr. Cavieres says.

With the arrival of pharma-specific AR and recent hardware improvements, the only remaining challenge for some organizations is simply going beyond comfort that comes from maintaining the status quo, Mr. Stracquatanio says.

However, this is a competitive industry and the status quo will no longer allow life-sciences manufacturers to thrive.

“Today, the potential benefits of AR are more obvious and tangible, resulting in fewer organizations hesitating to explore and deploy augmented reality systems," he says.

Whether it’s virtual or augmented, the reality is that companies should strive to create learning environments that empower better health.

“VR and AR are tremendous opportunities for data visualization," Mr. Chichester says. “The chance to see facts and figures in ways we never could before means patients and HCPs can understand health and treatment in a new light."

In one example, Mr. Chichester says Ogilvy Health translated autonomous longitudinal patient data into VR experiential learning. “The ability to literally walk into the data revealed trends and patterns that visually showed how many people dropped off treatment during various phases of a specific study — an Aha! moment that would never happen in reality," he says.

What the Future Holds

Although AR and VR will always co-exist, the accessibility of AR will keep it in the forefront. Everybody carries a device in their pocket or their bags that are augmented-reality enabled, such as iPads, iPhones, and Android devices.

These will be the gateway to getting people accustomed to using augmented reality. In the future, AR devices will become more affordable, consumer-ready, and head-mounted, like big glasses, for example.

“In three years time, augmented reality will be massively consumed," Mr. Mody says. “Today, investments run 40% for VR and 60% for AR. Five years from now, 80% to 90% of investments will be AR-related."

Brendan Gallagher, chief connected health officer at Publicis, says although AR/VR/mixed reality took a back seat to AI at SXSW this year, they were still prominently featured in many sessions and pop-ups.

“Jessica Brillhart, the former head of VR at Google and founder of Vrai Pictures, gave a fantastic convergence keynote on the challenges of designing and building immersive experiences," Mr. Gallagher recalls. “Beyond the well-documented problem of hardware distribution, things like file sizes, how we approach the concept of framelines versus worlds, and viewer versus visitor, are deeper conceptual chasms to cross than we may have initially expected in creating these experiences."

The true touchstone for AR/VR will be 5G, Mr. Gallagher says. 5G will deliver a 10X decrease in latency — the time lag between a request and a response, which is critical to truly immersive experiences.

According to a recent Forbes article, a reliable 5G network will help VR and AR applications evolve to the next level. Some even say the future of immersive is reliant on 5G. Both VR and AR require a cheaper, more substantial network with lower latency and more consistency if they are to continue to evolve.(PV)

~~~~~~~~~~~~~~~~~~~~~~~~~

Novartis Uses VR in Labs

According to the Novartis blog “Stories From Our Labs," investigators are adapting VR tools for use in the drug design process. Novartis jumped on the opportunity of VR the instant the technology became available on the consumer market in 2016, according to the blog. However, appropriate software had to be developed first to use with the hardware; software that could translate information about a protein’s structure — the location of every atom in the molecule and the shape of its machinery — into a 3D virtual view.

With this software, the Novartis team is using VR headsets to immerse themselves in the protein and literally take a walk around.

Back in 2017, Viktor Hornak, an investigator in the global discovery chemistry department at the Novartis Institutes for BioMedical Research (NIBR), said: “We spend a lot of time looking at structures and trying to understand how they interact with our drugs. In VR, I’m the size of the drug and I’m looking at the protein interactions around me. You can’t do that within the constraints of a flat computer screen. A 3D view is more natural."

These molecular walks are useful during the early stages of drug design, when chemists and biologists need to exchange a lot of ideas about a protein’s interactions with a potential small molecule drug. These conversations can be tricky because the researchers come from different scientific backgrounds and don’t always speak the same language.

With the prototype VR system, scientists can walk through a chemical structure together.

Seeing the same things at the same time helps them more rapidly settle on the best ideas for tweaking a drug’s chemical makeup so that it fits more easily into a protein’s machinery and more effectively alters the protein’s function to slow or stop a disease process.

By developing an in-house prototype VR system for drug discovery, Mr. Hornak and his colleagues are able to demonstrate the potential value of VR to developers of existing 2D molecular visualization tools.

“We would prefer to be leaders in the process and help shape its future than to wait," he says.

Ultimately, VR could have a very wide reach, especially if developers rally behind it and begin writing VR applications the way they write apps for the mobile phone.
“This is a new way of doing things, not just for science, but for all types of interactions," Mr. Hornak says.

~~~~~~~~~~~~~~~~~~~~~~~~~

VR Builds Empathy Fast

In late 2018, Stanford researchers published results from a study called “Becoming Homeless." The study showed that people who underwent a VR experience of what it’s like to be homeless were not only more empathetic toward those who were homeless, but were significantly more likely to take an action, such as signing a petition, than those exposed to 2D experiences.

The need for empathy in health is obviously huge, but the examples of VR/AR/mixed-reality experiences in the industry are few and far between, and none are really at scale. They are really only one-off tactics, such as the lauded Excedrin Migraine example from three years ago, or the more recent Novartis example that uses VR to aid in drug discovery.

Source: Brendan Gallagher, Publicis

Download PDF