What’s the role of UX in mixed reality?

Augmented reality (AR) and virtual reality (VR) have spent the last several years in the spotlight, with big players like Apple, Sony and Google all getting onboard.

IDC forecast worldwide spending on AR and VR to reach nearly $20.4 billion (£15.5 billion) in 2019, with commercial sectors representing over 64% of that spending. So, what are their potential applications?

What is virtual reality (VR)?

VR is a computer-generated simulation or recreation of a real-life environment that is a fully immersive experience. VR often uses headsets, like the Oculus Rift or Google Cardboard, to give users 360 degree experience of the simulated reality or block out the ‘real world’ altogether.

What is augmented reality (AR)?

AR layers computer-generated enhancements on top of our existing reality to add more meaning to the environment around us. AR is typically used with smartphones or tablets, using the camera to view and display the real-world on the device’s screen, then adding text or images on top of that view.

AR and VR in the Tourism industry

One of the more promising areas for AR and VR applications has been in the travel, leisure and days out industry. Helping users get the most out of attractions and destinations, we have seen AR and VR being used to bring to life the unique physical locations and artefacts central to the tourist experience.

Local Projects is one organisation paving the way, introducing AR to change the way visitors interact with pieces at the Cleveland Museum of Art. Their ArtLens app enables users to curate their own museum tour as well as browse the tours of others. Through AR, users can use their mobile phone to scan artworks across the museum, and engage with the pieces on a much deeper and more interactive level through superimposed photos, audio and video perspective onto the artwork.

Nomensa’s work with the National Trust used VR to breakdown accessibility barriers for people with impairments, in particular dementia. We spoke to carers and patients about the calming effect of walking in gardens, and of how exploring historic landmarks caused memories to resurface and prompted lucid moments.

So, we created a VR solution emulating the experience of exploring a National Trust property in a realistic way, designed around user needs and behaviour, to make the National Trust’s offering more inclusive and accessible to all. This project meant that people with dementia could still delight in the properties, even if they were unable to leave their care homes. You can read more about this in our case study.

Alder Hey children’s hospital has taken this concept of enhancing a physical location, beyond supporting tourism and has applied AR in a truly innovative way. Their app, Alder Play, lets their very young guests experience the hospital facilities through the lens of a smartphone or tablet – layering atop a playful animated tour guide, challenges and rewards to help them settle in.

AR and VR in the Retail industry

Several companies are beginning to offer their customers new ways to look through and interact with their product catalogues. IKEA’s Place app is leveraging AR to simplify buying furniture by letting users use their phone or tablet to see how furniture would look in their home or office. Retail giant, Amazon, is similarly trialling AR for larger products, as well as for jewellery and watches. They feel AR can help reduce the need for users to physically measure items and cut down the likelihood of returning them.

Although to a lesser extent, VR is also getting in on the action in the retail market. Both Audi and Cadillac are introducing VR to their car showroom experience, letting customers explore all their different car models in-depth, anywhere.

AR and VR in customer services and support

For industries that have technicians working out in the field, like telecoms or utility providers, AR and VR can give them instant access to work instructions or guidance, allowing them to diagnose and resolve unique or complex problems, all onsite. This can substantially reduce the time and number of experts needed in the field to resolve a service issue.

Similarly, AR is being used to help customers self-support. Our recent work with Virgin Media Ireland saw us design an AR interaction that would act as a visual step-by-step guide to self-install broadband. This empowered customers, led to less pressure on call centres and you can read all about it in our case study.

AR and VR in enterprise/professional services

Both VR and AR can be powerful tools for helping companies simulate situations that can be difficult, costly or dangerous to recreate. Lloyds Banking Group is among several companies using VR as part of their hiring practices. They do so to better understand the capabilities of potential recruits when faced with challenging systems issues or failures for their digital and IT programmes.

VR is also being used to streamline and cut costs on service R&D and innovation. In the case of FedEx, who were looking to improve the design and functionality of their drop box points, VR became a means to rapidly test and iterate prototypes without the expensive and labour intensive process of building realistic prototypes.

Image credit: Owen Lord, UX Designer at Nomensa

Likewise AR and VR alike are becoming an increasingly popular tool to help employee training and up-skilling. Companies, like Airbus, are using VR to help train their engineers. VR gives them the opportunity to work with life-sized representations of aircraft engines and efficiently connect trainees and experts. AR has been particularly beneficial for training and implementing safety practices. Through using a tablet or smartglasses (like Google Glass) guidance and step-by-step instructions in line with safety procedures can be layered on top of the task at hand.

What can AR and VR bring to your digital experience?

We are already seeing these technologies shatter problems that crop up across sectors. But this is just the beginning.

1. Accessibility and engagement: AR/VR delivers a richness of contextualised information about physical places and objects that individuals engage with. By providing elements of historical context and story-telling it has the potential to open new routes into education, culture and learning. It empowers people to self-support or self-guide through processes that used to be perceived as inaccessible or too complex. This has the potential to strip back presumptions and stigmas that may have once block people from those experiences.
2. Geographical barriers: Whether it’s getting the chance to experience a place of natural beauty you never thought possible, check out that new limited edition Tesla, or work with an expert in your field halfway around the world, VR has the potential to open up new market audiences and lines of communication in ways previously unimaginable.
3. Online purchasing: In e-commerce, one of the core blockers to to clicking buy is the often tactile experience of shopping – the need to “see” an item before you commit. Two of the main reasons potential customer give for not following through with an online purchase is that there was not enough detailed or visual descriptions of an item, and the feeling that having to return an unwanted or not-as-expected product would be too much hassle. Considering how simply providing a video clip of a product in use has been shown to substantially increase online sales across diverse retail platforms, imagine AR/VR giving customers the chance to directly interact with those products.
4. Reduce expenses: In terms of training, AR and VR can provide employees with hands-on experiences in a safe environment, and has been shown to reduce training time by up to 40%. From a customer support perspective, AR installation guides can substantially cut call time to support contact centres. With many other possible applications, having the capability to easily and cheaply create specific visually rich and interactive situations in real-time has the potential to revolutionise internal and external business processes.

The role of UX in AR and VR

The cost on the consumer for VR and AR equipment has been a traditional hindrance for wider adoption. But costs have been steadily decreasing, with AR fairing better by leveraging already pervasive devices such as smartphones and iPads. Conversely, companies themselves have voiced concerns about their technical capability to design, develop, release and maintain AR or VR experiences. Fortunately, we are seeing more and more software development support specifically for this, with the likes of Apple’s ‘ARKit’ and Oculus’ ‘Oculus for Business’ tools.

However, in a recent survey by Jabil, companies felt it was innovation in the creation of content being delivered to users via AR/VR as being the bigger caveat to wider uptake, rather than access to the technology itself (e.g., hardware, devices, software, etc.). Innovation in content creation hinges on delivering excellent storytelling, usability, design and development centred around the users you want to reach. Which is why utilising the combined expertise of designers, UXers and copywriters and working in the power of three is so pertinent.

UX has a vital role to play here in supporting organisations to successfully adopt AR and VR as part of their wider digital experience. As we change the way our customers or users interact with our digital offerings, there are a couple of considerations to keep in mind.

How will people interact with AR and VR?

We now have the chance to expand beyond the bounds of a mouse or touchscreen to interact with digital experiences, such as with voice, behaviours and gestures. With more options to play around with, content can bring more scope for confusion and poor understanding around how users can move through an experience. Exploring, understanding and choosing intuitive, user-goal driven, behaviours become more important.

How can we support individuals’ journey through the experience?

Giving users cues and feedback also becomes even more important in AR/VR. Where we may have helped users understand what options they have to explore content by visually signposting or highlighting text as a button or a drop-down menu, with AR and VR there is a lot that exists ‘off screen’. Understanding where and when additional cues, like visual, auditory or haptic feedback, are necessary, can help users explore, understand and find the content they need.

What context will the AR/VR experience be used in?

Both AR and VR pull the physical (or simulated) reality into our digital experience in a very new and different way, so users’ environment is a much larger consideration compared to traditional digital experiences. For example, an individual using AR/VR to overlay instructional content on a physical object for repair vs. using it for exploration a historical site, will have very different needs when it comes to interacting and exploring digital content. A deeper understanding in not only in how, but also where, your AR/VR experience is used will play a big role in user interface and interaction design choices.

One thing that hasn’t changed is that getting it right for your users opens up game changing possibilities for your digital experiences.