Running remote user research for NHS England and NHS Improvement

Over the course of a project, things don’t always go exactly as planned. But as a recent Discovery project with NHS England and NHS Improvement demonstrated, that doesn’t have to be a bad thing. If you can react quickly and use these changes to your advantage, it can really add to the overall success of a project.

Over the last few months, we’ve been exploring the potential creation of a new intranet for NHS England and NHS Improvement. This is a large organisation, with thousands of employees, so from the outset it was vital that we try to get as wide a range of feedback from these end users as possible. As this would be a new product, we knew we had to begin with Discovery.

What is Discovery?

Whether you are building a completely new service or product, or updating or replacing an existing one, Discovery is a smart way to start. A key factor in meeting GDS guidelines and standards, Discovery allows you and your stakeholders to understand and contextualise the problem that needs to be solved, before committing to costly development. It helps you to:

• Learn about the users (both external and internal) and their needs and goals
• Identify constraints from technology, business or organisational processes or policy and legislation
• Map a picture of the service in context and look for ways to improve or think differently
• Understand the intent – what is the service there to achieve?

Discovery allows a team to take time to ask questions, interrogate assumptions, investigate options and develop multiple hypotheses for potential solutions. From here, we can begin to build a picture of our path forward to Alpha, Beta and beyond.

Running remote user research

Discovery is usually where issues within a system emerge. But on top of the organisational tangles we often encounter, we also had to grapple with a new beast: conducting user research in a lockdown world. Fortunately, we already conduct a great deal of user research remotely, so we were on our way.

As nice as it may be to speak to everyone within the organisation, it is just not possible. There was a limit to the time and resources that we had available to us. It was important that we designed a methodology that would fit within these constraints, while still allowing us to speak to enough users, and gather enough quality insights to be able to confidently give recommendations for the future.

Initially we settled on the following plan:

1. An organisation-wide survey – This would allow us to quickly gather a wide range of qualitative and quantitative data from a large number of users, who we may not otherwise have reached.
2. Individual interviews – In-depth, one-on-one interviews would give us time to really dive into the details. While there was a limit on how many we could possibly do, we felt that with the right selection of users from across the business, we could get an accurate representation of the current intranet situation and what people really wanted and needed.
3. Engagement days – Two in person engagement days would have us going to a couple of the offices to speak to as many people as we could. With posters, questions and as many post-it notes as we could carry, we would set up in a high traffic area and grab anyone walking by. This format works great as people can give a quick comment on the way to a meeting, or if they have a bit more time, they can stay for a longer chat.

So, we created the survey, picked the offices to visit for our engagement days and put together a list of potential interviewees. The challenge was to speak to a wide enough selection of users to get a good view from across the organisation, while restricting to a manageable number based on our time and resources.

Balancing time and insight

Partway through this early planning, a new stakeholder entered the project, and they expressed some initial concerns. There were a lot more people they believed we needed to speak to for this discovery to be a success. So, more names got added and the list grew longer. It became a balancing act between speaking to even more users and having time to analyse the data and complete the project on time.

As so often happens, there was a bit of a mix-up. We ended up with some of the one-on-one interviews that we had planned for, but also a number of larger group sessions. In the end we were getting about double the number of participants than we had set out to do. The majority of these were going to be facilitated remotely in groups.

There were some concerns for us at first as this was a big change to our methodology. We had planned around having one-on-one, in-depth discussions that could last up to an hour at a time. Hosting group sessions, some with up to seven participants, would involve us taking a completely different approach.

Turning challenges into opportunities

Rather than going back, fixing this error and getting the original one-on-one sessions booked in, we realised that we could use this to our advantage. It was the perfect opportunity for us to speak to an even wider range of users. And if done correctly, it would allow us to gather even more valuable insight from users around the organisation.

We also noticed there was feeling around the organisation that there’s often too much focus on the offices we had selected for our engagement days. The other regions and offices often feel ignored. Mind you, the offices we had selected were the largest, so we were pretty much guaranteed the largest pool of users to speak to. And as much as we may have liked to give everyone a chance, there was no way two practitioners could get around over 50 offices spread across England.

Despite the strong reasoning behind selecting the locations that we had, it was important that these concerns weren’t ignored. A project like this relies on everyone being on board. For this Discovery to be successful, we knew we must give everyone the chance to have their say from the outset. For this reason, while we could not facilitate engagement days for these other offices in person, the decision was made to host a virtual version of these sessions.

Virtual engagement days

Remote user research isn’t a new concept to us; interviews, group discussions and workshops were all activities we have experience of carrying out successfully remotely. A remote session like this, however, was quite a novel idea. We were opening the session up to a lot of participants, and as such we had to be able to facilitate in a way that would work whether we had five people or fifty.

It became clear that we couldn’t just carry this out over a video conference. Managing such a large amount of participants talking over one and other was not going to be an easy task. We needed a way to let a large number of people interact without the need to use voice chat.

The session ended up being a real success. While it took a while to get used to presenting without any verbal participation from anyone else, the participants that turned up were engaged and gave some really positive feedback about the session. We found it helps to narrate the answers as they come in rather than have dead air as they provide their answers.

Learning from Discovery

With the project now wrapping up, I can look back and view it as a success. There were challenges, things didn’t quite go according to plan and we had to work hard to ensure all stakeholders were onboard. But overall, through the range of research methods we carried out, we were able to put together a strong overview of the current landscape as well as recommendations for the future direction of the organisations intranet.

In addition to turning potential issues into good opportunities, I think another key takeaway from this project is that with some creative thinking we can take these in-person activities and successfully move them online. And as we move into a period where our work needs to be done remotely, and more workshops and research methods are going online, this is more important than ever.

If you’d like to learn more about how Nomensa’s user research expertise can reveal compelling insights about your organisation, get in touch on hello@nomensa.com