Framing UX research questions

This short video provides advice about user research to avoid incorrect results, bad decisions and ultimately a sub-optimal product. Framing UX Research Questions

Hi, this is Mike from Nomensa In this short video, I would like to get across a very simple message about user research, but one that is still often forgotten. Failure to acknowledge it can lead to incorrect results, bad decisions and ultimately a sub-optimal product. User centred design should always involve some sort of qualitative feedback from users. This may come from interviews, testing, diary studies, or other methods, and each has pros and cons. In any case, you must take care how you frame your questions and be aware how this can affect the answers you receive.

So let’s imagine I am asking you to give me some feedback on a new homepage design. I have many options for framing this, but I’ll focus on 3 here:

Option 1: Comparing to current or older versions of the homepage. I could show you the old homepage and ask you what you think; then show you the new one and ask how it compares. It is common to want to frame the question this way, because this is what stakeholders want to know – is it better than what we had before? But there are problems. Because participants understand that a newer version should be better, they may be predisposed to a positive opinion about it. They also know that a lot of work has gone into the design, and will not want to criticise it. For example you might say ‘yes, this is a very clean, modern design, and I can get going with my task straight away’. We must recognise that this is a comparison, not an absolute measure. There is also an assumption that because it is new, the tool will be better at finding places to visit than the old site.

Option 2: Comparing to similar sites. I could show you a site with a similar proposition, and then ask you how it compares to this one. This is also useful to know. However, there is another important drawback to both option 1 and option 2. Whenever we give an opinion, we are expected to justify it. In user research, people know that they will be asked ‘why?’ Before they even decide on an answer, they will be mentally preparing the answer to the ‘why’ question. For example here, you might say ‘yes, I like this one’ while preparing that ‘why’ answer which might be that it is a simpler page. Again we must recognise it is a comparison, and also realise that it is one of the more obvious justifications. It might be the case that the page needs to be further simplified, or that there is something else about the design which is more important for usability, but we would never know if we took this feedback at face value.

Option 3: Comparing against expectations. I could start with nothing; describe the purpose of the site and say: “What sort of things are you expecting?” Then show you the site, and ask if it meets those expectations. This is generally a fairer way to assess a user’s opinion. Framing questions this way gets people thinking about their needs and their goals before you show them anything. This is more like ‘user centred design thinking’ and more likely to lead to accurate results.

Of course, using a mixture of questions will give you better results, but when writing those ‘opinion’ type questions, consider these two points:

1. Is it clear what the participant should be comparing against?
2. Is it equally easy to justify any response given?

This is a crucial first step to getting accurate results from user research.

Thanks.