Tips & Tricks

Have You Mismatched Your User Research?

Generative vs Descriptive vs Evaluative Research

Let’s consider you are building an aircraft maintenance system for airport workers, is it a good idea to test the prototypes with florists. It could be a very fun test to watch, but you will not learn much from it and aside from some possible GIF-worthy anecdotes you will waste everyone’s time. This is the same when and ask random people to test your product, who may be have diametrically opposite needs from your users. Make sure you match your test subjects with your intended users. 

Here’s another situation: Recall a time that a sibling had symptoms of the flu. If you want to understand how they feel, you don’t use an ear thermometer to measure their body temperature and then make a guess based on their body temperature how they must feel. You straight up ask them how they feel. If you suspect that they might have a lung infection, you don’t ask him to rate how likely they are to have a lung infection. You run some tests.

Using an ear thermometer to understand how someone feels is like running a usability test to understand how much future users would pay for your unreleased product. Asking someone to judge if they have a lung infection is like sending out a survey to understand why people abandon their carts online. Different research questions require different methods to get answers, and the process of picking the right methods for your research can mean the difference between a successfully clearing a lung infection or having to undergo surgery because you didn’t test the right thing. 

Generative Research is employed when you seek to uncover the problem and learn insights from your user base. The purpose is to formulate a problem statement and determine if it is a relevant problem to your business. This is where Shadowing methods and ethnographic approaches work well. Your subjects likely will not state the problem explicitly, you have to synthesize it from your observations.

Descriptive Research is when you have identified the problem but aren’t certain if you have a solution that will effectively solve that problem. So if you have those symptoms of a sore throat, do you have to take antibiotics or would some water and a good mouthwash clear up the soreness? You may have some trial and error here to validate the problem and see if the solutions you are thinking of might old weight. The goal is to have a more detailed understanding of the context of the problem.

Evaluative Research is used when you have a specific hypothesis to test, such as a checkout flow, calendar design, or landing page where you need to evaluate (get it?) how well the design is performing. This type of research you should conduct frequently and it even better if you can integrate into your product design workflow. A subset of this type of research may be causal (not casual!) which would test if A caused B. A simplistic example: Would a bright red text indicate a negative bank balance better than a black text with a negative sign in front of it?

You may think that these research methods are sequential and to some degree, when starting from scratch this may hold true, however all along your product development cycle, all of these methods can help you refine your problem to solve and the way you go about solving it (i.e. your product and your designs).

Ned to run some evaluative studies? Give Vempathy a try, your first project is on us!

Previous ArticleNext Article