Why leading questions are killing your UX
In the noble art of getting customer feedback, you might be creating dire consequences for your product.
Let’s play a game.
We’ll call it ‘Spot the problem’.
Malcom, a Product lead, is speaking to one of his customers’ Lisa.
They’re discussing Lisa’s experience with a new onboarding journey that’s being tested for their digital product.
Malcom is keen to get feedback to iterate on the journey and asks Lisa about a few screens.
Malcolm asks questions like:
‘Did you like the journey?’
’Was the options screen confusing?’
’Did you see the link to get help?’
Lisa, naturally answered with things like:
‘Yeah, it was nice and easy.’
’A little bit yeah, I had to think about it’
’Help? No, I didn’t see that actually, I didn’t need it much’
You might be thinking, everything feels normal, it’s just a conversation between a keen builder and their customer, what could possibly be wrong?
The answer is about the line of questioning.
There are 2 problems that already exist or will start to now exist as a result of the way Lisa has been asked questions.
Leading questions - Malcolm has enquired with Lisa in a very leading way. By framing his questions with emotive terms like ‘confusing’ or pinpointing out certain actions like help links - he is driving the narrative of response in terms of what he understands, not what the customer understands. This is a massive problem because the response you get now is going to be in line with what Malcom, the builder, wants to hear.
Misleading feedback - Malcolm is now going to inevitably take this ‘feedback’ to his team and say ‘Hey everyone, the customers are confused by onboarding, and they can’t see the help link’. Superlatives that really aren’t true, and will go on to get iterated on by the Product, Design & Engineering team and create marked improvements for customer experiences.
While it would be lovely to hear what we always want with glowing recommendations, the reason we speak to customers is to know what they understand, with as little bias as possible.
The more leading the question, more the bias, the less accurate customers’ responses, and the less accurate feedback is, which means the lower the likelihood for your iterations to solve their problems.
And thus you create this loop:
Here’s what you should be doing instead.
Ask open-ended and outcome-driven questions as much as possible. There are two main benefits to this:
Less bias - by asking about the outcome they need to get to - like ‘How would you get help?’ you are letting the customer think about all the ways they could do that, thus bringing out their mental models and a more natural response, rather than being pointed towards ‘a help link’.
Deeper understanding - by asking open-ended questions, you avoid a scenario of Yes and No to summarise a whole question. Imagine if you’re on a date and you want it to go well. You wouldn’t want short answers, you’d want stories, you’d want meaning and you’d want depth. We want to do the same here, so we get a deeper understanding of why they may be confused, or what they think would be better, rather than just ‘Yes I’m confused’ and ‘No this isn’t better’.
By asking more open-ended questions, we will find more insightful answers from our customers.
Here are a few examples and their open-ended equivalents:
Did you see the help link? → Where/how would you get help?
Did you notice the go-back button? → How would you go back to the previous screen?
Were you confused by the steps? → What did you feel when you saw these steps?
Did you see the estimated time for the journey? → How long did you expect this to take?
These are all examples that help you uncover more detail from your customers and truly improve your UX to serve them.
If you need any advice on interviewing users and leading questions - drop me an email with the subject line ‘Interviewing Users’.
Next week, stay tuned for why you’re wasting time with iterations by missing out on key stakeholders!