Confusing User Feedback.

Creating context in order to validate ideas.

Here’s the situation — you’ve gotten lots of feedback on your product and a few users recommend adding a particular feature. You’re then pushed to create that feature because real users have asked for it. But you’re unsure about the usefulness of it and whether it would add any value to the product.

What do you do?

Personally, this happens to me all time. For all products, we set up many feedback channels and get tons of recommendations without any situational context. Being pushed to create solutions for unknown problems is hard and tricky because not all user ideas are equal.

So how can you validate that an idea has merit and should be pursued?

  1. Make a problem statement.
    If someone has given you a recommendation, ask them to define the problem and give as much context as possible around it. Have the person who brought this issue to you answer these questions:

What is this solving?
Why does it need to be solved?
Who has this problem?
When does this problem occur?
Is there already a solution in the product for this?
If not, how are users solving this problem?

Accurately defining the problem and setting the context gives you enough detail to be able to further validate the feedback.

Locate the problem in the work flow
It’s very important to be able to put the problem in context of the system. Define the step-by-step tasks that gets the user to the issue and what happens once they are there. Do they abandon the process? Is there a workaround people create?

There may be a natural behavior occurring that could be the model for a solution. It’s also good to know how the addition of a solution would affect the current user flow.

Get analytic about it.
Analytics provide great data on lots of issues. Do you have completion rates of the task? Do you know how many users encounter the problem? Is there enough data here to back up the initial claim?

Talk to other users.
Just because you have feedback saying something is a problem doesn’t mean it is an issue for the majority of your users. It’s very possible that this only occurs for 2% of your users and most users never even have an issue. This doesn’t mean that if only 2% of users have a problem then it’s not important —it could turn out that it is a major pain point.
Also, don’t ask users directly if something is a problem. You probably already have enough responses of this nature. Instead, focus on having people complete tasks and watch what they do. If possible, quantify your results by showing the importance of the task, the level of user effort, barriers, and failure rates.

Prioritize.
Often the decision for new work is a big one so getting the perspectives from your team is very helpful in understanding the amount of work involved and the impact it would have on other priorities. With the product team/owners, ask these questions:
How does this affect the user flow or the impact the product?
How much development work would be involved?
Can you quantify the improvement in metrics?
When does this need to be done?

Design, Prototype, Test.
This is pretty straightforward. Design a solution, prototype it, and test it with users. If the tests are a success, then you can feel good about proceeding with development.

Also, if possible, test in development before you push to production. Even the best prototype can’t simulate everything and things usually change (even if just a little) in the actual product.