The Playbook on User Research & Testing
You’ve done it! You’ve come up with a great idea that solves a problem markets won’t touch. Now, it’s time to research the bejeezus out of the problem and test your concept. Let the user research and user testing begin!
“We only build features that were manually tested with our users. This enables us to ship features with confidence.”
– Jehron Petty, Founder & CEO, ColorStack
Start With User Research
First things first – talk to your users (also known as customers or beneficiaries). We’ll keep repeating ourselves: the user research never stops. From idea, to product, to continued iteration, your users should drive the product you build, so interviewing your customers at various milestones is critical.
Interview your users with a user-centered lens to allow opportunities to reveal themselves (read: without an agenda). It’s up to you how many interviews to conduct, but it should be enough that you have a broad sample of perspectives and a deep understanding of the problem from the point of view of people who experience it firsthand.
A few things you want to learn through user research:
-
Your future user’s experience with the problem.
-
Analog ways your future user is currently tackling the problem at hand.
-
The types of devices and networks your future user already has access to or frequents.
User Interview Do's
-
Do develop a question list before the interview so you can fall back on it.
-
Do ask open-ended questions that unlock the genuine experience your user has with the problem.
-
Do use the “5 Whys” technique – ask why as many times as you need to get to core truths.
-
Do interview as many users as you need to get a holistic view of the landscape.
User Interview Don'ts
-
Don’t limit your interviews to people you know.
-
Don’t interview your users with the intent to confirm a hypothesis.
-
Don’t ask questions tailored to a preconceived solution or a hypothetical product.
-
Don’t ask leading questions (a question that prompts or encourages a desired answer).
Pro Tip
Ask to Understand
A basic tenant of user research is this: ask open-ended questions about your user’s experience with the problem, as opposed to questions tailored to a preconceived solution you might have. This will unlock genuine insights that will inform the direction of your tech nonprofit.
You Can Never Do Enough User Research
Rey Faustino, Founder and CEO of One Degree, had ample experience with the problem he sought to solve (the accessibility of social services). Even so, when he was researching his idea for a platform that helps low-income families access social services, he set up tables in front of schools and talked to parents to learn more about their concerns around social services.
Once you’ve developed an understanding of the needs of your future user, use this research to build a hypothesis (or answer a question) around what the useful product might be. It’s common for this hypothesis to differ from your original concept! With your hypothesis in hand, it’s time to design a prototype that will inform your MVP – your Minimum Viable Product.
Pro Tip
Embrace the Contradictions
When doing user research, you often hear conflicting advice. Dig into that. Contradictory advice can be a gateway to a solution. You may find that you have to segment your customer base or that the addressable market is not as big as you thought. Conflict surfaces the good stuff and will push you to ask more questions.
Fast Forward’s User Research Process
Fast Forward’s mission is to support and grow the tech nonprofit sector. Fast Forward was recently at a crossroads: deciding what product to build next to increase our impact. So what did we do? We dove into user research to learn about existing behaviors. We first identified the goals of the user research, and then developed a question list.
Main Goal: Determine where tech nonprofits most need support and through which channels they’d like to receive that support.
Sub Goal 1: Understand the biggest challenges tech nonprofit leaders and employees face and where they go to get advice/questions answered around those challenges.
Sub Goal 2: Discover how tech nonprofits are currently engaging with tech employees (volunteers or mentors) and other ways these groups could be useful.
Interview Question List
Intro Questions
-
Tell us about yourself.
-
What stage do you consider your organization?
-
What are your preferred methods of communication with your team? For example: email, Slack, conference calls, in-person meetings.
-
What are your preferred methods of communication with your professional network?
-
Tell us about yourself.
-
What stage do you consider your organization?
-
What are your preferred methods of communication with your team? For example: email, Slack, conference calls, in-person meetings.
-
What are your preferred methods of communication with your professional network?
Sub Goal 1 Questions
-
Tell us about a recent time that you needed help with a task related to your work.
-
What ultimately enabled you to solve the problem at hand?
-
Where do you turn for support if your first attempt at solving a problem fails you?
-
When you have a problem, how do you decide where to go?
-
Where or who would you not feel comfortable turning to for support in the face of a problem?
-
Tell us about a time that an employee came to you with a question that you couldn’t answer. What was the first action you took when guiding the person towards an answer?
Sub Goal 2 Questions
-
Tell us about a time that an external tech employee was helpful to your tech nonprofit.
-
What has previously motivated you to seek support from someone who works in tech?
-
Is there anything that would have made finding a connection with a tech person easier?
-
When was the last time you met someone who was not a direct connection through your personal network?
-
If you needed support from a tech employee tomorrow, where would you go to find one?
-
How do you stay connected to a broader tech nonprofit community?
Pro Tip
How to Write a Hypothesis
[This product] will solve the problem of [problem] through [strategy] because [reason].
Example: Tarjimly will solve the problem of language barriers for refugees and aid workers through crowdsourcing translations because there are enough multilingual individuals to make a dent in this problem.
Definition
Prototype
An early visualization of a product. Prototypes can be analog versions of your vision displayed on screenshots or even paper. Prototypes enable you to validate the design direction of what you think you should build.
Research the Market Landscape
Once you have a deep understanding about the user you want to serve, it’s critical to take time to survey the landscape. This will be helpful – not just in thinking about what you build – but in how you communicate your impact to funders and followers. The interwebs are teeming with helpful studies and analyses that will deepen your knowledge about your issue area.
Before you build, you should be able to answer these important questions:
-
What’s the market size? In other words: how widespread is the problem I want to solve? How many people does it negatively impact today?
-
What will the world look like if I succeed? What’s the potential for positive impact?
-
How are my future users currently attempting to solve the problem? Is there an analog solution? And if so, what are the shortcomings of this analog solution?
-
Is there anyone else trying to solve this problem? How many “competitive” solutions are out there? What are the benefits and shortcomings of other solutions?
-
Are the existing solutions sufficient? (I.e. is there actually a need for a competing product?)
-
How can I leverage my nonprofit status to work with – not against – the “competition?”
Pro Tip
Let AI Simplify Data
The vast amount of market data can feel intimidating, but AI can streamline analysis and pull out what’s valuable. RebootRx, a nonprofit focused on drug repurposing, uses AI to sift through vast amounts of research data, quickly identifying viable drug candidates much quicker than manual research ever could.
Definition
MVP (Minimum Viable Product)
An early version of a new product used to collect the maximum amount of validated learning about customers with the least effort.
Test Your Idea with an MVP
This is when the bifurcation between brick and mortar nonprofits and tech nonprofits begins. If you wanted to run a brick and mortar nonprofit – say a soup kitchen – you could bring people into your home and feed them. You wouldn’t have to open the full kitchen to prove that you were capable of finding hungry people in need and feeding them well. This is not the case for tech nonprofits.
Tech nonprofits must build what we call an MVP (minimum viable product). You have to build an MVP early because until you put your solution into people’s hands, you’ll never know if your solution can solve the problem. It doesn’t have to be a fully functioning app. For example, Upsolve, which helps low-income Americans file for chapter 7 bankruptcy, built an MVP on Typeform (essentially a digital form) early on – but only after ample user research which included walking dozens and dozens of clients through the bankruptcy filing process manually.
Let’s say you want to launch a tech nonprofit that helps people find soup kitchens in their community. You have to get people to use your MVP to understand if the product is easy to use, if it delivers the service you promised, and if people are willing to use it. It’s hard to build a great tech product that people want to use. It is certainly harder than feeding people out of your home kitchen. You may have to go back to the drawing board many times before you lock in a solution that best serves your customer.
WHAT?! But I haven’t raised any money! We know, we know. But designing an MVP pilot is both critical and cheap. Conversations with users will only get you so far; observing the way your users engage with an MVP will take you to the next level. Since it’s a minimum viable product, it doesn’t need bells and whistles – nor does it even need to be a fully-fledged tech product. Your MVP can be a clickable demo, basic website, online form, or even a physical space where you’re helping your users IRL.
Keep it to the most basic features in the beginning. We’ve seen tech nonprofits get bogged down with features; you need to figure out which are the most essential. Or as you know we like to say at Fast Forward: keep the main thing the main thing.
Pro tip
Think First, Code Later
Wait as long as possible to write a line of code.
Definition
Pilot
A small-scale test that enables you to confirm or deny a hypothesis and answer critical questions before building and launching a fully fledged product.
Launch a Pilot
Once you have a clear idea of the market landscape and your user’s needs, the next step is to design a pilot using your MVP to determine viability.
Let’s say you’re building an edtech (educational technology) product. You’ve shown your MVP to a handful of committed teachers that are willing to be early adopters. The pilot is an opportunity to get your MVP in front of more teachers in different kinds of classrooms who may not be as committed to you as a founder. Pilot users tend to be a bit more discerning – and that’s a good thing. If you need to get creative in order to reach users who will pilot your product, consider teaming up with groups or organizations that serve your target customer currently using an analog competitive product.
Remember, use validated learnings to iterate as you go. Once you learn a key thing about your user, iterate so you can learn the next new thing! This phase is all about learning.
AI can supercharge this process of learning and iterating. For example, if the edtech platform Lenny Learning wanted to test their MVP with a group of teachers, an AI tool could assist in tracking and analyzing how teachers interact with it. The AI could then suggest improvements and redesigns much faster than a purely human-centered approach.
Once you’ve piloted your MVP for enough time that you’ve confirmed, denied, or altered your hypothesis – and you’re confident that you have a viable solution – it’s time to build a beta product.
Pro Tip
Goals of Running a Pilot
Test your hypothesis with more potential customers.
Learn where to find growth opportunities.
Understand what won’t work and why.
Case Study
CoachMe: The MVP Pilot in Action
Once Underwood had talked to dozens and dozens of future customers and understood their experiences with the problem, she first needed to answer the question: can we engage low-income patients in health coaching? To answer this, she designed a non-tech solution – a paper coaching sheet. Underwood used this sheet for user research, which consisted of in-person visits with patients. She discovered that the answer to her question was yes, patients were eager for support!
Next, Underwood needed to answer her second question: does text-based coaching work for low-income patients? She needed an infrastructure to test this, so Underwood took what she learned from the paper test and built a simple digital MVP. On the server-side, the tech stack consisted of Airtable forms and databases, and on the client-side, Underwood used Google Voice to facilitate free SMS and calls, as well as SMS logs that could be exported. The pilot revealed that the answer to this question was no. Patients were uncomfortable developing an SMS-only relationship, so CoachMe added weekly phone and video calls with patients. These critical learnings sent Underwood down the right path to eventually build out her beta. Starting with an MVP forced Underwood to quickly adapt a solution that met user needs, which led her to launch the impact-driven startup that later got acquired by CareMessage.
Definition
Iterate
To regularly add the highest value features to a given product to improve it.
Additional Resources
-
Build The Product Your User Needs with Geordie Graham and Mardi Daley, a free workshop from the Fast Forward Academy
User Research & User Testing Checklist
-
Complete a healthy amount of user interviews and observations.
-
Come up with a hypothesis around a useful tool to tackle the problem you’re focused on.
-
Design a simple MVP and run a pilot with different types of users.
-
Observe the MVP in your user’s hands, thereby validating learnings.
-
Confirm, deny, or alter your original hypothesis.