The Roadmap to a Successful Usability Test

Successful Usability Tests
Usability Tests

onducting a usability test is far from easy, and there are numerous downfalls that you will face if the test is not conducted properly. Usability testing is an important UX Research methodology that is used in every project because the outcomes are mind-blowing. But conducting the usability testing properly is important because it can transform the entire test into a failure

First of all, what is usability testing?

According to IxDF, Usability testing is the process of evaluating a design’s usability with a group of users that represent the target audience. It normally occurs by watching users attempting to complete a set of tasks and that can be applied to a variety of designs. Usability Testing usually occurs repeatedly from early development until the product is released to the market.

Evaluating Designs
Evaluating Designs
Evaluating Designs (Image Source: Unsplash)

So, in a more summarized version, we can say that it’s the way that a user thinks of a product. Any product which the user uses must not bring them any hardships and confusion. To avoid these issues in a final product, we should conduct usability testing to get to know any of these issues beforehand.

Source: What is Usability Testing? by Interaction Design Foundation

The ultimate goal of usability testing

The definition of usability in the ISO 9241 standard is:

“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”

As stated by WQ usability, this definition can be expanded into 5 characteristics that are important to consider when the user meets the product.

  • Effective
  • Efficient
  • Engaging
  • Error Tolerant
  • Easy to Learn

These 5 characteristics are the ultimate 5 goals of usability and they can be called “The 5 Es”. Anyone who users a product that is effective, efficient, engaging, error-tolerant, and easy to learn, would use that product for a very long time. To achieve these 5 Es, we should conduct thorough usability tests to make sure that we provide the best product to the market.

As we know, before the start of anything it's always best to organize everything properly so you would not face any issues. Like so, when conducting a usability test you must plan ahead about every single aspect of it.

Source: ISO 9241–11:2018, What Does Usability Mean: Looking Beyond ‘Ease of Use’ by WQ usability

Create a solid foundation

As the first step, you should define the scope of the usability test and create a solid plan accordingly. Ask yourself the following questions and note down the answers,

  • Why do you need a usability test?
  • What do you want to achieve from it?
  • What problem are you trying to solve?
  • What areas do you want to improve?
  • What specific tasks do you need to be tested?

These questions will help you realize what outcomes are you trying to achieve from the usability test because the scope of the usability test is the foundation that you need to have. When you have the specific outcomes of the usability test noted down and the scope is defined, you can conduct the usability test without facing any issues.

Scope of a Usability test
Scope of a Usability test
Scope of a Usability test (Image Source: Unsplash)

Defining team members and recruiting users

After defining what you want to achieve from this usability test, you have to think about who will assist you on this and the test users that you are going to recruit.

According to Jakob Nielson, 5 test users are usually enough for a usability study evaluation for one test cycle because 85% of the problems would surface by then. As usability testing is iterative, from each test round you can find what issues are available until everything is solved.

Recruiting usability test participants
Recruiting usability test participants (Image Source: Unsplash)

Recruiting test users is not an easy task. There are several ways in which you can find participants for your usability tests.

  • Existing users — You can recruit previous users that have participated but you cannot assume that existing users will always take part.
  • Recruiting from an online service — There are different sites just for recruiting test users such as TestingTime and UserTesting. These sites are very easy to use because you can put in all the requirements you expect and even demographic information and get the exact test users that you need.
  • Research agencies — Research agencies have big databases filled with demographic information that you will need specifically for your usability test. You can find the best agency that fits your purpose and register for their services.

You can choose either one of the above methods that you would think is the best and recruit your test participants but remember, 5 test users are enough!

Source: Why You Only Need to Test with 5 Users by Jakob Nielsen, How to recruit users for Usability Studies by Interaction Design Foundation

Set user tasks for a realistic goal

The most important part of a usability test is the user responses. These responses impact everything that your project is about therefore you must define the correct tasks to get the correct responses that you expect. In addition to this, the user tasks that you define must be according to a realistic goal. If you try to make tasks that are not practical for the user, even the user will face hardships when completing the tasks and this will impact the entire project.

User tasks
User tasks (Image Source: Unsplash)

The user tasks can be open (exploratory) or closed (specific). A mix of these tasks for a usability test is always good practice. In addition to this, when you are building the tasks for the users, you shouldn’t showcase the task to the user as if it is a question that they are supposed to answer. You must make sure to display meaningful actionable tasks according to the goal you want to achieve.

Another important aspect is the specificity of your tasks. Users should not face any confusion or they should not have any questions about the task like “What am I supposed to do next?”. But you should not give instructions on what they should do next either! Giving instructions as to what the user must do will not provide the outcomes that you expect because you will be telling them each and every step to the user. Usability testing is not that! Therefore, you must make sure the task is simple but actionable and specific.

Source: The 7 step roadmap for effective usability testing by invesp

What type of testing should you choose?

Usability tests have no one type of method that you should do. It usually depends on the requirements and the outcome that you need. You should choose if your test should be In-person or remote, Monitored or unmonitored and the data that you collect is quantitative or qualitative.

Usability Testing
Usability Testing (Image Source: Unsplash)

Qualitative VS Quantitative

The outcomes of the usability test that you expect depends on the type of data that you need, that is Qualitative or Quantitative.

Qualitative usability testing aims to gather information about how customers use the product or service. Qualitative usability testing is the most effective method for identifying issues with the user interface.

Quantitative usability testing is involved with gathering metrics that characterize the user experience. Task success and time on task are two of the most popular metrics obtained in quantitative usability testing.

Remote Testing VS In-person Testing

Remote testing and In-person testing are mostly similar because the facilitator will interact with the participant and ask to perform specific tasks. The only difference between these two is in remote testing the facilitator will be supervising the participant through an online platform which has screen sharing facilities such as Google meet or skype

Monitored VS Unmonitored

Monitored usability tests will have the facilitator in the same room as the participant interacting with them with every task the participant does. In unmonitored testing, the user will perform the tasks independently and on their own terms. To set up written tasks for the participant, the researcher uses a dedicated online remote-testing tool. The task instructions and any follow-up questions are delivered by the testing tool. Unmonitored testing is mostly done with remote testing.

But for some projects, you would come across, if you need to do quick and dirty testing you could do guerilla research. This is a low budget and a quick way to get fast results when you are in short of time. Guerilla research can be done by testing your design informally on your colleagues or passers-by.

Source: Usability Testing 101 by Kate Moran

Conducting the usability test

When conducting the usability test it is best to record the entire session but with the user’s consent. You can take notes as the session goes on but it will be more efficient if it can be recorded so for future matters you can refer to these recordings for clarification.

When you are conducting usability testing, it is important that the user is comfortable with everything. Have casual conversations with the participant and always non-judgmental to make your users feel at ease. Make sure to give the user their own space. Don’t be a supervisor and track every movement.

For example, if you’re doing an exam, you would feel uncomfortable if the supervisor stood in front of you the whole time looking at your paper right?

It’s the same when it comes to the usability test! So, make sure your users are always comfortable to get the best outcomes. Steve Krug’s documentation on “Things a therapist would say” shows how you should communicate with your users which you should definitely make use of. Most importantly, you should never give any blame to the user if something goes wrong with the test. You should treat the participants like MVPs (Most Valuable Person).

Casual conversation after the test
Casual conversation after the test
Casual conversation after the test (Image Source: Unsplash)

After the session is completed, you should never let the users leave at that instance. Often the participants of a usability test would feel like a test subject and cause frustration therefore post-test questions are important. To comfort the user, you can have a casual conversation on how the test was and how their experience on it is. A conversation like so will suggest that the serious part of the test is complete, and they can now freely express their true feelings. It really opens some people up, and even if they didn’t say anything during the test, they may offer a lot of feedback during the post-test questions. Finally, when the participant is leaving, never forget to thank them for their valuable time!

Source: All you need to know to run a successful Usability Testing by Anna-Zsófia Csontos

Creating the test report

The final step of any usability test is to analyze your findings. For this, there is no specific way you can analyze the information. You can discuss with your team members all the information that was collected and work with any method that you would think is practical with the information. You will identify the product’s problem areas and come up with solutions based on the information you gathered.

Analyze the findings
Analyze the findings (Image Source: Unsplash)

After analyzing the findings, you can create the test report based on the conclusions that you have drawn. The type of report always depends on the products’ strategy and requirement. The final report can be used to make the new adjustments to the design solution that you have created and the new changes can be tested again with usability testing.


Even though this process may seem a lot complicated, it is actually not. As you know usability testing holds a high place when it comes to the Design process therefore to provide the best product to the users you must always repeatedly test the product to see if it gives the outcome that is expected.

Some references and useful resources for you!

What is Usability Testing? by Interaction Design Foundation

What Does Usability Mean: Looking Beyond ‘Ease of Use’ by WQ usability

Why You Only Need to Test with 5 Users by Jakob Nielsen

How to recruit users for Usability Studies by Interaction Design Foundation

The 7 step roadmap for effective usability testing by invesp

Usability Testing 101 by Kate Moran

All you need to know to run a successful Usability Testing by Anna-Zsófia Csontos

6 steps to successful usability testing by Bansi Mehta

What is Usability Testing? by Kitty Wong

A UX designer & UI developer who also has a passion for music. Enjoy capturing moments with a click. spend leisure time by creating videos.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store