The Roadmap to a Successful Usability Test
The Ultimate Guide
Conducting a usability test is far from easy, and there are numerous downfalls that you will face if the test is not conducted properly. Usability testing is an important UX Research methodology that is used in every project because the outcomes are mind-blowing. But conducting the usability testing properly is important because it can transform the entire test into a failure
First of all, what is usability testing?
According to IxDF, Usability testing is the process of evaluating a design’s usability with a group of users that represent the target audience. It normally occurs by watching users attempting to complete a set of tasks and that can be applied to a variety of designs. Usability Testing usually occurs repeatedly from early development until the product is released to the market.
So, in a more summarized version, we can say that it’s the way that a user thinks of a product. Any product which the user uses must not bring them any hardships and confusion. To avoid these issues in a final product, we should conduct usability testing to get to know any of these issues beforehand.
Source: What is Usability Testing? by Interaction Design Foundation
The ultimate goal of usability testing
The definition of usability in the ISO 9241 standard is:
“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”
As stated by WQ usability, this definition can be expanded into 5 characteristics that are important to consider when the user meets the product.
- Effective
- Efficient
- Engaging
- Error Tolerant
- Easy to Learn
These 5 characteristics are the ultimate 5 goals of usability and they can be called “The 5 Es”. Anyone who users a product that is effective, efficient, engaging, error-tolerant, and easy to learn, would use that product for a very long time. To achieve these 5 Es, we should conduct thorough usability tests to make sure that we provide the best product to the market.
As we know, before the start of anything it's always best to organize everything properly so you would not face any issues. Like so, when conducting a usability test you must plan ahead about every single aspect of it.
Source: ISO 9241–11:2018, What Does Usability Mean: Looking Beyond ‘Ease of Use’ by WQ usability
Create a solid foundation
As the first step, you should define the scope of the usability test and create a solid plan accordingly. Ask yourself the following questions and note down the answers,
- Why do you need a usability test?
- What do you want to achieve from it?
- What problem are you trying to solve?
- What areas do you want to improve?
- What specific tasks do you need to be tested?
These questions will help you realize what outcomes are you trying to achieve from the usability test because the scope of the usability test is the foundation that you need to have. When you have the specific outcomes of the usability test noted down and the scope is defined, you can conduct the usability test without facing any issues.
Defining team members and recruiting users
After defining what you want to achieve from this usability test, you have to think about who will assist you on this and the test users that you are going to recruit.
According to Jakob Nielson, 5 test users are usually enough for a usability study evaluation for one test cycle because 85% of the problems would surface by then. As usability testing is iterative, from each test round you can find what issues are available until everything is solved.
Recruiting test users is not an easy task. There are several ways in which you can find participants for your usability tests.
- Existing users — You can recruit previous users that have participated but you cannot assume that existing users will always take part.
- Recruiting from an online service — There are different sites just for recruiting test users such as TestingTime and UserTesting. These sites are very easy to use because you can put in all the requirements you expect and even demographic information and get the exact test users that you need.
- Research agencies — Research agencies have big databases filled with demographic information that you will need specifically for your usability test. You can find the best agency that fits your purpose and register for their services.
You can choose either one of the above methods that you would think is the best and recruit your test participants but remember, 5 test users are enough!
Source: Why You Only Need to Test with 5 Users by Jakob Nielsen, How to recruit users for Usability Studies by Interaction Design Foundation
Set user tasks for a realistic goal
The most important part of a usability test is the user responses. These responses impact everything that your project is about therefore you must define the correct tasks to get the correct responses that you expect. In addition to this, the user tasks that you define must be according to a realistic goal. If you try to make tasks that are not practical for the user, even the user will face hardships when completing the tasks and this will impact the entire project.
The user tasks can be open (exploratory) or closed (specific). A mix of these tasks for a usability test is always good practice. In addition to this, when you are building the tasks for the users, you shouldn’t showcase the task to the user as if it is a question that they are supposed to answer. You must make sure to display meaningful actionable tasks according to the goal you want to achieve.
Another important aspect is the specificity of your tasks. Users should not face any confusion or they should not have any questions about the task like “What am I supposed to do next?”. But you should not give instructions on what they should do next either! Giving instructions as to what the user must do will not provide the outcomes that you expect because you will be telling them each and every step to the user. Usability testing is not that! Therefore, you must make sure the task is simple but actionable and specific.
Source: The 7 step roadmap for effective usability testing by invesp
What type of testing should you choose?
Usability tests have no one type of method that you should do. It usually depends on the requirements and the outcome that you need. You should choose if your test should be In-person or remote, Monitored or unmonitored and the data that you collect is quantitative or qualitative.
Qualitative VS Quantitative
The outcomes of the usability test that you expect depends on the type of data that you need, that is Qualitative or Quantitative.
Qualitative usability testing aims to gather information about how customers use the product or service. Qualitative usability testing is the most effective method for identifying issues with the user interface.
Quantitative usability testing is involved with gathering metrics that characterize the user experience. Task success and time on task are two of the most popular metrics obtained in quantitative usability testing.
Remote Testing VS In-person Testing
Remote testing and In-person testing are mostly similar because the facilitator will interact with the participant and ask to perform specific tasks. The only difference between these two is in remote testing the facilitator will be supervising the participant through an online platform which has screen sharing facilities such as Google meet or skype
Monitored VS Unmonitored
Monitored usability tests will have the facilitator in the same room as the participant interacting with them with every task the participant does. In unmonitored testing, the user will perform the tasks independently and on their own terms. To set up written tasks for the participant, the researcher uses a dedicated online remote-testing tool. The task instructions and any follow-up questions are delivered by the testing tool. Unmonitored testing is mostly done with remote testing.
But for some projects, you would come across, if you need to do quick and dirty testing you could do guerilla research. This is a low budget and a quick way to get fast results when you are in short of time. Guerilla research can be done by testing your design informally on your colleagues or passers-by.
Source: Usability Testing 101 by Kate Moran
Conducting the usability test
When conducting the usability test it is best to record the entire session but with the user’s consent. You can take notes as the session goes on but it will be more efficient if it can be recorded so for future matters you can refer to these recordings for clarification.
When you are conducting usability testing, it is important that the user is comfortable with everything. Have casual conversations with the participant and always non-judgmental to make your users feel at ease. Make sure to give the user their own space. Don’t be a supervisor and track every movement.
For example, if you’re doing an exam, you would feel uncomfortable if the supervisor stood in front of you the whole time looking at your paper right?
It’s the same when it comes to the usability test! So, make sure your users are always comfortable to get the best outcomes. Steve Krug’s documentation on “Things a therapist would say” shows how you should communicate with your users which you should definitely make use of. Most importantly, you should never give any blame to the user if something goes wrong with the test. You should treat the participants like MVPs (Most Valuable Person).
After the session is completed, you should never let the users leave at that instance. Often the participants of a usability test would feel like a test subject and cause frustration therefore post-test questions are important. To comfort the user, you can have a casual conversation on how the test was and how their experience on it is. A conversation like so will suggest that the serious part of the test is complete, and they can now freely express their true feelings. It really opens some people up, and even if they didn’t say anything during the test, they may offer a lot of feedback during the post-test questions. Finally, when the participant is leaving, never forget to thank them for their valuable time!
Source: All you need to know to run a successful Usability Testing by Anna-Zsófia Csontos
Creating the test report
The final step of any usability test is to analyze your findings. For this, there is no specific way you can analyze the information. You can discuss with your team members all the information that was collected and work with any method that you would think is practical with the information. You will identify the product’s problem areas and come up with solutions based on the information you gathered.
After analyzing the findings, you can create the test report based on the conclusions that you have drawn. The type of report always depends on the products’ strategy and requirement. The final report can be used to make the new adjustments to the design solution that you have created and the new changes can be tested again with usability testing.
Conclusion
Even though this process may seem a lot complicated, it is actually not. As you know usability testing holds a high place when it comes to the Design process therefore to provide the best product to the users you must always repeatedly test the product to see if it gives the outcome that is expected.
Some references and useful resources for you!
What is Usability Testing? by Interaction Design Foundation
What Does Usability Mean: Looking Beyond ‘Ease of Use’ by WQ usability
Why You Only Need to Test with 5 Users by Jakob Nielsen
How to recruit users for Usability Studies by Interaction Design Foundation
The 7 step roadmap for effective usability testing by invesp
Usability Testing 101 by Kate Moran
All you need to know to run a successful Usability Testing by Anna-Zsófia Csontos
6 steps to successful usability testing by Bansi Mehta
What is Usability Testing? by Kitty Wong
