Usability testing is one of the most useful tools to uncover how a website works for users. With effective planning, recruitment, and execution, any UX team or individual will be able to uncover a site’s main pain points and formulate recommendations that will help to improve the overall user experience in very little time.
A method of usability testing that can be useful to engage with in the beginning of a project is simply discount usability testing. These tests are planned with a small amount of users, since we are not looking for statistical significance in terms of usability issues. Instead, you are going to plan a test and recruit users to try and identify qualitative information and observe how users interact with our website while conducting a number of tasks. To accomplish this, we don’t need to test with more than seven users (generally).
Here’s how I would approach it.
Number of participants
For our tests, we only need to recruit eight testers — one being an alternate in the event of a cancellation. We are aiming for seven users for a number of reasons. According to Jakob Nielsen, an expert on usability and user-centered design, after testing with more than five users, you start to see the same issues and don’t learn additional insights. Our limited time frame only gives us a chance to run one round of usability testing. With that, we need to maximize our time with our testers, and recruiting eight testers (one alternate) is our best course of action.
Testing with only a few participants
Testing with less than 10 users may appear insignificant to draw major conclusions around a website’s usability issues. However, industry research tells us that we can uncover nearly 75% of the major usability issues in five sessions or less (Nielsen 1993). Again, we aren’t looking for statistical significance since our method of usability testing is not conducted in a true academic sense. Instead, we are attempting to identify what about a product is difficult to use. We will use direct observation, and encourage our testers to “think aloud” as they perform the tasks set for them. This will help us uncover the major usability issues that are contributing to a website’s main issues.
In addition, the severity of issues uncovered during usability tests are also related to the first few participants in the rounds of testing. In other words, usability issues and their severity and frequency are significantly correlated, which implies that severe issues are commonly discovered during the first few tests (Virzi 1992). This means we can get more value out of only testing with seven users since it will save us considerable amount of time, while still giving us strong usability insights.
The testing plan
Our testing plan is designed to accommodate the development team’s agile software development process. We will be efficient with our time, but take real consideration when we are drawing conclusions, findings, and recommendations in order to be smart about any changes we propose to the website. While the development team is working in week-long sprints, the UX group will need three weeks to complete our work.
| Week 1 – Plan & Recruit users | -Begin recruiting eight users for testing -Develop facilitator script, key tasks and questions for participants, and collaborate with the the development and business teams to make sure our tasks are reflecting our site’s main tasks -Begin to schedule usability sessions with participants -Brainstorm a possible honorarium we can provide testers (gift card, cash, etc.) |
| Week 2 – Test | -Conduct usability test sessions in person at our usability lab (UX group facilitator will lead sessions)Invite development team or interested parties to observe remotely -After each session there will be a debrief with anyone that observed, and we will discuss top user pain points, user emotional reactions (nonverbal body language) and any other issues -Sessions will be limited to two per day |
| Week 3 – Analysis & Recommendations | -Analyze usability findings and synthesize major themes -Develop recommendations that address major findings -Depending on severity of issues uncovered, work with development team to fold changes into weekly sprints thereafter. |
In addition to the first round of usability testing, I would recommend building this type of iterative testing into your software development process. Since we can observe significant usability issues with only a few tests, conducting more regular analysis will help to proactively identify usability issues that otherwise could hurt our sales and frustrate our users.
This usability testing plan allows you to work efficiently and will not cost us much money. While we often have a limited timeframe, our tests are planned to maximize our testing resources. Considering no other issues come up, the a UX group should be able to plan, test, debrief, and recommend development changes to the site just in time to not upset your scrum master.