Creating a product that is pleasant and easy to use requires more than technical tests or conforming to the WCAG 2.1 guidelines.
The best way to find out if your product works for your intended target audience, is to test it with them. Usability testing gives you rich insights in how your target audience uses your product. Find out which problems people encounter and discover opportunities to improve your product.
This guide will help you organise and run a usability test for people with and without a disability.
How to prepare for a usability test
Define questions
Start by writing down the questions you have about your product. For example: how easy is it for the user to reach goal X? or: does the order in which content is presented feel logical to the user?
Make a prototype
See if your prototype has all the features needed to answer your questions. You may need to postpone some questions or expand your prototype.
Invite
people
Define your target audience. Who might use your product?
Write an invitation text and an information letter.
Invite a diverse group of people, with and without a disability, so you can find as many UX issues as possible.
Write
your script
Write a program or script for the entire usability test.
A script contains a checklist, an instructional text, questions for the participant and tasks that you will give them (to interact with your product).
Test your script with a co-worker, neighbor or friend.
Plan your test
When you have defined your research questions you can start working on your test tasks or task scenarios. You give your testers tasks so you can test specific parts of your product and see if they can succeed without usability or accessibility issues.
- Write specific and goal-oriented tasks. Break up long tasks into multiple tasks.
- Do not use words that guide the tester too much into a direction (eg. saying ‘add this to your cart’ when the button also says ‘add to cart’)
- Let your tester perform the task as they would at home
More on writing tasks: Turn User Goals into Task Scenarios for Usability Testing.
There is a free usability test plan dashboard you can fill in. It includes questions about test objectives, participants, equipment and tasks.
Inviting and informing participants
Inviting 3 to 8 people to test your product is a great way to start. It’s better to do multiple usability tests with three people, than to invite ten people for one test at the end of development.
Find people from different educations, age groups, reading proficiency (immigrant, low literacy, dyslexic) and disabilities (hearing, visual, speech or motor impairment). Keep in mind that a single (disabled) person is not a representation of everyone with similar characteristics.
What should you say in your invite?
When inviting people to participate in your usability/accessibility study, these are the things they want to know before agreeing to participate:
- What is this study/test about?
- Date and time, can I schedule it outside of work hours?
- What are we going to do and how long does it take?
- Where is it and how do I get there?
- Do I get any compensation and/or travel allowance?
- Should I bring anything?
- Is the session being recorded and if so, what will you do with it?
Send the info in plain text in an e-mail. Don’t send a pdf (it might not be accessible) or a poster (image). Download an example letter below that is approved by the HAN research ethics committee.
What to discuss when a person agreed to participate?
Find out which assistive technology your participants use. Ask them to bring any hardware (like a Braille reader, magnifier or a special mouse) and make sure you install any software they need (eg. JAWS, NVDA, VoiceOver) on a backup computer. Consider whether you want to let the tester use their own device.
- Provide info on how to start using your prototype: do you need a specific phone or computer, do you need to install anything prior, programs needed to install, login details etc.
- If you invite a participant to travel somewhere for the test, know that traveling might cost them extra time and energy. Consider going to them instead, or if possible, meet online.
- You should pay for any travel expenses.
- Provide clear directions to the testing location and wait for them at the entrance or at the bus stop/train station. Give your phone number just in case.
- Make sure the location is wheelchair accessible.
- We always give participants something to express our gratitude. As we cannot gift money at HAN, we give them a gift certificate/voucher or a goodie bag with local produce (no alcohol or meat). A carefully composed package is usually most appreciated. Gift certificates are generally not very accessible for people with a visual impairment.
Consent form
Only collect participants data (eg. record audio/video or store personal information) if really needed. Make sure you ask permission first. Below is an example of a written consent form.
Verbal consent script
Sometimes you can’t use a written consent form. Like when your participant has trouble reading, has a visual impairment or joins you for a remote test. Asking a visually impaired participant to sign something they can’t see, raises some ethical questions. They might not even have a signature.
Record the permission, and store separately from the test/interview. An audio or video recording is a legally valid way of getting consent.
Conducting a usability test
Run a trial test with a colleague or neighbor to check all equipment and whether your tasks provide the info you need to reach your goal.
Interviewing someone during a test is a skill you need to practice. If you’re new to facilitating usability tests, the checklist on this webpage will help you ask the right questions. You could also check out some best practices as visualized in this comic by Dataveyes (not accessible).
Tips for testing with people with disabilities
- Check if your testing room is accessible. If you meet online: find out which programs are accessible. Microsoft Teams works well.
- When you meet a participant with a visual impairment: ask them if and how they want to be guided. Inform them about obstacles or things that are on the desk (like a glass of water).
- Keep in mind that participants may bring their service dog, which needs space under the desk (and would love a bowl of water).
- Informed consent: don’t let people with a visual impairment or low literate people sign a written consent form. Read the consent form out loud (see example) and record their verbal permission.
- If you can’t follow a screen reader’s speed, ask the participant to modify the speed.
Learn more
- Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. Berkeley, CA: New Riders.
Nielsen Norman Group. How to Conduct Usability Studies for Accessibility.
Nielsen Norman Group. Usability Testing 101.
- Nielsen Norman Group. Turn User Goals into Task Scenarios for Usability Testing.
