Remote usability testing allows you to conduct user research with participants in their natural environment by employing screen-sharing software or online remote usability vendor services. In general, tests should be about 15–30 minutes long made up of about 3-5 tasks.
Remote usability tests can be “Moderated” in the same manner you would for an in-person lab test or, “Un-moderated” where participants complete the tasks independently.
When to Consider Remote Testing
There are certain conditions when it makes sense to consider remote usability testing. Some of those conditions include:
- Timelines might prevent in-person testing due to scheduling issues
- The target audience/participants are geographically dispersed making travel for them or the experimenters difficult
- The participants need to use a particular work machine due to software or security requirements
- The participants have accessibility issues which require that they use their own software or equipment
- You can potentially run multiple tests at once
Although this method is flexible, as with any test, you must consider the benefits and cost of this methodology.
Developing a Remote Usability Test
The processes for developing a remote test are much the same as developing the test and materials for a test you are conducting in-person. Often you can leverage an in-person test you might have run for use in a remote setting. It’s important to:
- Remember that tests should be about 15–30 minutes long made up of about 3-5 tasks.
- Develop straightforward tasks that have well-defined end states
- If you are using a screener, be sure to include the minimum system requirements, both of the site or tool you are testing, but also for the screen sharing service that you propose to use
- Make sure you have the correct contact information for your participants for reminders and follow-up if needs be
- Prepare introductory and/or test materials so the participants will know what is expected of them as well as what they can expect from you
- Prepare test consent forms
- Prepare compensations and receipts for compensation should you choose to pay your participants
Remote testing differs mainly is in the technology. You will need to assure that:
- Whatever you are testing is accessible outside the firewall of your business, agency or dev environment
- You will need to determine if any firewall issues might affect the ability of your participants to access the site or tool you are testing
- Participants can download/access the screen-sharing software, or online remote usability vendor services
Moderated Remote Usability Testing
During moderated remote testing, participants are observed on interacted with while they complete the tasks for the test. Moderated testing is best for complex tasks that do not have a structured sequence of steps or where a more interaction and questioning will benefit testing.
To conduct a moderated remote usability test, in addition to the general guidance noted above, you will need to:
- Define the test and decide what tasks you are going to ask participants to perform, the order of the tasks, and any follow-up questions
- Schedule test sessions
- Build in enough time in the test session to allow for resolving technology issues
During the moderated remote usability tests you will be able to:
- Assign interview-based task performance
- Ask follow up questions
- Ensure test completion
- Test and analyze as planned
Depending on the technology or methodology used, you may also be able to analyze such metrics as task-completion rate, time on task, time on page, clickstream paths, and satisfaction ratings or opinion rankings.
Un-moderated Remote Usability Testing
During un-moderated remote testing, participants independently complete testing without an interaction with the test moderator. Un-moderated testing is most effective when you have very specific questions about how people use a user interface for relatively straightforward tasks.
To conduct an un-moderated remote usability test, in addition to the general guidance noted above, you will need to:
- Define the test and decide what tasks you are going to ask participants to perform, including the order of the tasks
- Determine who will participate. Many un-moderated testing tools allow you to either intercept users on your Web site or recruit them from the tool developers’ own list of test participants
- Determine ahead of testing whether your screen sharing service either has recording capability or will allow for the use of your recording equipment or software.
- Be aware that picture-in-picture(PIP) or webcams may not be captured if your software can only accept one video input.
- If choosing from a pool of recruited participants, make sure they are representative of your target audience
- Include qualitative research questions when possible and analyze self-reported feedback
- Pay attention to time-on-task and open-ended feedback to filter out participants who may be participating for the sole purpose of earning the incentive
- If your participants have any vision challenges, consider asking if they are familiar with the screen share software you are using. If not consider asking whether a colleague or family member will be available during testing to assure that they can share their screen
During an un-moderated remote usability test, you will need to:
- Launch the test and send email invitations to participants to complete it.
- After the sessions have been completed you can analyze your results. Many un-moderated testing tools automatically generate analysis for you.
Depending on the technology un-moderated remote usability tests may also allow you to analyze such metrics as task-completion rate, time on task, time on page, clickstream paths, satisfaction ratings or opinion rankings, web analytics data (such as browser, operating system, and screen resolution).
- “Remote Research: Real Users, Real Time, Real Research” by Nate Bolt Rosenfeld Media; 1st edition (February 8, 2010)
- Remote Usability Testing: Good or Evil