Minimize text size Reset text to normal size Increase text size

Mobile Device Testing

Testing mobile devices such as phones, tablets, and eReaders requires special equipment and methodology.  Since traditional desktop screen-capture software cannot adequately capture touch interactions, usability practitioners have been using strategically placed cameras to record usability test interactions on these mobile devices.

Planning and Preparing to Perform Mobile Device Testing

Methodology and devices to facilitate usability testing on phones, tablets, and eReaders continue to develop. When planning mobile device testing, you should think about:

  • Your timeframe and budget. Knowing these will help you to determine which processes and equipment will work best based your needs. 
  • The physical setup of the space and how you will capture the test.  This may range from a lower-fidelity arrangement to using a specialized platform and camera device and/or maybe using eye tracking software.
  • Your target audience and devices. Use web analytics to check how many mobile users visit the site, which device they use, and the operating system.  Understanding this information will help you to know which devices to test.
  • What’s included in your screener.  It’s important to be specific about:
    • The type of phone the participant must have
    • What generation of phone, if necessary
    • What activities they must do regularly on their device (i.e. “Please indicate which of the following online activities you have participated in the last 30-days”).
  • How the test is structured.
  • Consider how frequently you will need to perform testing.
  • Ensure there is connectivity in the space. Make sure that there is a good cross section of wireless carriers getting service in the area you intend to test.  Connectivity is not always universal and you must pilot test before your participants arrive.

Camera Setup Considerations

When you are deciding on a camera setup, ideally you should look for the following:

  • Easy setup
  • Fixed distance between device and camera
  • Works for phones and tablets
  • Portrait and landscape screen capture
  • The user can hold/use their device as “normally” as possible while ensuring camera captures all action

a cell phone on a usability testing palette

Creating Test Scenarios

Task or topic order is even more important on mobile sites. When creating test scenarios, be cognizant that you may need to lower the number of tasks and interact more with the participant in order to catch their process. 

  • Where logical, consider sending them back to home between tasks.
  • Consider asking questions related to the completeness of the content, confidence that they found the information they needed.
  • As part of wrap up, consider asking whether the site had:
    • More or less content than a full site?
    • When they would need to go to the full site?
    • Would they use the mobile site?
    • Would they recommend the mobile site?

Conducting Mobile Device Testing

When conducting mobile device testing, there are several things that you need to keep in mind:

  • It is advisable to have them test using their own devices.  This eliminates issues associated with an unfamiliar, device or operating system. Encourage user to use the device as normally as possible and adapt the camera as possible or needed.
  • Be aware of the lighting in the test room or lab. Overhead lighting can cause a glare on the screen, which may impair the recording.  Dimming the lights in the room or adjusting the position of the camera or the device might help.
  • Not only should you pilot test the scenarios and the connectivity of the room in which you are testing, but also pilot with as many of the anticipated phone types and carriers as possible. 
  • Consider having dry glass cleaning cloths available to clean the screens before testing – just to improve the clarity of the image
  • Be careful not to have the device too close to the test computer.  Devices that emit pulses, like phones – may cause noisy feedback for remote observers.
  • Read the tasks to them, and/or have the tasks printed out

References