https://pds.blog.parliament.uk/2017/01/23/getting-a-mobile-testing-lab-off-the-ground/

Getting a mobile testing lab off the ground

At PDS we like to push ourselves and aim big. After our visit to GDS at the end of November we felt inspired to get our user testing lab set up. Our Christmas wish was to do user testing on mobile devices by January.

In the past we've used interviews and guerrilla research. However they don't give the same richness of data we can get from people in a lab using something live.

Our first step was to find people. We worked with a recruiter to find participants that fitted our personas and set dates for people to come in.

Before we could run live sessions there were a few minor things to figure out. How to handle many camera feeds? How to stream to another room in Parliament? How to get mobile phones to test?

But we did it and here's our guide on how we got mobile testing up and running.

Our guide to mobile testing

Recruit some people

You’re going to need some people. The more details you can give to your recruiter, the more accurately they can recruit to your specifications. Keep in contact in the build up to the testing.

Book a room ahead of time

Space is tight in our building but we pulled some favours and got enough space for people to observe and somewhere to welcome participants. Advertise where the viewing party will be so people can put it in their calendars.

Order your hardware as early as possible

We had a desktop computer already but we needed some phones. We ordered them through the due process and waited. And phoned to check. And waited. Luckily they arrived the day before.

Test your feeds

For the observations we wanted to get a view of the participant, the mobile phone screen and what they were touching. We combined the computer's in-built camera, a webcam looking down at the phone and a feed of the phone's screen. We used a cable and Quicktime for Apple phones and a Chrome plugin called Vysor for Androids. We figured out we could combine the feeds using a piece of software called OBS. Google hangouts solved the issue of streaming to another laptop.

Build your prototype

The designers on our team (Jack Craig and Laurence Grinyer) and Joe Strawson, our content expert, built a series of wireframe screens for people to navigate through. Getting this done as early as possible allows you to develop your discussion guide and test the script. The prototype was loaded into Marvel and allowed our testers to use it like a website.

Bring out your inner engineer

The only thing we missed was something to hold the webcam over the phone so we could see what the person was touching. The solution most people pointed us to was Mr. Tappy. Unfortunately we didn't order this in time and it had to come from New Zealand so we needed another short term solution.

I had to bring out my inner engineer.

At primary school, I made a race car out of rolled sheets of paper, nuts and bolts, a battery, motor and some wheels. I got a certificate for doing well and got to race it in front of the whole school. While running into work I remembered this story and thought that something similar could work to hold the webcam.

Sketch of the device which held the camera
Sketch of the device which held the camera

What we used:

  • 4 sheets of A3 paper
  • 7 screws and 7 wing nuts (at a cost of £2.10 from a local hardware store)
  • some card, foam and tape
  • a hole punch
  • scissors or scalpel

It's strong, light and gave a good view of what the participants were using to navigate or hover over. I cut out a hole for the camera so the phone could lie flat against the card. I also scalloped the edges to allow people to hold the phones easier. I used some sticky dots to attach the phone. My colleague has named it Ms. Swipey.

The device with the camera attached to monitor activity on the phone
The device with the camera attached to monitor activity on the phone

Running the lab sessions

The lab was now ready. We asked participants to work through the prototype answering questions. In another room, members of the team were eagerly watching.

Our first lab session excited the team and they could also see how all of the pieces they were working on could fit together.

As we hadn’t done this before we were unsure if there were going to be any problems. As a backup, we had an observer in the room watching the stream and taking notes. If the feeds failed at least they would still be able to hear what was going on. Using the GDS guidance for observing, the observer took notes of each participant. The following day, we analysed the results to feed back into future research and the next iteration of the website prototype.

Dana, a user researcher, during a feedback session
Dana, a user researcher, during a feedback session

Overall the sessions went smoothly. The only change we made between sessions was to use an external (and better) microphone to pick up the people who spoke quietly. For the next round of lab sessions we'll hopefully have a Mr Tappy.

We’ve also updated the recruitment screener to adjust the type of people we see. We’re going to organise a notetaking exercise to improve the observations that come out. Hopefully now with the lab up and running we’ll be able to do usability testing with more products across Parliament.

Please email us or leave a comment if you want to know more about our research.

Leave a comment