https://pds.blog.parliament.uk/2017/04/10/analysing-research-data-as-a-team/

Analysing research data as a team

After doing a round of user research sessions, we review the results in a workshop. We call them collaborative analysis workshops. We've used these workshops for many of the projects we’re supporting, including the Intranet redesign, Historic Hansard, Newsletters and Committee Reports. It’s one of the main ways that we share the findings of research with the teams involved.

In this post, I’ll explain what collaborative analysis workshops are, and why we use them.

What collaborative analysis workshops are

Collaborative analysis uses raw data to help show the findings of research. An important part of being a user researcher is making sure that teams across Parliament understand their users. These workshops are a good way to do that.

How to run a collaborative analysis workshop

Collaborative analysis workshop
Stakeholders presenting their findings at a collaborative analysis workshop

To prepare, our researchers print anonymised transcripts of the interviews. Each transcript is on a separate page marked with a participant number.

The room also has ‘topics’ printed out on the walls. These can either be the research questions or some themes that the researcher thinks will come out of the data. When using this process to analyse the results from usability tests, screenshots of each page of the software can be used for this bit (as demonstrated by GDS).

The workshop is in two parts.

In the first part, each stakeholder is given a transcript and asked to read through and copy any relevant information onto Post-its. They’re asked to put one observation per Post-it with each note tagged with the participant’s number. They then stick their Post-its against the relevant topic on the wall.

Once everyone’s done this, and everyone's read the transcripts, we ask stakeholders to talk about their notes. This helps to build a shared awareness of what came out of the interviews.

Then we have a tea break - very important. If there's time, a researcher will use this break to go around the room and group similar items next to each other.

For the second part of the workshop, we split into groups of three and review the topics on the wall. For each topic, we group items that are describing the same thing, and add a ‘header’ Post-it that describes the sentiment or issue within the group. Sometimes the same themes come up under different topics, so we’ll combine them when this happens.

Once all the groupings are complete, they're discussed, along with the findings from that grouping.

The output of the workshops is a list of 'top level' findings, each with the original data behind it, captured in our notes. A researcher can take these away and write them up – either as user needs, or in other formats appropriate for the project. They’ll send this to the rest of the team, adding their own insight or details missed in the workshop.

Why the workshops are worth doing

These workshops take longer to schedule, prepare and run than if the researcher was doing affinity mapping by themselves. If the goal was just to write a report then the value might not be obvious. Our real goal is to make sure everyone understands their users so it’s worth spending more time running these workshops.

The main benefit is that these workshops make sure every stakeholder has exposure to real user feedback. Both on an individual level and the aggregated findings. This helps them understand and appreciate that the findings are valid, and can give them a greater level of empathy and understanding about the needs of users. They wouldn’t get this if we just presented them with a report.

Another benefit is stakeholders often have a lot of subject matter knowledge that user researchers don’t have, particularly when dealing with complex procedures (which are very common in Parliament). By involving them in this stage, we can get a sanity check, ask questions, or improve our representation of the findings. This means we can reduce errors that distract from the findings, or affect our credibility.

Future of collaborative analysis workshops

Our team are currently using workshops to share our research findings and we're always looking for ways to make them better. For example, we'd like to spend time learning how the choice of topics prepared on the wall affects the final groupings. We’ll continue to iterate and improve how we run these workshops in future.

Leave a comment below or get in touch with the User Research Team to find out more about their work.

Leave a comment