Search

User research in Beta stage

In the Beta stage, you’ll learn ways to improve your service. Including different kinds of experiences users may have, including usability and accessibility issues.

There are 2 different research stages in Beta: 

1. Researching as you build the beta service. 
2. Researching with users of a working beta service. 

Private Beta research  

As you build out your Beta service, you'll continue to do task-based usability testing with a range of users. This time, you are aiming to refine a solution for launch.

When you do the task-based usability testing:  

• decide on a hypothesis, this is how you think the design will meet a user need 
• use a structured approach to evaluate, this will make it clear what work needs to be done 
• test with people who have different needs.

Public Beta research  

In Public Beta you will continue with your research and explore the information and data you gained from your public users. 

In this stage you will conduct interviews and feedback from users and front line staff and do user shadowing.

You will also 

• create 2 versions of the service to see which performs better, this is called A/B testing
• collect analytics and key performance indicators
• do audits including usability and accessibility tests 
• collect operational data to measure service performance, such as customer service insights  
• do follow-up interviews to collect detailed feedback from service users.

User research in Live stage 

In the Live stage you will test and collect feedback to make sure your service meets user needs. These research activities will build your understanding of common issues to help you improve the service. 

You can learn more about user needs through: 

• web analytics and operational data  
• surveys or follow-up interviews  
• face-to-face and remote usability tests  
• A/B testing on new and changed features. 

The frequency and depth of the research you do in Live depends on what you are trying to find out. For example, you should start the Discovery process again before adding a new component to your service. 

Sort observations 

When you’re ready, ask your team to place their observations on a wall or virtual canvas. As a group you will work to sort your observations into similar themes.  

The idea is to look for patterns or clusters in the data by grouping the information until clear themes emerge. You can group by:

• common topics, for example identity, delivery, payment 
• stages in a user journey, for example ‘supply photo’, ‘attend interview’, ‘pay’ 
• individual pages or steps in a transaction 
• types of users, for example, first time user, business user. 

Name your groups 

Once you’ve sorted your observations, agree on a title that represents the cluster. From there, you can break large groups into smaller themes by matching observations.

For example, if users need to supply a photo to use your service, you might have a ‘photos’ group that could broken down into:

• photo rules and requirements 
• using a photo booth or store photographer 
• taking a photo at home 
• reasons a photo might be rejected. 

Determine findings 

The final part of the analysis is determining what the observations mean. When you agree on what you’ve learned, write it as a finding or insight and add it to the relevant group on your affinity map. Write findings as short statements that summarise what you’ve learned, for example:

• ‘the legal declaration is threatening and difficult to understand’
• ‘people think they can click the progress bar to navigate’ 
• ‘users are confused about what they need to do because so many questions are optional’.

Confirm the actions 

Use your findings to make decisions about what to work on or change. This supports the agile method of continuous planning with new facts or requirements. As a group, discuss if there are any actions you want to take. Write these on sticky notes in another colour. Add them to the relevant group on your affinity map.  

Actions might include:

• new design ideas to work on 
• new questions to include in user research 
• things you want to change in a prototype and test in another research session 
• new user stories to add to the product backlog 
• new details you need to add to an existing story 
• strategic insights you can use to develop your user needs, proposition or product roadmap. 

Share your findings 

Collate your findings so you can share them with the wider team and stakeholders. This is sometimes called a shareback.

You can share insights in different ways. If you've been testing prototypes you might show printouts with comments on sticky notes. If you've only just started, you might read out quotes and observations.

Use an electronic presentation to share your findings or whatever medium suits your team. 

Using a consent form in an interview 

You should start interviews with users by explaining the purpose of the research. Show participants the consent form. 

Explain that they have a choice about what they wish to consent to. Get permission before starting any form of recording, audio, visual or written. 

Sometimes a participant may say something that they don’t feel comfortable sharing. After the interview, ask them again if they are happy for the conversation to be used as part of the research. Make sure they still consent to you using the information. 

Leave a copy with the participant 

Leave a copy of the consent form with the participant at the end of the session. This gives them a record of what they have agreed to. It also lets them know how they can withdraw consent if they want to later. 

Incentives and consent forms

Consent forms and incentive receipts are separate parts of the research process.

Make sure you treat consent as a separate discussion from giving the incentive payment and the receipt, otherwise participants may feel a financial obligation to consent to the research.

Keep personal information secure

There are rules around how the government can use personal information.

You need to make sure the information you collect is secure from misuse or unauthorised access. 

Help people to share their story widely 

It’s important to allow people to protect their privacy. It's also important to help people to share their stories more widely. Many people are very keen for their experiences with governments to be shared with anyone who can help improve them.