User research recruitment during a pandemic
In 2020, I took part in a private Beta project for the Department for Education (DfE) which was focused on building a service to help Schools Buying Professionals (SBPs) to plan and fill out a specification for a new catering contract for their school or trust. I was brought onto the project to support the Research team in the form of a Research Operations role, helping the team with certain aspects of the research process, such as recruitment, governance and data management, so that our researchers had the time and headspace to do their best work.
As the project took place during the COVID-19 outbreak, everything was handled remotely. It’s strange to say that we’re almost ‘used’ to this now having developed new ways of working over the past year. However, we did face particular challenges with recruitment as we had to frequently adapt our recruitment plan to align with the COVID rule changes, which were having a significant impact on our users. We had to ensure we were extra mindful when approaching our users as many schools were going through a difficult and unprecedented time.
Who were our users?
The primary users for our service are people who buy/procure catering contracts for their school (Schools Buying Professionals). Other roles that may fall within this category include:
- Estates Manager
- Chief Financial Officer / Finance Officer
- Schools Buying Manager / Finance / Business Manager
- Trust Finance Manager
- Procurement Officer / Manager
- Director of Operations
- School Business Leader
Our scope was focused on primary and secondary schools in England. We were especially interested in users situated within DfE’s 12 opportunity areas i.e. locations that are part of the government’s national plan for dealing with social mobility through education.
In the Alpha phase of the project, we predominantly spoke with users who had used a DfE service in the past to help them buy for their school, mainly from the North West and South West regions. Therefore in this private Beta, we wanted to prioritise people who hadn’t used a DfE service before, and so we initially looked for ‘unengaged’ users in the North East and South East regions. This meant we couldn’t use recruitment lists from the DfE (except as a reference to see which schools were already engaged with DfE’s services) and that we had to ‘free find’ participants ourselves.
At the time of starting recruitment, the UK went into the COVID Tier systems; tier 1 - medium alert; tier 2 - high alert; tier 3 - very high alert. We were advised by the DfE to avoid contacting schools in tier 3 locations as they may be under increased burden e.g. implementing remote learning, cancelling exams, organising extra cleaning, purchasing extra PPE and all the stress that comes with these rapid changes. These tiers were updated every 2 weeks, so we had to be mindful about which tier each school was in before we contacted them, and how the evolving situation may impact our project.
Set up for remote research
Regarding processes, the set up wasn’t greatly affected by having to do everything remotely. Our user group works with digital tools and has access to the necessary tech to run remote user research sessions. We set up booking links using Calendly so that participants could choose a suitable date and time to book in a session with us.
To find our participants, I used the Get information about schools service in order to ‘free find’ relevant users. I focused on filtering primary and secondary schools in our opportunity areas first, and then worked my way down the country. If I couldn’t find the SBP’s email on the school website, then I would send an email to the school’s office and ask for it to be forwarded to the right person. I made sure to include all the necessary information and booking link in one email so they didn’t feel I was pestering them with multiple emails.
It was quite a time consuming process and I only had a 2% success rate of getting people booked in. It was especially difficult during half term breaks or if I had less than 2 weeks to find users as their calendars were usually already full. To try and resolve this, I asked the team if I could schedule the research dates for the first 4 sprints so that users could pick dates over the next couple of months, and thankfully this method worked a lot better.
2021 COVID restrictions
During the Christmas break England introduced tier 4, and by the time we returned to the project, the whole country was in either tier 3 or 4. Shortly after, lockdown 3.0 came into effect with schools having to close and move to remote learning, and secondary schools having to implement the roll out of COVID testing. This increased burden on schools meant we were completely blocked for recruitment. The DfE later announced that all in-progress or planned user research should pause their recruitment and await new guidance from the DfE on how to proceed.
The situation was changing rapidly, so whilst we waited for the DfE’s official response, we decided to do some scenario planning so that we were prepared for any eventuality.
Scenario Planning
Scenario planning is a way to think and talk about the future whilst acknowledging that you can’t predict it. We used the following headings to plot out our scenarios:
- Choose an area to focus on (e.g. there’s a national lockdown and schools move to remote learning)
- How far ahead are you planning (e.g. until the end of the lockdown, mid-Feb 2021)
- What is the expected service model/how should it work usually (e.g. there are no tier restrictions or increased pressure on our users, so we have free reign of who we can contact)
- Develop a plausible scenario (e.g. research and recruitment is delayed until mid-Feb)
- Evaluate the probable outcome (e.g. development of the service continues without validated user needs)
- Create a preferable plan and update your strategy (e.g. we prioritise research in the remaining sprints to test what has been developed and ask for an extension to mitigate the delay and allow for development iterations)
Image: An overview of our scenario planning on Miro (it took us a few goes before we came up with our final version)
Shortly after we prepped our scenario plans, the DfE’s official response requested all projects would need to submit a research plan for review. We presented our plans and our delivery team was approved to continue recruitment but with additional care and mindfulness when approaching schools.
Adapted recruitment plan
Following the DfE’s approval, we came up with a renewed strategy for recruitment and developed a mindfulness plan which included:
- Assessing each school’s situation before we contact them as best we can
- This proved to be very difficult. In the end, we decided to only contact primary schools for the next sprint as secondary schools had the increased pressure of having to implement COVID testing for the first time.
- Offering more flexible times for people to speak to us
- Looking at our email format with our content designers to tweak any language and see what can be condensed
- Streamlining the booking in process as much as possible for participants, e.g. sending as few emails as possible; providing as many research dates as possible for them to choose from; allowing them to cancel last minute; we take on the burden of any rescheduling admin etc.
- Our research team also adapted their scripts and ways of working to ensure that the participants’ and researchers’ wellbeing was looked after throughout the research process.
Our first strategy was to continue our free find approach, but also make use of the DfE’s recruitment list of users that had previously engaged with their services in Alpha (from the North & South West regions). My assumption was that people who were already engaged with DfE’s services would provide a better response rate than non-engaged people had so far (any potential biases would be taken into account during the research). Fortunately my assumption was correct and we successfully filled a lot of our research slots.
Our secondary strategy was to engage with a recruitment agency to understand if they were able to conduct recruitment of our primary users on the delivery team’s behalf. This meant that we could offer incentives to help boost our response rates further. It was also needed as my budgeted time on the project was running out and we wanted to ensure the recruitment would be maintained for the last few sprints.
Takeaways
Here are a few things I took away from this experience:
- Scenario planning is very helpful in fluid situations. It helped me to focus on something during uncertain times and I felt prepared and ready to go once we could proceed.
- Free finding users is quite time consuming and having a recruitment agency ready and waiting as a backup is a good idea. I also found providing participants with research dates for the next few months worked really well for this particular project and user.
- It was really helpful that the research team had weekly catch ups with DfE’s research stakeholders so that they were always aware of what we were up to and we had regular access to them when blockers popped up. Building this relationship with them throughout the project definitely helped it run smoother and I’d recommend it for future projects.
- It’s a strange time and it’s OK not to be OK. Being more mindful, empathetic and creating as safe a space as possible for our participants (and ourselves) is a good start to helping us all feel more comfortable in our research.
Follow Paper on LinkedIn