Since March 2020, we have been living in challenging times. For the past four months the COVID-19 pandemic has forced a huge variety of professions to pivot and scramble in order to cope with the fast changing health and economic landscapes that we now operate in. The relaxation in social distancing and drivers to get economies back on their feet again are taking place at different rates across the world, depending on a country’s individual circumstances as it relates to the decline of infection and mortality rates.
Travelling through such challenging times, we believe the collective and growing challenge to engage, understand, collaborate with and better serve citizens, communities, businesses and government in crisis situations (e.g. COVID-19 pandemic, bushfires, droughts, floods and other disasters) is one that has to be met through digital innovation.
As an international digital transformation practice with experience in designing and delivering national programmes for both Australian and UK Governments, Informed Solutions’ team of user researchers are well-versed in executing remote research including; user interviews and moderated and unmoderated usability testing, with teams on both sides of the world connecting and engaging with people in remote locations and different time zones.
With remote and colocation hybrid working patterns set to continue for the foreseeable future, our teams in the UK and Australia share thoughts on the critical, early stages of service design around successful remote user and stakeholder engagement as a platform for successful digital service delivery.
Testing Environments
Executing research purely online, the researcher has little control over the user’s environment except through the online conferencing tools that are available and accepted. Just coming out of lockdown in Australia, Informed Solutions was part of a user group undertaking a combination of guerrilla and remote user testing to better understand the user experience of citizens using digital identity to apply online for government welfare during COVID-19 instead of physically proving their identity by visiting a welfare services shopfront.
Taking a mobile app for the purposes of usability testing into the public domain and asking passers-by for feedback had its challenges - we had little control over the environment, however we identified the geographic areas for the project ahead of time and prepared for our different approach and the difference in acceptance of passers-by in both city centre and more remote community locations. Additionally, although the testing was carried out with full compliance to social distancing and sanitisation/cleanliness guidance, peoples’ attitudes to risk differed and so the speed and efficiency of feedback capture varied too.
The guerrilla testing moved quickly to remote user research as full lockdown was confirmed, which introduced both challenges and benefits with regard to the ease and effectiveness of conducting research solely through digital channels.
Engagement
For remote user research, it’s really important to be interested and engaged with your participant for any research, but especially around sensitive and challenging COVID-related issues. When you’re on a video call it can be challenging to pick up on the micro expressions and social cues you would normally and relatively easily identify in face-to-face settings. You’ll also find you have to be more engaged than usual to make an authentic connection. It can be difficult to empathize with people over an online connection, so actually seeing the person on the other side of a remote research call has a significant impact on your ability to connect with people, gain a decent measure of engagement and secure meaningful feedback on service design.
From experience, participants will give more thoughtful responses and be open to conversations that delve deeper into issues if they can see who they are speaking with. Added to this, with people working from home some may be more open with us if they feel more relaxed and abstracted from the workplace and the pressures that brings.
When cameras are off, the act of capturing accurate feedback and responses becomes even more challenging: User research depends not only on what users say, but how they say it combined with a physical, body language-related reaction. If you can't observe someone's facial expressions in order to understand their emotional response to a question or a particular piece of information, user feedback can become somewhat of a guessing game – judging silence or pauses in a dialogue against thoughts or emotions that might vary all the way from confusion to discomfort.
Technology and Preparation
Another interesting challenge for remote user testing relates to observing a user’s experience navigating a digital service through a variety of devices to fully understand the end-to-end user experience through a particular web service or device.
In face-to-face settings, user researchers are armed with task cards that can help the flow of a session. In remote, online settings, researchers have to prepare digital task cards and work out timings in order to best present when needed. Added to this, the researcher will have to take screen size and screen real estate into consideration for a user to be able to view both the test prototype and the task card. The tech should not get in the way of the user experience.
Outside of channel shifting, we allowed extra time to test the tech and conduct dry runs to ensure everything ran smoothly, including order and timing for feedback completion as well as screen sharing and control functions that allow a user to navigate a prototype system effectively.
Widened Participation
Alongside the different challenges associated with online user research there are also benefits including; the ability to recruit for a wider geographic spread of participants - particularly those living remotely who often have different user needs to those that are easy to access. This change to ‘all remote’ research really drives you to have higher engagement with this group, whereas in ‘normal’ times there may be a tendency to concentrate on those users that are easier to recruit – another positive of the times.
Additionally, when you’re working with an agile team, remote research allows you to fit many user sessions into a single day and iterate quickly. It can be an extremely valuable addition to your research tool belt. Although a positive, it presents the researcher with an interesting balancing act as researchers might be expected to speak with more people, however you still need to spend the same amount of time doing analysis and downloads of findings. Even though you might be able to fit more slots in, you often can’t actually engage with a huge increase of people if the requirement is to derive meaningful insights from each session - something that a user researcher always needs to be aware of in terms of structuring a round of research properly.
Analysis is Key
There are many tools out there to help you conduct remote user research. But collecting data is the easy part, what you do with it is vital, especially when it relates to the collection of information concerning citizen welfare. Research alone provides data, not insights. The key to making any piece of research a success is how you use that data to generate actionable, prioritised recommendations that produce effective online services
Capturing this type of data allows us to accurately articulate the complex problems that require solving and most importantly work collaboratively with project stakeholders to align user and organisational needs. This alignment and putting people first is what helps ensure that a digital design succeeds the first time around.
As the winner of a Queen’s Award for Innovation, we work with some of the world’s largest and most respected organisations, delivering digital services that are used by millions of people every day. The critical first steps outlined in this piece provide a reliable foundation for our digital transformation projects. They allow us to progress onto designing, building, testing and delivering optimal solutions that users are satisfied with and continue to use with ease. Our thinking is always shared across Informed Solutions’ Communities of Practice allowing us to share, shape and refine best practice for remote user research tools and techniques across our international operation.
