Strategies for success: Usability testing & user research of government websites and public services

It used to be that testing websites was exclusively done in the private sector. Conversely, in the public sector it was perceived as an unnecessary cost and often fell between the responsibilities of different teams. Today, that perception is changing. This article will detail strategies for success for your next project.

By conducting usability testing & user research you will not only learn how usable your website/service is, but the research will ensure that you’re fulfilling your organization’s mandate. Research and testing has evolved from simply indicating if the design of a page is functional – it can pinpoint how to create more meaningful relationships with the public and your clients. 

Next time you test your website/service, consider using that project/budget as an opportunity to gain insights about greater challenges that have strategic value across your entire organization.

Here are a few example projects of how our team at PH1 Research have expanded usability testing into opportunities to learn critical insights about the mandates of public sector organizations:

Legal Aid

• Mandate: To provide legal aid to the public

• Testing: Ensure that legal aid services are easy to understand, accessible, and that barriers to entry are identified

QuitNow

• Mandate: To help you quit or reduce tobacco and e-cigarette use

• Testing: Identify how to best support users across each stage of their journey to quit

University of Waterloo

• Mandate: To support excellence in education through leadership, collaboration, and shared expertise

• Testing: Improve the information and support offered to high school students across each stage of learning about and attending university

Translink

• Mandate: To connect the region and enhance its livability by providing a sustainable transportation network

• Testing: Ensure that trip planning tools are accessible by all, including those who are not familiar with the region and those with accessibility challenges

Key considerations: Public vs. private sector

Much of the content you’ll find online and most of the vendors offering usability testing services focus on the private sector.  Those firms, their methods, and their approaches don’t address the special considerations of public sector projects. 

The primary reason is that for-profit websites/services are designed to serve their most profitable users, whereas in the not-for-profit sector the goal is to support everyone. This means that those corporations and their vendors look at their users from the perspective of who will make them the most money and building to their needs, rather than those whose voices need to be heard.

Public sector organizations have a responsibility to address the needs of the many, not the exclusive few. If your research followed that model you’d disregard the voices of people from smaller communities, those who are new to the country, and the people who need help to use your services.

COVID further amplified this divide between the private and public sector services. Where clients of public services and crown corporations once could visit in-person offices, they were forced to transition to digital.  

Strategy #1: Understand your bubble before trying to empathize with others

It is impossible to build effective services without understanding how and why the public need those services. This is harder to do than you imagine since your teams typically live in major cities, tend to be university educated, and likely have not been marginalized.

Empathizing with the people you want to help begins by recognizing our own bubbles. The risk is that we bring our bias, judgement, and false assumptions into the project and predetermine the outcome. We all believe we’re capable of self-awareness and critical thinking —I encourage you to take these additional steps to assure it.

Begin every project with a blank slate, with humbleness and a desire to learn. Find out who in your organization has the most interactions with clients and the public. They likely come from a variety of teams; some may operate phone lines, others work in community offices, and some come from those marginalized communities themselves. Your goal is to learn as much as possible about the breadth of backgrounds, questions asked, and motivators of clients and the public.

Their perspectives will help you begin to create your project roadmap, one that details who to recruit, what to test, why to test, and what the organization must achieve to meet its mandate.

Strategy #2: Recruit a truly representative audience

COVID exposed a major weakness of many public sector organizations: they have under-invested in their digital services. 

With offices shut down for much of 2020, organizations quickly had to adapt to the reality that their websites are essential services. Downtime, steep learning curves, and perceived barriers to entry could harm people. 2020 shows that all citizens had to have equitable access, regardless of their location, technical ability, or english language proficiency. 

For many organizations this transition went poorly because they had been following the private sector model of researching the habits/needs of primary users only.

2021 is the start of an age of renaissance for digital services. 

The organizations that want to embrace it have the opportunity to redefine their positions in communities by providing inclusive, accessible, secure, and universal services at scale. You’ll be able to engage remote communities and support those with low english language proficiency in ways that would have been unsustainable before. Alternatively, the organizations that don’t embrace this change will face a wave of public critique as expectations rise, and  standards evolve to become law.

Mitigate future risk by seizing the opportunity to elevate your research today. Start by ensuring every development project is based on and validated with a representative audience of users. Use what you learned from Strategy #2 to find the silent majority and the disenfranchised audiences. Recruit from across the geography you serve and from the many segments who rely on you.

I recommend recruiting audiences who make at least 5% of your client base, or at least 2% if you are feeling ambitious.

Strategy #3: Test participants who address the different reasons for using the product/service

As your recruitment strategy evolves to speak to a representative audience, so too must your testing strategy. 

Remember this: It may only be for a split second, but your product/service improves the lives of your users. Testing has been so obsessed with whether or not users can complete tasks that it fails to ask about the entire customer journey and the emotion of using your service. 

Ensure your tests engage the public when and how they use the product, not only under ideal circumstances. This means looking for testers who have experienced imperfect circumstances or simulating them:

• Find your most dissatisfied users and retrace their steps

• Challenge assumptions about how/why the public find out about the product/service

• Explore situations when users are under extreme stress or have acute problems

• Evaluate what happens when they have no recall of your brand or service

• Simulate poor service and document what they do

Considering and evaluating diverse contexts will change how you look at your website/service. As you start learning how many of your clients are seeking support in times of crisis/frustration, it will shift your information architecture and content strategy. And as you learn what motivates the decision to use your service, the more you can contextualize solutions and support.

Strategy #4: Expect to walk away with more questions than answers

This approach will yield wildly unexpected results, some of which will rattle your organization. No one wants to hear that their brand, information architecture, and communication strategy are undermining your website/service. That is what will happen for some of you and you should be proud of it. Websites/services should not be looked at monolithically, where they are either successful or not. They must constantly learn, adapt, and evolve. That’s why this research is critical.

Before you begin planning your next usability testing or user test project consider the risk of not asking these questions. Continuous improvement processes like these are the foundation of the world’s most innovative organizations and help create more anti-fragile websites/services.

Thank you for your time. If you have any questions feel free to contact me: arpy@ph1.ca or via http://PH1.ca. Our firm evaluates your CX & UX to improve conversions, usability, and inclusiveness.