Tips & lessons
Strategies for success: Usability testing & user research of government websites and public services
– By Arpy Dragffy
It used to be that testing websites was exclusively done in the private sector. Conversely, in the public sector it was perceived as an unnecessary cost and often fell between the responsibilities of different teams. Today, that perception is changing. This article will detail strategies for success for your next project.
By conducting usability testing & user research you will not only learn how usable your website/service is, but the research will ensure that you’re fulfilling your organization’s mandate. Research and testing has evolved from simply indicating if the design of a page is functional – it can pinpoint how to create more meaningful relationships with the public and your clients.
Next time you test your website/service, consider using that project/budget as an opportunity to gain insights about greater challenges that have strategic value across your entire organization.
Here are a few example projects of how our team at PH1 Research have expanded usability testing into opportunities to learn critical insights about the mandates of public sector organizations:
Mandate: To provide legal aid to the public
Testing: Ensure that legal aid services are easy to understand, accessible, and that barriers to entry are identified
Mandate: To help you quit or reduce tobacco and e-cigarette use
Testing: Identify how to best support users across each stage of their journey to quit
University of Waterloo
Mandate: To support excellence in education through leadership, collaboration, and shared expertise
Testing: Improve the information and support offered to high school students across each stage of learning about and attending university
Mandate: To connect the region and enhance its livability by providing a sustainable transportation network
Testing: Ensure that trip planning tools are accessible by all, including those who are not familiar with the region and those with accessibility challenges
Key considerations: Public vs. private sector
Much of the content you’ll find online and most of the vendors offering usability testing services focus on the private sector. Those firms, their methods, and their approaches don’t address the special considerations of public sector projects.
The primary reason is that for-profit websites/services are designed to serve their most profitable users, whereas in the not-for-profit sector the goal is to support everyone. This means that those corporations and their vendors look at their users from the perspective of who will make them the most money and building to their needs, rather than those whose voices need to be heard.
Public sector organizations have a responsibility to address the needs of the many, not the exclusive few. If your research followed that model you’d disregard the voices of people from smaller communities, those who are new to the country, and the people who need help to use your services.
COVID further amplified this divide between the private and public sector services. Where clients of public services and crown corporations once could visit in-person offices, they were forced to transition to digital.
Strategy #1: Understand your bubble before trying to empathize with others
It is impossible to build effective services without understanding how and why the public need those services. This is harder to do than you imagine since your teams typically live in major cities, tend to be university educated, and likely have not been marginalized.
Empathizing with the people you want to help begins by recognizing our own bubbles. The risk is that we bring our bias, judgement, and false assumptions into the project and predetermine the outcome. We all believe we’re capable of self-awareness and critical thinking —I encourage you to take these additional steps to assure it.
Begin every project with a blank slate, with humbleness and a desire to learn. Find out who in your organization has the most interactions with clients and the public. They likely come from a variety of teams; some may operate phone lines, others work in community offices, and some come from those marginalized communities themselves. Your goal is to learn as much as possible about the breadth of backgrounds, questions asked, and motivators of clients and the public.
Their perspectives will help you begin to create your project roadmap, one that details who to recruit, what to test, why to test, and what the organization must achieve to meet its mandate.
Strategy #2: Recruit a truly representative audience
COVID exposed a major weakness of many public sector organizations: they have under-invested in their digital services.
With offices shut down for much of 2020, organizations quickly had to adapt to the reality that their websites are essential services. Downtime, steep learning curves, and perceived barriers to entry could harm people. 2020 shows that all citizens had to have equitable access, regardless of their location, technical ability, or english language proficiency.
For many organizations this transition went poorly because they had been following the private sector model of researching the habits/needs of primary users only.
2021 is the start of an age of renaissance for digital services.
The organizations that want to embrace it have the opportunity to redefine their positions in communities by providing inclusive, accessible, secure, and universal services at scale. You’ll be able to engage remote communities and support those with low english language proficiency in ways that would have been unsustainable before. Alternatively, the organizations that don’t embrace this change will face a wave of public critique as expectations rise, and standards evolve to become law.
Mitigate future risk by seizing the opportunity to elevate your research today. Start by ensuring every development project is based on and validated with a representative audience of users. Use what you learned from Strategy #2 to find the silent majority and the disenfranchised audiences. Recruit from across the geography you serve and from the many segments who rely on you.
I recommend recruiting audiences who make at least 5% of your client base, or at least 2% if you are feeling ambitious.
Strategy #3: Test participants who address the different reasons for using the product/service
As your recruitment strategy evolves to speak to a representative audience, so too must your testing strategy.
Remember this: It may only be for a split second, but your product/service improves the lives of your users. Testing has been so obsessed with whether or not users can complete tasks that it fails to ask about the entire customer journey and the emotion of using your service.
Ensure your tests engage the public when and how they use the product, not only under ideal circumstances. This means looking for testers who have experienced imperfect circumstances or simulating them:
Find your most dissatisfied users and retrace their steps
Challenge assumptions about how/why the public find out about the product/service
Explore situations when users are under extreme stress or have acute problems
Evaluate what happens when they have no recall of your brand or service
Simulate poor service and document what they do
Considering and evaluating diverse contexts will change how you look at your website/service. As you start learning how many of your clients are seeking support in times of crisis/frustration, it will shift your information architecture and content strategy. And as you learn what motivates the decision to use your service, the more you can contextualize solutions and support.
Strategy #4: Expect to walk away with more questions than answers
This approach will yield wildly unexpected results, some of which will rattle your organization. No one wants to hear that their brand, information architecture, and communication strategy are undermining your website/service. That is what will happen for some of you and you should be proud of it. Websites/services should not be looked at monolithically, where they are either successful or not. They must constantly learn, adapt, and evolve. That’s why this research is critical.
Before you begin planning your next usability testing or user test project consider the risk of not asking these questions. Continuous improvement processes like these are the foundation of the world’s most innovative organizations and help create more anti-fragile websites/services.
View More Resources
Strategy, Travel, Lessons10 business lessons we learned travelling around the world
Trends, Strategy, Design2016 Design trends: Do not fear the competition, no matter how big they are.
Trends, InnovationWin Audiences in 2016: 6 Strategic Tips for Entrepreneurs and Business Leaders
Strategy, Gamification, MarketingGuide to gamification: strategies to increase engagement and make your audience feel special
Tips4 ways we helped entrepreneurs and business leaders around the world
Innovation, Strategy, DisruptionThe disruption no one is talking about
Innovation, AssessmentYour business model will be disrupted if this describes your organization
AI, Innovation, Design ThinkingThe key to solving problems and designing solutions that an AI can’t: Irrational behaviour
Innovation, Future-proofingThanks to Google, the end of the mobile app is near
Innovation, Engagement, FutureForget VR — Mobile cameras are the greatest innovation your business must embrace
Travel, Tourism, TrendsAsia-Pacific travellers: about to transform the global tourism industry
Article, Innovation, CompetitionIt’s time to realize you’re already competing with Amazon, Google, Facebook, Apple
Not only is health care challenged by aging, so is the way of life of rural citizensJapan and North America's Aging Problem that startups must address
CX, UX, ResearchInnovating your organization begins by understanding your customer experience
CX, UX, ResearchBecoming customer-centric: What most organizations get wrong
Research, Public Health, StrategyOur personal views about health are changing the way we want to be engaged
CX, ToolkitsBecoming customer-centric: 3 toolkits to measure your Customer Experience
COVID-19, Resources, ResearchPH1's research initiative to improve the CX of organisations impacted by COVID-19
BlackLivesMatter, AccessibilityResearchers can help end systemic racism that plagues products and services
Research, COVID-19, Lessons3 lessons: Becoming an adaptive & resilient organization
Public health guideInformation Architecture for Public Health Organizations
Tips & lessonsStrategies for success: Usability testing & user research of government websites and public services
Tips & lessonsRecruit the right audience for your research: Social media recruitment
Methodology and ExamplesIntro to Customer Experience
Tips & lessonsHow to create customer journey maps & journey mapping tips
Outcome of our annual planning retreatPH1 Research's 2022 Strategic Vision
Methodology and ExamplesIntro to Service Design
Methodology and ExamplesIntro to Innovation Sprints
UX, strategy, human-centred designUX needs to set the bar higher than usability
Innovation methodologyTurning Fandom into Revenue with Web3
About UsOur Epic Annual Remote Retreat: Colombia Edition
Innovation research guideGuide to researching new products & testing concepts
Innovation methodologyA guide to overcoming uncertainty impacting your product or service
Innovation methodologyFor More Impactful UX Strategy, Use Futures Thinking
Audit & analysisWeb3 has a major UX problem
Tips & lessons10 lessons about how to improve tech & your success as a researcher, designer, product manager
Strategy, Methodology, Service DesignPH1 Research's Service Transformation Methodology