May 9, 2023 - User experience testing

Back to Federal Provincial Territorial web management working group

Agenda

Updates

User experience testing results
- What are you doing to make the most of user testing?
- What is the trigger for you to run testing given challenges such as money, time, and resources?
- How do you go about setting up testing and what do you do with your results?
- Do you have anecdotes from testing for specific design needs?
- Do you have tools to share that you have or use that others may benefit from?
- Can you share findings, ideas, processes, or tools that might apply to others?

In attendance

Michèle-Renée Charbonneau, Digital Transformation Office, TBS (Government of Canada)
Kathleen McDonald, Indigenous Services Canada
Beth Doucette, Govt of PEI (IT Shared Services)
Kayte McLaughlin, Government of New Brunswick!
Allison Leonard, Government of Nova Scotia web team
Sarah Dunn, Alberta.ca UX
Jennifer Kowton, Communications and Public Engagement (CPE) at the Government of Alberta
Peter Smith, Digital Transformation Office, TBS (Government of Canada)
Wayne Bouwmeester, Province of Alberta
Lily Gontard, Online communications manager, Government of Yukon
Laura Piper, Digital Transformation Office, TBS (Government of Canada)
Simon Cossette, Principal Publisher, ESDC, Federal Gov
Chelsey Donohue, Digital Transformation Office, TBS (Government of Canada)

DTO updates

What’s next for FPT group: We can’t continue to chair this community and manage the work that goes into it. We are hoping some people from the community who see the value in this group may be able to take this over. If you are interested please let us know:

dto.btn@tbs-sct.gc.ca

There is also the International design in government community and slack channel where people share best practices in a less formal way.

Strike: The strike is now over and we are glad to have everyone back. We are still getting caught up from that.

Coronation content: Canadian celebrations of the Coronation

Alberta wildfires: Doing some work on web content related to the wildfires in Alberta.

User experience testing results

Government of Yukon

Program name change

Do user research if we change a program name
Recently changed the social housing program name to something more abstract
Have started doing some testing to better understand the language clients use so we can use that in our titles, headings and content

Major web revamps

If someone wants to do an overhaul of content they have to do the research first
Did research when we redid our website in 2018
Made UX testing a requirement - if someone wanted to move their content they had to do user research first
Do it in-house, not with contractors

Professional development

See research as part of professional development
Encourage web representatives to do Nielsen Norman training -
Very small group so everyone has the opportunity to work on research - it helps to slow down turnover and engage people
This keeps the team more engaged and gives them different files to work on
The more we do research, the more people are interested in it

Indigenous Services Canada

Triggers

Trying to shift the criteria that trigger user research away from focusing only on hot topics in our mandate or political issues, and more toward maintenance activites

Using analytics and workshops

Sharing our analytics platform with our program and policy people so they can see what is happening in their web content
- That gets them asking why things aren’t working the way they thought they would

Do collaborative workshops with stakeholders to make sure the smaller web program content is informed by data as well

Digital Transformation Office

Passive data collection

We have two new(ish) inputs that help us decide whether to pursue in-depth user research:
- signals from our ongoing task success survey
- signals arising from on-page user feedback

We monitor inputs so we can gather evidence to build a case as to why we should do more research later

Government of Alberta

Ambassadors

Alberta.ca has a team of 3 web designers
Conduct a lot of research on our website and then work with the content team
Have a working group across the sector with 15 people who cross-train and learn about UX and then implement the processes in their own situations
This gives us more people who advocate with clients about the importance of UX

Presentation from Sarah Dunn - Alberta.ca UX

Presentation: Alberta.ca user research program

Focus on:

Approvals
Discovery
Recruitment
Interviews
Analysis

Approvals

Takes about 9-12 weeks to run a UX project
We did a services deep dive project to see how services were being represented on the web
Worked with Alberta Public Service to do testing of recruitment content and jobs board
Did testing on COVID
Get pushback about testing when people are afraid it will create a political situation
- Create a research brief to help programs brief their stakeholders/management to get buy in
- If we get buy-in we work directly with their content strategist

Recruitment

Decide what kind of people we want to user test with (ie. audience, relationship to program, age, gender, location, etc.)
Methods for finding participants:
- Alberta.ca fly-ins
- Newsletter/listserv
- Stakeholder
- Social media
- Pop-ups on the website targeted to specific demographics, which will give us upwards to 200 responses

Before adding a recruiter pop-up to a page, we let the communications directors know to make sure there’s no contentious issues with that page
Decide on participants and then communicate the project to them
Have cut down on no-shows in testing by offering incentives ($25 Amazon gift cards)
- Got support from leadership
- No-shows and trouble with scheduling was really holding things up
- Looked at what that time was worth and were able to get some budget assigned to user testing for gift cards so we could get people to sit down for an actual session
Always so happy to see how many people are willing to sit down and help make the website better

Interviews

We usually lose about 1/3 of the people between recruitment and interviews
Sessions are between 45 to 60 minutes
Participants share their screen with us as we interview and observe
The script for testing includes an intro and a consent statement that we get them to agree to before we jump into the interview and tasks

Analysis

We go through what we heard from participants using Miro
The sticky notes are all comments taken from our usability sessions
We group common comments together
At the end we present a findings report where we summarize what we learned and provide recommendations

What we’ve learned so far

We have conducted over 100 hours of usability testing on Alberta.ca and its prototypes
Frequent users have a very particular way of navigating
- If they have to stray from that they get lost
- People want a clear place to start
- People have trouble moving laterally through related content

Heard things like “Where is the home button”
Visual aids help organize and break up content to keep it from being overwhelming
Irrelevant content becomes very frustrating very quickly
Accordions tested very well
- People like being able to see a lot of information summarized that they can expand as needed

Users want a website to be be useable
People were skipping our navigation tools
People don’t go looking for what they don’t know exists
Users want help understanding how what they read on the website applies to their lives

Recruiting for accessibility testing

Communications and Public Engagement (CPE) at the Government of Alberta has an Office for accessibility where we are trying to recruit people with accessibility issues for testing
DTO is trying to recruit for this right now but it is difficult