May 9, 2023 - User experience testing

Back to Federal Provincial Territorial web management working group

Agenda

• Updates

• User experience testing results
  • What are you doing to make the most of user testing?
  • What is the trigger for you to run testing given challenges such as money, time, and resources?
  • How do you go about setting up testing and what do you do with your results?
  • Do you have anecdotes from testing for specific design needs?
  • Do you have tools to share that you have or use that others may benefit from?
  • Can you share findings, ideas, processes, or tools that might apply to others?

In attendance

• Michèle-Renée Charbonneau, Digital Transformation Office, TBS (Government of Canada)
• Kathleen McDonald, Indigenous Services Canada
• Beth Doucette, Govt of PEI (IT Shared Services)
• Kayte McLaughlin, Government of New Brunswick!
• Allison Leonard, Government of Nova Scotia web team
• Sarah Dunn, Alberta.ca UX
• Jennifer Kowton, Communications and Public Engagement (CPE) at the Government of Alberta
• Peter Smith,  Digital Transformation Office, TBS (Government of Canada)
• Wayne Bouwmeester, Province of Alberta
• Lily Gontard, Online communications manager, Government of Yukon
• Laura Piper, Digital Transformation Office, TBS (Government of Canada)
• Simon Cossette, Principal Publisher, ESDC, Federal Gov
• Chelsey Donohue, Digital Transformation Office, TBS (Government of Canada)

DTO updates

What’s next for FPT group: We can’t continue to chair this community and manage the work that goes into it. We are hoping some people from the community who see the value in this group may be able to take this over. If you are interested please let us know:

• dto.btn@tbs-sct.gc.ca

There is also the International design in government community and slack channel where people share best practices in a less formal way.

• International design in government community website
• International design in government Slack channel

Strike: The strike is now over and we are glad to have everyone back. We are still getting caught up from that.

Coronation content: Canadian celebrations of the Coronation

Alberta wildfires: Doing some work on web content related to the wildfires in Alberta.

User experience testing results

Government of Yukon

Program name change

• Do user research if we change a program name
• Recently changed the social housing program name to something more abstract
• Have started doing some testing to better understand the language clients use so we can use that in our titles, headings and content

Major web revamps

• If someone wants to do an overhaul of content they have to do the research first
• Did research when we redid our website in 2018
• Made UX testing a requirement - if someone wanted to move their content they had to do user research first
• Do it in-house, not with contractors

Professional development

• See research as part of professional development
• Encourage web representatives to do Nielsen Norman training -
• Very small group so everyone has the opportunity to work on research - it helps to slow down turnover and engage people
• This keeps the team more engaged and gives them different files to work on
• The more we do research, the more people are interested in it

Indigenous Services Canada

Triggers

• Trying to shift the criteria that trigger user research away from focusing only on hot topics in our mandate or political issues, and more toward maintenance activites

Using analytics and workshops

• Sharing our analytics platform with our program and policy people so they can see what is happening in their web content
  • That gets them asking why things aren’t working the way they thought they would

• Do collaborative workshops with stakeholders to make sure the smaller web program content is informed by data as well

Digital Transformation Office

Passive data collection

• We have two new(ish) inputs that help us decide whether to pursue in-depth user research:
  • signals from our ongoing task success survey
  • signals arising from on-page user feedback

• We monitor inputs so we can gather evidence to build a case as to why we should do more research later

Government of Alberta

Ambassadors

• Alberta.ca has a team of 3 web designers
• Conduct a lot of research on our website and then work with the content team
• Have a working group across the sector with 15 people who cross-train and learn about UX and then implement the processes in their own situations
• This gives us more people who advocate with clients about the importance of UX

Presentation from Sarah Dunn - Alberta.ca UX

Presentation: Alberta.ca user research program

Focus on:

• Approvals
• Discovery
• Recruitment
• Interviews
• Analysis

Approvals

• Takes about 9-12 weeks to run a UX project
• We did a services deep dive project to see how services were being represented on the web
• Worked with Alberta Public Service to do testing of recruitment content and jobs board
• Did testing on COVID
• Get pushback about testing when people are afraid it will create a political situation
  • Create a research brief to help programs brief their stakeholders/management to get buy in
  • If we get buy-in we work directly with their content strategist

Recruitment

• Decide what kind of people we want to user test with (ie. audience, relationship to program, age, gender, location, etc.)
• Methods for finding participants:
  • Alberta.ca fly-ins
  • Newsletter/listserv
  • Stakeholder
  • Social media
  • Pop-ups on the website targeted to specific demographics, which will give us upwards to 200 responses

• Before adding a recruiter pop-up to a page, we let the communications directors know to make sure there’s no contentious issues with that page
• Decide on participants and then communicate the project to them
• Have cut down on no-shows in testing by offering incentives ($25 Amazon gift cards)
  • Got support from leadership
  • No-shows and trouble with scheduling was really holding things up
  • Looked at what that time was worth and were able to get some budget assigned to user testing for gift cards so we could get people to sit down for an actual session
• Always so happy to see how many people are willing to sit down and help make the website better

Interviews

• We usually lose about 1/3 of the people between recruitment and interviews
• Sessions are between 45 to 60 minutes
• Participants share their screen with us as we interview and observe
• The script for testing includes an intro and a consent statement that we get them to agree to before we jump into the interview and tasks

Analysis

• We go through what we heard from participants using Miro
• The sticky notes are all comments taken from our usability sessions
• We group common comments together
• At the end we present a findings report where we summarize what we learned and provide recommendations

What we’ve learned so far

• We have conducted over 100 hours of usability testing on Alberta.ca and its prototypes
• Frequent users have a very particular way of navigating
  • If they have to stray from that they get lost
  • People want a clear place to start
  • People have trouble moving laterally through related content

• Heard things like “Where is the home button”
• Visual aids help organize and break up content to keep it from being overwhelming
• Irrelevant content becomes very frustrating very quickly
• Accordions tested very well
  • People like being able to see a lot of information summarized that they can expand as needed

• Users want a website to be be useable
• People were skipping our navigation tools
• People don’t go looking for what they don’t know exists
• Users want help understanding how what they read on the website applies to their lives

Recruiting for accessibility testing

• Communications and Public Engagement (CPE) at the Government of Alberta has an Office for accessibility where we are trying to recruit people with accessibility issues for testing
• DTO is trying to recruit for this right now but it is difficult