Difference between revisions of "July 31 - DTO - TSS results"

From wiki
Jump to navigation Jump to search
(Created page with "Back to GC Web Priorities Meetings - Réunions des Priorités Web du GC ==Meeting information== ===Agenda=== 1. Updates 2. Roundtable =Notes and action item= ===Updates==...")
 
Line 8: Line 8:
 
===Updates===
 
===Updates===
 
'''DTO'''
 
'''DTO'''
−
*No updates this week
 
−
'''PCO'''
 
−
*Campaigns - We’ve reached out to encourage people to start early working on their campaigns
 
−
*Check out the [https://gc-web-comms.slack.com/archives/C018SF3J4KF/p1721760193367859 Slack thread about campaigns], tips such as avoiding carousels
 
−
*Big thanks to Principle Publisher for new biography for Minister Mckinnon
 
−
*Today is International Self Care Day
 
−
'''Principle Publisher'''
 
−
*Removed mention of Minister Seamus O’Regan and replaced with new Minister Steven MacKinnon
 
−
*We have removed the Share this page widget from AEM, if you’re not on AEM, please remove by end of August
 
−
'''PCO - [https://impact.canada.ca/en Impact Canada] - Harry MacKay'''
 
  
−
'''[[:en:images/2/24/AB_testing_IIU_EN_2024-04-26_for_posting.pptx|A Beginner’s Guide to A/B Testing on Canada.ca]]'''
+
* Working on stabilization of topic page template with Principal Publisher
−
*We were established in 2017
+
** We will follow up with messaging to explain the updates more fully, and will speak about it at an upcoming meeting
−
*Partners such as:
 
−
**ECCC
 
−
**NRCan
 
−
**PHAC, Health Canada etc…
 
−
*What is A/B testing?
 
−
**A lot of website design is guided by top down principles of usability, composition etc..
 
−
**AB testing is a way to empirically test two or more versions of a product
 
−
**Variants are randomly assigned to subjects as they enter the page
 
−
*Considerations for good test:
 
−
**DTO has provided valuable information on A/B testing
 
−
**Canada.ca is mandated to provide clear information to the public
 
−
**Ideally, A/B testing focus on one single element at a time
 
−
*Examples of A/B testing on Canada.ca
 
−
**Example of CRA test, in this example the element you were supposed to click was more prominent, with a 20% increase in desire outcome
 
−
**OCAO ran a similar test, making button more “obviously clickable” with favorable results
 
  
−
*Considering statistical power
+
[[Mailto:lana.stewart@cds-snc.ca|Lana Stewart]] DTO - TSS results
−
**You have to make sure you have enough page traffic for meaningful results
 
−
**The sample size is determined by incoming page views
 
−
**The only thing you really have control over to increase amount of people, is to run the test for a longer time
 
−
**This means knowing the average amount of visitors per day for your page is critical
 
−
***Checklist of statistical power
 
−
****Average number of daily visitors - Canada.ca analytics
 
−
****Baseline rate of target behavior
 
−
****Expected effect size of manipulation during testing
 
−
**Adobe provides a tool that can estimate how many visitors you will need to collect with a given lift rate and baseline conversion rate
 
−
*[https://business.adobe.com/ca/products/target/adobe-target.html Adobe Target] is the software we use for A/B testing, Principle Publisher manages access to this software
 
−
**Adobe Target GCpedia page: https://www.gcpedia.gc.ca/wiki/Adobe_Target_User_Guide
 
−
Privacy and ethical considerations
 
−
*Personal information is not collected
 
−
*Use your own discretion using Adobe Target in environments with Protected information
 
−
*Departments are mandated to provide clear information
 
−
**This may limit ability to change wording/text on you page
 
−
*You may want to submit your plans to ethics review if you have it at your department
 
−
*Stop at a predetermined time to avoid biases
 
−
*The Sequential Analysis approach has predefined stopping rules, such as in a drug trial, to protect against the ethical ob subjecting users to underperforming version of critical web page
 
−
Key Takeways
 
−
*A/B testing is a powerful tool to improve usability
 
−
*Statistical power is the biggest concern when planning your project
 
−
**Average lift (improvement) is around 2%, Not every page on Canada.ca receives enough traffic to run a test
 
−
*The testing is relatively low cost
 
−
'''QUESTIONS'''
 
  
−
'''Q:''' How do you consider this vs. other kinds of testing?
+
Overview of the GC’s results for Q1 of 2024-25 FY
  
−
'''A:''' It’s a good method once your website has been set up. It’s really for optimizing. Doesn’t address accessibility very well. It has its place in combination with other kinds of testing which for example might be better for specific users or other considerations.
+
* 117,646 visitors completed the survey.
 +
* 69.6% of respondents said that they were able to complete the reason for their visit.
 +
* We received a total of 89,066 comments from survey respondents.
  
−
Comment: A/B testing is good for settling points, for example disagreements between clients or the Minister's Office.
+
To see all the results, [https://www.gcpedia.gc.ca/wiki/Government_of_Canada_Task_Success_Survey_reports download the report from GCpedia.]
  
−
Comment, we recently did a presentation on Adobe Target, sharing here: https://gcxgce.sharepoint.com/:p:/t/1000163/ESGBlOV_32RNuYTcPJKTU8EBO9J9Quk0Ef9jMHej1NYSAg?e=4nGzEc
+
== Roundtable ==
 +
Public Safety (PJE Theme)
  
−
'''Comment:'''
+
* In support of the GC response to wildfire event across Canada and especially in Jasper, PP created a newsroom subject tag Forest fires/ Incendie de forêt  for wildfire event news products. These will automatically be linked to the Wildfires Emergency event page:
 +
** https://www.canada.ca/en/public-safety-canada/campaigns/wildfires.html
 +
** [[July 31 - DTO - TSS results/www.canada.ca/en/public-safety-canada/campaigns/wildfires.html|https://www.canada.ca/en/public-safety-canada/campaigns/wildfires.html]]
  
−
Doing usability testing on a prototype can also help overcome some of the potential ethical considerations that Harry mentioned.
+
ISED (Business and science themes)
  
−
For example, if you want to test the impact of removing an entire paragraph of content and don't want to risk some users having information that other users do not, you can use usability testing instead of an A/B test.
+
* Still working on our AI subtopic page prototype for all things GC AI; planning first version by end summer
−
==Roundtable==
 
−
'''ISED'''
 
−
*Reminder; we are still looking for departments to share any AI content pages you have so we can build the subtopic - aiming for September to launch
 
−
Contact: [mailto:Christopher.colyer@ised-isde.gc.ca christopher.colyer@ised-isde.gc.ca]
 

Revision as of 16:33, 19 August 2024

Back to GC Web Priorities Meetings - Réunions des Priorités Web du GC

Meeting information

Agenda

1. Updates

2. Roundtable

Notes and action item

Updates

DTO

  • Working on stabilization of topic page template with Principal Publisher
    • We will follow up with messaging to explain the updates more fully, and will speak about it at an upcoming meeting

[Stewart] DTO - TSS results

Overview of the GC’s results for Q1 of 2024-25 FY

  • 117,646 visitors completed the survey.
  • 69.6% of respondents said that they were able to complete the reason for their visit.
  • We received a total of 89,066 comments from survey respondents.

To see all the results, download the report from GCpedia.

Roundtable

Public Safety (PJE Theme)

ISED (Business and science themes)

  • Still working on our AI subtopic page prototype for all things GC AI; planning first version by end summer
Retrieved from "https://wiki.gccollab.ca/index.php?title=July_31_-_DTO_-_TSS_results&oldid=124064"