Third Review of the Directive on Automated Decision-Making

About the Third review

Background

Treasury Board of Canada Secretariat (TBS) is completing the third review of the Directive on Automated Decision-Making. The review takes stock of the current state of the directive and identifies risks and challenges to the government’s commitment to responsible artificial intelligence (AI) in the federal public sector. These issues highlight critical gaps that limit the directive’s relevance and effectiveness in supporting transparency, accountability, and fairness in automated decision-making. They also identify problems with terminology, feasibility, and coherence with other federal policy instruments.

Periodic reviews are not intended to be exhaustive. They seek to adapt the directive to pertinent trends in the Canadian and global AI landscape, while gradually refining the text of the instrument to support interpretation and facilitate compliance. The first review sought to clarify and reinforce existing requirements, update policy references, and strengthen transparency and quality assurance measures. The second review informed the development of guidelines supporting the interpretation of the directive.

Policy recommendations

As part of the third review, TBS is proposing 12 policy recommendations and accompanying amendments to the directive:

1. Expand the scope to cover internal services.
2. Clarify that the scope includes systems which make assessments related to administrative decisions.
3. Replace the 6-month review interval with a biennial review and enable the Chief Information Officer of Canada to request off-cycle reviews.
4. Replace references to Canadians with more encompassing language such as clients and Canadian society.
5. Introduce measures supporting the tracing, protection, and appropriate retention and disposition of data used and generated by a system.
6. Expand the pre-production testing requirement to cover model bias testing.
7. Mandate the completion of Gender Based Analysis Plus during the development of a system.
8. Establish explanation criteria in support of the explanation requirement and integrate them into the Algorithmic Impact Assessment (AIA).
9. Expand the AIA to include questions concerning an institution's reasons for pursuing automation and potential impacts on persons with disabilities.
10. Mandate the publication of complete or summarized peer reviews and require completion prior to system production.
11. Align the contingency requirement with relevant terminology established in Treasury Board security policy.
12. Mandate the release of AIAs prior to the production of a system.

Expected outcomes

The recommendations would ensure that automated decision systems impacting federal public servants are fair and inclusive; reinforce transparency and accountability; strengthen protections against discrimination and harm; clarify requirements; and support operational needs.

Stakeholder engagement

TBS is engaging with a broad range of stakeholders during the third review, including in academia, civil society, other governments, and international organizations. The goal of stakeholder engagement is to validate the policy recommendations and provisional amendments proposed in the review and identify additional issues that merit consideration as part of this exercise or in future reviews.

In September 2022, TBS launched the second phase of stakeholder engagement. This phase will involve outreach to federal AI policy and data communities; agents of parliament; bargaining agents; and international organizations. The first phase ran between April and July 2022, and drew on the expertise of federal partners and subject matter experts in academia, civil society, and other governments. The What We Heard Report linked below provides a summary of the outcomes of this phase.

Contacted stakeholders are invited to review and comment on the proposal for amending the directive and AIA, which is laid out in the consultation materials below.

Consultation materials

• Key issues, policy recommendations, and provisional amendments [link to be added]
• One-page overview of policy recommendations [link to be added]

Reference materials

• What We Heard Report (phase 1 of stakeholder engagement) [link to be added]
• Report on the third review of the Directive on Automated Decision-Making [link to be added]

Contact

Please submit any questions to ai-ia@tbs-sct.gc.ca