Changes

[[File:CaseStudyofResponsibleAIatDFO.jpg|alt=|center|frameless|996x996px]]

[[File:Res ai poster Dec8.jpg|alt=|center|frameless|996x996px]]

== Executive Summary ==

== Executive Summary ==

Automated Decision Systems (ADS) are computer systems that automate, or assist in automating, part or all of an administrative decision-making process. Currently, automated decision systems driven by Machine Learning (ML) algorithms are being used by the Government of Canada to improve service delivery. The use of such systems can have benefits and risks for federal institutions, including the Department of Fisheries and Oceans Canada (DFO). There is a need to ensure that automated decision systems are deployed in a manner that reduces risks to Canadians and federal institutions and leads to more efficient, accurate, consistent, and interpretable decisions.

Automated decision systems are computer systems that automate, or assist in automating, part or all of an administrative decision-making process. Currently, automated decision systems driven by Machine Learning (ML) algorithms are being used by the Government of Canada to improve service delivery. The use of such systems can have benefits and risks for federal institutions, including the Department of Fisheries and Oceans Canada (DFO). There is a need to ensure that automated decision systems are deployed in a manner that reduces risks to Canadians and federal institutions and leads to more efficient, accurate, consistent, and interpretable decisions.

For the Fiscal Year (2020 - 2021), the Office of Chief Data Officer (OCDO), and Information Management and Technical Services (IMTS) co-sponsored an Artificial Intelligence (AI) pilot project to determine the extent to which AI, coupled with other data analytics techniques, can be used to gain insights, to automate tasks, and to optimize outcomes for different priority areas and business objectives at DFO. The project was funded through the 2020-21 Results Fund and was developed in partnership with programs and business areas within DFO. The results included the development of three proof of concepts:

For the Fiscal Year (2020 – 2021), the Office of the Chief Data Officer (OCDO), and Information Management and Technical Services (IMTS) co-sponsored an Artificial Intelligence (AI) pilot project to determine the extent to which AI, coupled with other data analytics techniques, can be used to gain insights, to automate tasks, and to optimize outcomes for different priority areas and business objectives at DFO. The project was funded through the 2020 – 2021 Results Fund and was developed in partnership with programs and business areas within DFO. The results included the development of three proof of concepts:

* Predictive models for detecting vessels’ fishing behavior, to fight     noncompliance with fishing regulations.

* Predictive models for detecting vessels’ fishing behavior, to fight non-compliance with fishing regulations.

* A predictive model for the identification and classification of     endangered North Atlantic Right Whale upcalls, to minimize vessels’ strike    to endangered whales.

* A predictive model for the identification and classification of endangered North Atlantic Right Whale upcalls, to minimize vessel strikes to endangered whales.

* A predictive model for clustering ocean data, to advance ocean     science.

* A predictive model for clustering ocean data, to advance ocean science.

Supported by the (2021 – 2022) Results Fund, the OCDO and IMTS are prototyping automated decision systems, based on the outcome of the AI pilot project. The effort includes defining an internal process to detect and mitigate bias, as a potential risk of ML-based automated decision systems. A case study is designed to apply this process to assess and mitigate bias in the predictive model for detecting vessels’ fishing behavior.  

Supported by the (2021 – 2022) Results Fund, the OCDO and IMTS are prototyping automated decision systems, based on the outcome of the AI pilot project. The effort includes defining an internal process to detect and mitigate bias, as a potential risk of ML-based automated decision systems. A case study is designed to apply this process to assess and mitigate bias in the predictive model for detecting vessels’ fishing behavior.  

Eventually, in the long term, the goal will be to develop a comprehensive responsible AI framework to ensure responsible use of AI within DFO and to ensure compliance with Treasury’s Board directive on automated decision making.

Eventually, in the long term, the goal will be to develop a comprehensive responsible AI framework to ensure responsible use of AI within DFO and to ensure compliance with the Treasury Board Directive on Automated Decision-Making.  

== Introduction ==

== Introduction ==

Unlike traditional Automated Decision Systems (ADS), Machine Learning (ML)-based ADS do not follow explicit rules authored by humans. ML models are not inherently objective. Data scientists train models by feeding them a data set of training examples, and the human involvement in the provision and curation of this data can make a model's predictions susceptible to bias. Due to this, applications of ML-based ADS have far-reaching implications for society. These range from new questions about the legal responsibility for mistakes committed by these systems to retraining for workers displaced by these technologies. There is a need for a framework to ensure that accountable and transparent decisions are made, supporting ethical practices.

Unlike traditional automated decision systems, Machine Learning (ML)-based automated decision systems do not follow explicit rules authored by humans <ref>V. Fomins, "The Shift from Traditional Computing Systems to  Artificial Intelligence and the Implications for Bias," ''Smart  Technologies and Fundamental Rights,'' pp. 316-333, 2020.</ref>. ML models are not inherently objective. Data scientists train models by feeding them a data set of training examples, and the human involvement in the provision and curation of this data can make a model's predictions susceptible to bias. Due to this, applications of ML-based automated decision systems have far-reaching implications for society. These range from new questions about the legal responsibility for mistakes committed by these systems to retraining for workers displaced by these technologies. There is a need for a framework to ensure that accountable and transparent decisions are made, supporting ethical practices.

=== Responsible AI ===

=== Responsible AI ===

Responsible AI is a governance framework that documents how a specific organization is addressing the challenges around AI from both an ethical and a legal point of view.

Responsible AI is a governance framework that documents how a specific organization is addressing the challenges around artificial intelligence (AI) from both an ethical and legal point of view.  

[[File:5gp.png|thumb|352x352px|alt=|''Figure 1: Five main guiding principles for Responsible AI'']]In an attempt to ensure Responsible AI practices, organizations have identified guiding principles to guide the development of AI applications and solutions. According to the research “The global landscape of AI ethics guidelines” <ref>A. Jobin, M. Ienca and E. Vayena, "The global landscape of AI ethics guidelines," ''Nature Machine Intelligence,'' p. 389–399, 2019</ref> , some principles are mentioned more often than others. However, Gartner has concluded that there is a global convergence emerging around five ethical principles <ref>S. Sicular , E. Brethenoux , F. Buytendijk and J. Hare, "AI Ethics: Use 5 Common Guidelines as Your Starting Point," Gartner, 11 July 2019. [Online]. Available: <nowiki>https://www.gartner.com/en/documents/3947359/ai-ethics-use-5-common-guidelines-as-your-starting-point</nowiki>. [Accessed 23 August 2021].</ref> :

In an attempt to ensure Responsible AI practices, organizations have identified guiding principles to guide the development of AI applications and solutions. According to the research “The global landscape of AI ethics guidelines” [1], some principles are mentioned more often than others. However, Gartner has concluded that there is a global convergence emerging around five ethical principles:

[[File:5gp.png|center|thumb|339x339px]]

=== Treasury Board Directive on Automated Decision-Making ===

Treasury Board’s Directive on Automated Decision-Making is defined as a policy instrument to promote ethical and responsible use of Artificial Intelligence. It outlines the responsibilities of federal institutions using AI-automated decision systems.

 

 

 

 

=== The Treasury Board Directive on Automated Decision-Making ===

The Treasury Board Directive on Automated Decision-Making is defined as a policy instrument to promote ethical and responsible use of AI. It outlines the responsibilities of federal institutions using AI-based automated decision systems.

==== The Algorithmic Impact Assessment Process ====

==== The Algorithmic Impact Assessment Process ====

According to the Directive, an Algorithmic Impact Assessment (AIA)  must be conducted before the production of any ADS, to assess the risks of the system. The assessment must be updated at regular intervals when there is a change to the functionality or the scope of the Automated Decision System.

According to the Directive, an Algorithmic Impact Assessment (AIA)  must be conducted before the production of any automated decision systems to assess the risks of the system. The assessment must be updated at regular intervals when there is a change to the functionality or the scope of the Automated Decision System.

The AIA tool is a questionnaire that determines the impact level of an automated decision system. It is composed of 48 risk and 33 mitigation questions. Assessment scores are based on many factors including systems design, algorithm, decision type, impact, and data. The tool results in an impact categorization level between I (presenting the least risk) and IV (presenting the greatest risk).   Based on the impact level (which will be published), the Directive may impose additional requirements. The process is described in Figure 2.

== Bias Detection and Mitigation ==

== Bias Detection and Mitigation ==