Changes

=== Responsible AI ===

=== Responsible AI ===

Responsible AI is a governance framework that documents how a specific organization is addressing the challenges around AI from both an ethical and a legal point of view.

Responsible AI is a governance framework that documents how a specific organization is addressing the challenges around AI from both an ethical and a legal point of view.

[[File:5gp.png|thumb|364x364px|alt=|'''Figure 1: Five main guiding principles for Responsible AI''']]In an attempt to ensure Responsible AI practices, organizations have identified guiding principles to guide the development of AI applications and solutions.  According to the research “The global landscape of AI ethics guidelines” <ref>A. Jobin, M. Ienca and E. Vayena, "The global landscape of AI ethics guidelines," ''Nature Machine Intelligence,'' p. 389–399, 2019</ref> , some principles are mentioned more often than others. However, Gartner has concluded that there is a global convergence emerging around five ethical principles <ref>S. Sicular , E. Brethenoux , F. Buytendijk and J. Hare, "AI Ethics: Use 5 Common Guidelines as Your Starting Point," Gartner, 11 July 2019. [Online]. Available: <nowiki>https://www.gartner.com/en/documents/3947359/ai-ethics-use-5-common-guidelines-as-your-starting-point</nowiki>. [Accessed 23 August 2021].</ref> :

[[File:5gp.png|thumb|426x426px|alt=|'''Figure 1: Five main guiding principles for Responsible AI''']]In an attempt to ensure Responsible AI practices, organizations have identified guiding principles to guide the development of AI applications and solutions.  According to the research “The global landscape of AI ethics guidelines” <ref>A. Jobin, M. Ienca and E. Vayena, "The global landscape of AI ethics guidelines," ''Nature Machine Intelligence,'' p. 389–399, 2019</ref> , some principles are mentioned more often than others. However, Gartner has concluded that there is a global convergence emerging around five ethical principles <ref>S. Sicular , E. Brethenoux , F. Buytendijk and J. Hare, "AI Ethics: Use 5 Common Guidelines as Your Starting Point," Gartner, 11 July 2019. [Online]. Available: <nowiki>https://www.gartner.com/en/documents/3947359/ai-ethics-use-5-common-guidelines-as-your-starting-point</nowiki>. [Accessed 23 August 2021].</ref> :

·      Human-centric and socially beneficial

·      Human-centric and socially beneficial

==== The Algorithmic Impact Assessment Process ====

==== The Algorithmic Impact Assessment Process ====

According to the Directive, an Algorithmic Impact Assessment (AIA)  must be conducted before the production of any automated decision systems to assess the risks of the system. The assessment must be updated at regular intervals when there is a change to the functionality or the scope of the Automated Decision System.

According to the Directive, an Algorithmic Impact Assessment (AIA)  must be conducted before the production of any automated decision systems to assess the risks of the system. The assessment must be updated at regular intervals when there is a change to the functionality or the scope of the Automated Decision System.

The AIA tool is a questionnaire that determines the impact level of an automated decision system. It is composed of 48 risk and 33 mitigation questions. Assessment scores are based on many factors including systems design, algorithm, decision type, impact, and data. The tool results in an impact categorization level between I (presenting the least risk) and IV (presenting the greatest risk).   Based on the impact level (which will be published), the Directive may impose additional requirements. The process is described in Figure 2.

The AIA tool is a questionnaire that determines the impact level of an automated decision system. It is composed of 48 risk and 33 mitigation questions. Assessment scores are based on many factors including systems design, algorithm, decision type, impact, and data. The tool results in an impact categorization level between I (presenting the least risk) and IV (presenting the greatest risk).   Based on the impact level (which will be published), the Directive may impose additional requirements. The process is described in Figure 2.

 

== Bias Detection and Mitigation ==

== Bias Detection and Mitigation ==

To comply with the  Treasury Board Directive on automated decision-making, DFO is working on establishing an internal process, as part of the Results Fund project for 2021 – 2022,  to detect and mitigate potential bias in ML-based automated decision systems.  

To comply with the  Treasury Board Directive on automated decision-making, DFO is working on establishing an internal process, as part of the Results Fund project for 2021 – 2022,  to detect and mitigate potential bias in ML-based automated decision systems.

=== Bias Detection and Mitigation Process ===

=== Bias Detection and Mitigation Process ===

The process of bias detection and mitigation is dependent upon the identification of the context within which bias is to be assessed. Given the breadth of sources from which bias can originate, exhaustive identification of the sources relevant to a particular system and the quantification of their impacts can be impractical. As such, it is recommended to instead '''view bias through the lens of harms that can be induced by the system'''  <ref>S. Bird, M. Dudík, R. Edgar, B. Horn, R. Lutz, V. Milan, M. Sameki, H. Wallach and K. Walker, "Fairlearn: A toolkit for assessing and improving fairness in AI," Microsoft, May 2020. [Online]. Available: <nowiki>https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/</nowiki>. [Accessed 30 11 2021].</ref>. Common types of harm can be represented through formal mathematical definitions of fairness in decision systems. These definitions provide the foundation for the quantitative assessment of fairness as an indicator of bias in systems.

The process of bias detection and mitigation is dependent upon the identification of the context within which bias is to be assessed. Given the breadth of sources from which bias can originate, exhaustive identification of the sources relevant to a particular system and the quantification of their impacts can be impractical. As such, it is recommended to instead '''view bias through the lens of harms that can be induced by the system'''  <ref name=":0">S. Bird, M. Dudík, R. Edgar, B. Horn, R. Lutz, V. Milan, M. Sameki, H. Wallach and K. Walker, "Fairlearn: A toolkit for assessing and improving fairness in AI," Microsoft, May 2020. [Online]. Available: <nowiki>https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/</nowiki>. [Accessed 30 11 2021].</ref>. Common types of harm can be represented through formal mathematical definitions of fairness in decision systems. These definitions provide the foundation for the quantitative assessment of fairness as an indicator of bias in systems.

Harms are typically considered in the context of error rates and/or rates of representation. Differences in error rates across different groups of people induce disproportionate performance of the system, leading to unjust consequences or reduced benefits for certain groups. Differences in rates of representation, even in the absence of differences in rates of error, can lead to unfair distributions of benefits or penalties across groups of people. The types of harm applicable to a particular system are highly dependent upon the intended usage of the system. A flow chart to assist in identification of the metric of fairness most relevant to a system is provided in Figure 3.

Harms are typically considered in the context of error rates and/or rates of representation. Differences in error rates across different groups of people induce disproportionate performance of the system, leading to unjust consequences or reduced benefits for certain groups. Differences in rates of representation, even in the absence of differences in rates of error, can lead to unfair distributions of benefits or penalties across groups of people. The types of harm applicable to a particular system are highly dependent upon the intended usage of the system. A flow chart to assist in identification of the metric of fairness most relevant to a system is provided in Figure 3.

Having laid out the potentials for harm in the system, the next step is to identify protected attributes. In the context of bias detection and mitigation, a protected attribute refers to a categorical attribute for which there are concerns of bias across the attribute categories. Common examples of protected attributes are race and gender. Relevant protected attributes for the application of the system must be identified in order to investigate disproportionate impacts across the attribute categories.

Having laid out the potentials for harm in the system, the next step is to identify protected attributes. In the context of bias detection and mitigation, a protected attribute refers to a categorical attribute for which there are concerns of bias across the attribute categories. Common examples of protected attributes are race and gender. Relevant protected attributes for the application of the system must be identified in order to investigate disproportionate impacts across the attribute categories.

Once the assessment of potential harms and affected groups has been conducted, a bias detection and mitigation tools can be applied to attempt to remedy issues of unfairness in the system. This overall process can thus be broken up into four steps:

Once the assessment of potential harms and affected groups has been conducted, bias detection and mitigation tools can be applied to attempt to remedy issues of unfairness in the system. This overall process can thus be broken up into four steps:

 

[[File:BiasProcess.png|alt=Bias detection and mitigation process|center|thumb|503x503px|'''Figure 4: Bias detection and mitigation process''']]

 

 

3.      Bias detection

 

4.      Bias mitigation

=== Bias Detection and Mitigation Tools ===

=== Bias Detection and Mitigation Tools ===

There is a growing number of technical bias detection and mitigation tools, which can supplement the AI practitioner’s capacity to avoid and mitigate AI bias. As part of the 2021 -2022 Result fund project, DFO has explored the following bias detection tools that AI practitioners can use to detect and remove bias.

There is a growing number of technical bias detection and mitigation tools, which can supplement the AI practitioner’s capacity to avoid and mitigate AI bias. As part of the 2021 – 2022 Results Fund project, DFO has explored the following bias detection tools that AI practitioners can use to detect and remove bias.

==== Microsoft’s FairLearn ====

==== Microsoft’s FairLearn ====

An open-source toolkit by Microsoft that allows AI practitioners to detect and correct the fairness of their AI systems. To optimize the trade-offs between fairness and model performance, the toolkit includes two components: an interactive visualization dashboard and bias mitigation algorithm [2].

An open-source toolkit by Microsoft that allows AI practitioners to detect and correct the fairness of their AI systems. To optimize the trade-offs between fairness and model performance, the toolkit includes two components: an interactive visualization dashboard and bias mitigation algorithms <ref name=":0" />.

==== '''IBM AI Fairness 360''' ====

==== '''IBM AI Fairness 360''' ====

An open-source toolkit by IBM that helps AI practitioners to easily check for biases at multiple points along their machine learning pipeline, using the appropriate bias metric for their circumstances. It comes with more than 70 fairness metrics and 11 unique bias mitigation algorithms [3].

An open-source toolkit by IBM that helps AI practitioners to easily check for biases at multiple points along their ML pipeline, using the appropriate bias metric for their circumstances. It comes with more than 70 fairness metrics and 11 unique bias mitigation algorithms <ref>IBM Developer Staff, "AI Fairness 360," IBM, 14 November 2018. [Online]. Available: <nowiki>https://developer.ibm.com/open/projects/ai-fairness-360/</nowiki>. [Accessed 28 July 2021].</ref>.

 

In the following section, details are provided on an investigation into one of the AI pilot project POCs using this proposed process. Through this investigation, further information can be gleaned into the practical considerations of bias detection and mitigation.

== Case Study: Predictive Model for Detecting Vessels’ Fishing Behavior ==

== Case Study: Predictive Model for Detecting Vessels’ Fishing Behavior ==

Through the AI work funded by the 2020-2021 Results Fund, a POC was developed for the automated detection of vessel fishing activity using a machine learning model. The resulting predictive model takes the vessel tracks as input and outputs the vessel tracks annotated with its activity i.e. fishing or not-fishing, as illustrated in Figure 4. The model uses features such as vessel speed deviation, course deviation, and gear type to detect vessel activity. The insight gained from the predictive model is then combined with other data sources, such as fisheries management areas and license conditions to detect non-compliance with fisheries regulations.

In this section, details are provided on an investigation into one of the AI pilot project proof-of-concepts using the proposed  Bias Detection and Mitigation Process. Through this investigation, further information can be gleaned into the practical considerations of bias detection and mitigation.

The fishing detection predictive model will be used as the backbone for building an automated decision system for the detection of non-compliant fishing behavior. After consultation with TBS, DFO has established that the TBS directive on automated decision systems applies to the production of the model.

The fishing detection predictive model will be used as the backbone for building an automated decision system for the detection of non-compliant fishing behavior. After consultation with the Treasury Board of Canada, DFO has established that the Treasury Board Directive on Automated Decision Systems applies to the production of the model.

To comply with the Directive, and to ensure that this automated decision system is both effective and fair, an investigation has been conducted into the potential for bias in the fishing detection model, and bias mitigation techniques have been applied.

To comply with the Directive, and to ensure that the resulting automated decision system is both effective and fair, an investigation has been conducted into the potential for bias in the fishing detection model, and bias mitigation techniques have been applied.

=== Terms and Definitions ===

=== Terms and Definitions ===

'''Bias''': In the context of automated decision systems, the presence of systematic errors, or misrepresentation in data and system outcomes. Bias may originate from a variety of sources including societal bias (e.g., discrimination) embedded in data, unrepresentative data sampling or preferential treatment imparted through algorithms.

'''Bias''': In the context of automated decision systems, the presence of systematic errors or misrepresentation in data and system outcomes. Bias may originate from a variety of sources including societal bias (e.g., discrimination) embedded in data, unrepresentative data sampling and preferential treatment imparted through algorithms.

'''Disparity''': A systematic lack of fairness across different groups. Formally measured as the maximal difference between the values of a metric applied across the groups defined by a ''protected attribute''.

'''Disparity''': A systematic lack of fairness across different groups. Formally measured as the maximal difference between the values of a metric applied across the groups defined by a ''protected attribute''.

'''False positive (negative) rate''': The percentage of negative (positive) instances incorrectly classified as positive (negative) in a sample of instances to which binary classification is applied.

'''False positive (negative) rate''': The percentage of negative (positive) instances incorrectly classified as positive (negative) in a sample of instances to which binary classification is applied.

'''Protected attribute''': An attribute that partitions a population into groups across which fairness is expected. Examples of common protected attributes are gender and race.

'''Protected attribute''': An attribute that partitions a population into groups across which fairness is expected. Examples of common protected attributes are gender and race.  

=== Results of Bias Assessment   ===

=== Results of Bias Assessment   ===

 

The bias assessment process has been conducted following the four steps proposed in the Bias Detection and Mitigation Process. Details on the results of the investigation for each of the steps are provided below.

The bias assessment process has been conducted following the four steps proposed as indicated in the Bias Detection and Mitigation Process. Details on the results of the investigation for each of the steps are provided below.

==== Identification of Potential Harms ====

==== Identification of Potential Harms ====

In the binary (i.e., yes or no) detection of fishing activity, the system is susceptible to two types of errors:

In the binary (i.e., yes or no) detection of fishing activity, the system is susceptible to two types of errors:

1.      False positives – Reporting fishing activity where it has not occurred

# False positives – Reporting fishing activity when it has not occurred

 

# False negatives – Reporting an absence of fishing activity when in fact it has occurred

2.      False negatives – Reporting an absence of fishing activity when in fact it has occurred

When the fishing detections are checked against license conditions, these errors can then take on two alternate forms:

When the fishing detections are checked against license conditions, these errors can then take on two alternate forms:

1.      False positives may lead to incorrect reporting of non-compliance

# False positives may lead to incorrect reporting of non-compliance

 

# False negatives may lead to missed detections of non-compliance

2.      False negatives may lead to missed detections of non-compliance

As the system is designed to assist in the identification of cases in which punitive action is required, the false positives have the potential to cause harms to the operators of the vessels that are incorrectly flagged for non-compliance. It is important to note that the intended use of this system is not to make final determinations on the occurrence of non-compliant behavior but rather to assist fishery officers in the identification of cases that warrant further investigation. False positives are therefore unlikely to result in undue punitive actions. However, they result in unwarranted scrutiny of vessel activities, which is both undesirable and unfair to vessel operators. While perfect detection of fishing activity may be unattainable, it is nevertheless critical to put in best efforts to reduce error rates and to mitigate disproportionate (i.e., biased) impacts across different groups of people.

As the system is designed to assist in the identification of cases in which punitive action is required, the false positives have the potential to cause harms to the operators of the vessels that are incorrectly flagged for non-compliance. It is important to note that the intended use of this system is not to make final determinations on the occurrence of non-compliant behavior but rather to assist fishery officers in the identification of cases that warrant further investigation. False positives are therefore unlikely to result in undue punitive actions. However, they result in unwarranted scrutiny of vessel activities, which is both undesirable and unfair to vessel operators. While perfect detection of fishing activity may be unattainable, it is nevertheless critical to put in best efforts to reduce error rates and to mitigate disproportionate (i.e., biased) impacts across different groups of people.

Although false positives have been identified as the primary source of potential harm through this investigation, it is important to note that some degree of less direct harm exists in false negatives as well. Missed detections of non-compliance may result in illegal fishing going uncaught. This has a cost to both DFO as well as the commercial fisheries. Should certain fisheries or geographic areas be more susceptible to false negatives than others, this may lead to disproportionate distributions of these costs.

Although false positives have been identified as the primary source of potential harm through this investigation, it is important to note that some degree of less direct harm exists in false negatives as well. Missed detections of non-compliance may result in illegal fishing going uncaught. This has a cost to both DFO as well as the commercial fisheries. Should certain fisheries or geographic areas be more susceptible to false negatives than others, this may lead to disproportionate distributions of these costs.

==== Bias Detection ====

==== Bias Detection ====