245
edits
Changes
Towards a Responsible AI Framework for the Design of Automated Decision Systems in DFO: a Case Study of Bias Detection and Mitigation (view source)
Revision as of 12:48, 8 December 2021
, 12:48, 8 December 2021→Bias Detection
==== '''IBM AI Fairness 360''' ====
==== '''IBM AI Fairness 360''' ====
An open-source toolkit by IBM that helps AI practitioners to easily check for biases at multiple points along their ML pipeline, using the appropriate bias metric for their circumstances. It comes with more than 70 fairness metrics and 11 unique bias mitigation algorithms <ref>IBM Developer Staff, "AI Fairness 360," IBM, 14 November 2018. [Online]. Available: <nowiki>https://developer.ibm.com/open/projects/ai-fairness-360/</nowiki>. [Accessed 28 July 2021].</ref>.
An open-source toolkit by IBM that helps AI practitioners to easily check for biases at multiple points along their ML pipeline, using the appropriate bias metric for their circumstances. It comes with more than 70 fairness metrics and 11 unique bias mitigation algorithms <ref name=":1">IBM Developer Staff, "AI Fairness 360," IBM, 14 November 2018. [Online]. Available: <nowiki>https://developer.ibm.com/open/projects/ai-fairness-360/</nowiki>. [Accessed 28 July 2021].</ref>.
== Case Study: Predictive Model for Detecting Vessels’ Fishing Behavior ==
== Case Study: Predictive Model for Detecting Vessels’ Fishing Behavior ==
==== Bias Detection ====
==== Bias Detection ====
To assess model fairness across different gear types, the data is first partitioned according to the gear type and then False Positive Rate (FPR) disparity [3] is measured. The FPR is given by the percentage of negative instances (of fishing activity) that are mislabeled by the model as being positives instances. The FPR disparity is measured as the greatest difference between the FPR of each grouping of vessels by gear type. The greater this difference is measured to be, the greater the degree of bias and unfairness in the system.
To assess model fairness across different gear types, the data is first partitioned according to the gear type and then False Positive Rate (FPR) disparity <ref name=":1" /> is measured. The FPR is given by the percentage of negative instances (of fishing activity) that are mislabeled by the model as being positives instances. The FPR disparity is measured as the greatest difference between the FPR of each grouping of vessels by gear type. The greater this difference is measured to be, the greater the degree of bias and unfairness in the system.
In this investigation FairLearn [2], has been applied to implement the FPR disparity measurement experiments. As a point of reference, IBM AI Fairness 360 [3], applies a threshold of 10 on similar disparity metrics as the point beyond which the model is considered to be unfair in an interactive demo [4]. Results from the initial fishing detection model are shown in Table 1. Due to an excessively high FPR for the troll gear type, there is an FPR disparity difference of 52.62. These results highlight an unacceptable level of bias present in the model which must be mitigated.
In this investigation Fairlearn <ref name=":0" />, has been applied to implement the FPR disparity measurement experiments. As a point of reference, IBM AI Fairness 360 <ref name=":1" />, applies a threshold of 10 on similar disparity metrics as the point beyond which the model is considered to be unfair in an interactive demo [7]. Results from the initial fishing detection model are shown in Figure 6. Due to an excessively high FPR for the troll gear type, there is an FPR disparity difference of 52.62. '''<u>These results highlight an unacceptable level of bias present in the model which must be mitigated</u>'''.
[[File:Chart 1.png|alt=Results for FPR and accuracy are shown for each gear type. The FPR disparity difference is measured as the difference between the highest and lowest FPR, giving a value of 52.62.|center|thumb|703x703px|'''Figure 6: Results for FPR and accuracy are shown for each gear type. The FPR disparity difference is measured as the difference between the highest and lowest FPR, giving a value of 52.62.''']]
|2.67
|92.95
|98.92
|96.65
|90.64
==== Bias Mitigation ====
==== Bias Mitigation ====
Bias mitigation algorithms implemented in FairLearn and other similar tools can be applied at various stages of the machine learning pipeline. In general, there is a trade-off between model performance and bias such that mitigation algorithms induce a loss in model performance. Initial experimentation has demonstrated this occurrence leading to a notable loss in performance to reduce bias. This can be observed in the results shown in Table 2 where a mitigation algorithm has been applied to reduce the FPR disparity to 28.83 at the cost of a loss in fishing detection accuracy.
Bias mitigation algorithms implemented in Fairlearn and other similar tools can be applied at various stages of the ML pipeline. In general, there is a trade-off between model performance and bias such that mitigation algorithms induce a loss in model performance. Initial experimentation has demonstrated this occurrence, leading to a notable loss in performance to reduce bias. This can be observed in the results shown in Figure 7 where a mitigation algorithm has been applied to reduce the FPR disparity to 28.83 at the cost of a loss in fishing detection accuracy.
[[File:Chart2.png|alt=Results for FPR and accuracy after bias mitigation via Fairlearn. The FPR disparity difference is 28.83.|center|thumb|691x691px|'''Figure 7: Results for FPR and accuracy after bias mitigation via Fairlearn. The FPR disparity difference is 28.83.''']]
Based on these results, it was determined that efforts on bias mitigation should be focused in the data cleaning and preparation stages of the ML pipeline as this is the earliest possible point where this can be achieved. Through additional exploratory data analysis, notable issues were identified in terms of disproportionate representation across gear types and imbalance between positive and negative instances of fishing activity for some gear types. Through adjustments to the data cleaning process as well as the application of techniques such as data balancing and data augmentation, a new version of the model training data, better suited to the task of bias mitigation, was produced. The results of these modifications can be seen in Figure 8. The FPR has been greatly reduced across all gear types while preserving an acceptable level of fishing detection accuracy. Notably, the FPR disparity difference has been reduced from its original value of 52.62 down to 2.97.
[[File:Chart3.png|alt=Results for FPR and accuracy after improvements were made to the data preparation process. The FPR disparity difference is 2.97.|center|thumb|704x704px|'''Figure 8: Results for FPR and accuracy after improvements were made to the data preparation process. The FPR disparity difference is 2.97.''']]
|39.52
|10.69
|-
|'''Accuracy'''
Based on these results, it was determined that efforts on bias mitigation should be focused in the data cleaning and preparation stages of the machine learning pipeline as this is the earliest possible point where this can be achieved. Through additional exploratory data analysis, notable issues were identified in terms of disproportionate representation across gear types and imbalance between positive and negative instances for some gear types. Through adjustments to the data cleaning process as well as the application of techniques such as data balancing and data augmentation, a new version of the model training data, better suited to the task of bias mitigation, was produced. The results of these modifications can be seen in Table 3. The FPR has been greatly reduced across all gear types while preserving an acceptable level of fishing detection accuracy. Notably, the FPR disparity difference has been reduced from its original value of 52.62 down to 2.97.
|4.44
|1.98
|-
|'''Accuracy'''
=== Bias Assessment Next Steps ===
=== Bias Assessment Next Steps ===
DFO is in the process of defining guiding principles to guide the development of AI applications and solutions. Once defined, various tools will be considered and/or developed to operationalize such principles.
DFO is in the process of defining guiding principles to guide the development of AI applications and solutions. Once defined, various tools will be considered and/or developed to operationalize such principles.
<references />