AI-Enabled Electronic Monitoring Solutions
Introduction
The Department of Fisheries and Oceans (DFO) Canada manages over 200 fisheries on the three coasts. Successful fisheries management is dependent on effective fisheries monitoring. The goal of fisheries monitoring is to provide accurate, timely, and accessible fisheries data, which is needed to effectively implement management measures such as target and bycatch limits, quotas, and closed areas.
The at-sea observer program and the Electronic Monitoring (EM) program are both adopted by DFO to monitor fishing activities, collect scientific data, and monitor industry compliance with fishing regulations and license conditions. Both programs collect fishing data such as the number of fish caught, fishing effort (e.g., number of hours or days spent fishing), and bycatch. The collected data can be used to audit logbooks completed by harvesters to ensure that catch is accurately represented.
The at-sea observer program relies on physical observers onboard the vessels to manually collect this data. On the other hand, the electronic monitoring program uses an electronic monitoring system onboard fishing vessels to capture sensor data and video footage. The system consists of multiple cameras and sensory devices. Each fish must be held in front of a measuring board in clear view of the camera, which allows for size verification, and species identification before release.
Electronic monitoring is a safer and a more cost-effective solution that can provide better coverage and transparency. In addition to DFO’s interest in promoting the adoption of electronic monitoring, the department is also interested in maximizing the provision of high-quality data and ensuring that solutions align with departmental objectives for the electronic monitoring program.
Electronic Monitoring Challenges
Although electronic monitoring comes with many benefits, there are many challenges associated with storing and analyzing the large volume of the captured data. For example, a single fishing trip can generate over 2TB of video data which leads to very high storage and transmission costs. More data is often collected than analyzed. An audit-based approach is adopted where only around 10% of the available data is analyzed. Currently, the review process of monitoring footage is done manually by EM reviewers who review the footage to verify and provide the relevant catch information. The manual process may result in several efficiencies in terms of the quality and accuracy of the provided information, and the time required to review the footage. DFO seeks to pilot and adopt new innovative technologies to create efficiencies in the Electronic Monitoring data workflows to support the adoption of the electronic monitoring program.
Project Goal and Objectives
Implementation Approach
In 2022-2023, DFO issued a competitive Request for Proposal in collaboration with Public Service and Procurement Canada to solicit bids from contractors qualified on the Government of Canada's AI suppliers list. As a result of this process, the company AI.Fish LLC won the contract and is currently developing the solution. The initial scope of the project is to develop an AI-assisted solution for fishing catch estimation and apportionment in Pacific trawl fisheries. As of the fiscal year 2022-2023, the project has entered Stage C: the Proof of Value phase.
AI-assisted Catch Estimation and Apportionment
The existing manual process for estimating catch comprises several steps. Human reviewers are required to observe every tow from the time the gear is retrieved until all catch is aboard the deck. During this review, the total catch for each tow is estimated based on the volume and density of the catch, variables that depend on the catch composition. Furthermore, reviewers watch a randomly selected set of tows (10% for wet boats, 25% for RTVs with a minimum of 1 tow) from the time the gear is retrieved until all the catch has been processed or stowed. For these tows, reviewers estimate the weight of the catch by species and utilization to compare against logbook data.
On the other hand, the AI-assisted process for estimating catch involves different steps. During the review of all tows, EM reviewers use tools to select video frames suitable for catch apportionment when they assess that all catch is aboard. The AI model then processes these selected frames to apportion the catch. The resulting catch apportionment is displayed to the reviewer for acceptance or rejection.
Through the use of AI, a substantial portion of the manual process can be automated. AI models can analyze selected video frames to apportion the catch, thereby diminishing the reliance on human reviewers to estimate catch volumes and densities. This AI application not only streamlines the process but also potentially enhances accuracy and efficiency, enabling reviewers to focus more on data validation rather than the time-consuming task of initial estimation.
The AI-assisted solution aligns with the business goal of enhancing the quality of data collected through the electronic monitoring program by minimizing human error in catch apportionment and lessening the impact of catch apportionment errors on catch estimates (weight). Furthermore, this solution is in sync with the aim of improving the efficiency of the review process, ensuring timely access to catch information. It generates the potential for accurate catch estimates (weight) on every tow in every trip, maintaining the same reviewer time investment, which increases review efficiency and effectiveness. Additionally, it provides the potential for summary data at the vessel, fishery, and regional levels, thereby aiding compliance with total catch quotas.
Model Development and Performance Results
For a detailed description of the AI modeling approach and results, please click here.
Demo
The Proof of Value Phase
The project currently is in the proof of value phase. The process for reviewing involves reviewers following their usual routine, where clips relevant to the participating boats from different tows are extracted from the loaded footage into FishVue Interpret and then loaded into the Proof of Concept solution. The reviewer is tasked with selecting at least 3 to 5 images from each tow for catch apportionment analysis. Afterward, they will review these catch apportionment results, utilizing them to inform and make more accurate catch estimates. Moreover, a sample of tows will undergo a full manual review. This step is crucial as it allows for the assessment of the impact on the accuracy of the catch estimates.
The POV phase will be conducted in 2 iterations to facilitate an opportunity for incorporation of feedback on either algorithm or user interface performance.The Key Performance Indicators (KPIs) for the Proof of Value phase include the following:
- Number of catch apportionment results rejected by reviewers (single images).
- Number of total catch apportionment results rejected by reviewers (tows).
- Accuracy of catch apportionment (deviation compared with manual review).
- Delta in results between first and second iterations.
- Change in administrative user feedback between iterations if any UI/UX adjustments are made.
Next Steps
Following the successful completion of both the proof of concept and the proof of value, the Department of Fisheries and Oceans (DFO) has plans to promote the AI-assisted solution for Pacific trawl fisheries into production in the fiscal years 2023-2024. In addition, the department is also planning another pilot project for the application of AI to monitor Snow Crab fisheries in the Quebec region.