Ghost Gear Detection using AI

Introduction

Gear detection is a method to pre-determine locations of lost gear to facilitate retrieval efforts. Various approaches exist for lost gear detection and monitoring. Side Scan Sonar (SSS), is one of the detection methods that is increasingly being used in underwater detection as it yields distinctive results pertaining to objects that are visible on the seafloor.

Challenges with Ghost Gear Detection

Ghost gear detection is typically carried out by manually reviewing underwater imagery, either off-site to plan subsequent retrieval operations or on-site to pinpoint targets for retrieval. Local knowledge and historical data help identify areas where ghost gear might be located. However, these processes are time-consuming and challenging, making it costly to efficiently plan and allocate resources.

Furthermore, the expertise needed for more advanced analysis often does not exist in-house. Cleanup organizations must outsource this assistance at a significant cost. As a result, these groups are either left to interpret data with minimal training or with limited control over their data. Additionally, these analyses have been found to yield a high rate of false positives, causing resources to be squandered on incorrect targets and wasting valuable time at sea.

These challenges make ghost gear detection a complicated process, particularly when retrieval projects must operate within restricted timeframes, regardless of sonar analyst capabilities. This renders traditional gear detection methods unsuitable for large-scale retrieval projects.

AI-assisted Solutions for Ghost Gear Detection and Mapping

Artificial Intelligence (AI)-assisted solutions can automate data analysis for lost gear detection, significantly reducing post-processing time and enhancing efficiency in large-scale retrieval projects.

Project Goal

The Office of the Chief Data Steward (OCDS) at the Department of Fisheries and Oceans (DFO) is collaborating closely with the Ghost Gear Program to pilot an innovative platform that enables cleanup organizations to analyze imagery collected during SSS surveys. This platform is designed to automate the ghost gear detection process, making it less time-consuming and more cost-effective. It is built to transform raw data into actionable information, integrating seamlessly into the user's existing workflow. The objective is to enhance efficiency in the analysis process and the location of ghost gear, both during operation planning and active searching.

Project Scope

The initial scope of the project is to develop a Proof of Concept (POC) to automate the detection of lobster traps and ropes, as well as crab pots and ropes, from SSS images, and to map their respective locations.

Object Detection in Marine Imagery

The technical problem domain under consideration is the computer vision task of object detection. Given an image, the task is to identify the types of objects present in the image and localize them by drawing a box around each of them. Because the images are geocoded, these detections also provide the precise geographic location of the detected ghost gear.

This can be accomplished using convolutional neural networks, which are state-of-the art AI models used for many types of computer vision tasks. A prototype model for detections in sonar imagery has already been trained using the open-source library for the YOLOv8 object detection model. By using a technique called transfer learning, it was possible to take a highly effective general-purpose model and train it on a small set of lab-created sonar data to achieve comparable performance to training from scratch on a very large set of sonar data, which would be much more difficult to obtain. Sample detections from the prototype are shown in the leftmost image. In the righthand pair of images, a sample from experiments run with a pretrained open-source model for detection of floating marine debris in satellite imagery can also be seen.

Object Detection in Marine Imagery

DFO is currently working with subject matter experts to obtain and annotate training data from real retrieval efforts.

The Public Service Data Challenge

We are proud to announce that DFO participated in the 2022-2023 inaugural Public Service Data Challenge with this project idea. The DFO team's concept centered on utilizing AI to optimize the cleanup efforts of ghost gear in collaboration with the Ghost Gear Program. Among 150 ideas, DFO advanced to the final event, where the top four ideas were selected for presentation to senior data and digital leaders across the government.

Click here to watch the 2023 inaugural event and to learn more about the project's business case.