Image-based quantification of scale loss in fish using machine learning and computer vision

Abstract

Ensuring fish welfare is vital for sustainable salmon farming, with skin condition being a key health indicator. Despite stringent regulations, the salmon farming industry is still experiencing high mortality rates, emphasizing the need for innovative solutions to measure and improve fish welfare along with operational efficiency. This paper proposes a system for detecting and quantifying scale loss in fish by leveraging state-of-the-art artificial intelligence methods. The system first detects the fish in an image using a state-of-the-art object detection model. Further, the fish’s skin and scale loss are segmented using two instance segmentation models. Finally, the relative skin area of the scale loss is determined, effectively quantifying the amount of scale loss. To achieve this, machine-learning models were trained with a specialized new dataset. In the test data set, the fish detection model achieved a mean average precision (mAP)50-95 of 87.5% with 24.69 frames per second (FPS) on a NVIDIA 1660Ti GPU. The skin segmentation model achieved a mAP50-95 of 98.0% with 24.15 FPS. The scale segmentation model achieved an F1 score of 76.3% with 5.94 FPS, resulting in an estimated scale loss percentage with a mean square error (MSE) of 0.278 on 10 test images. This resulted in a workflow that analyzes four fish per second on a low-budget GPU with high accuracy. Furthermore, threshold values mapping scale loss percentage to a discrete LAKSVEL welfare score was established, achieving a welfare score accuracy of 85.7% over 140 test images. The test data used to evaluate these models originated from a controlled setup above water. Additional tests were performed using real-world underwater images, understandingly showing that the models were not immediately applicable to the new data. However, the models showed a remarkable ability to be fine-tuned with as few as 18 images, to result in F1 scores of 88.2%, 98.9%, and 61.5% for fish detection, skin segmentation, and scale loss segmentation, respectively. A user-friendly application, ScaleGuard, was also developed to make these advanced models accessible to non-technical personnel.