Vision technology and AI automates insight into lobster population in the Eastern Scheldt
10 December 2024
The LobStAR project, a collaboration between lobster fishermen in the Eastern Scheldt and Wageningen Marine Research, is a good example of how vision technology and AI can create big benefits on a relatively small scale. Researcher Edwin van Helmond talks about its added value and future uses.
“The LobStAR project is quite a small project by Wageningen standards. The launch of the ‘Lobster Stock Assessment and Regulatory approaches’ (LobStAR) project was initiated by lobster fishermen in the Eastern Scheldt. They came to us at Wageningen Marine Research (WMR) asking if we could help them with assessing the lobster stock and population, and its development. Their goal was to be able to regulate the fishing industry in a more data-driven way and respond to social criticism about overfishing of the Eastern Scheldt at the same time.”
Fishery lines lottery
Van Helmond explains that the imaginary lines in the Eastern Scheldt along which each of the current 42 boats is allowed to fish are allocated annually by means of a lottery. “However, this current form of regulation through fishing licences is not linked to the size of the lobster population. But that is something the fishermen want. To understand the size (length) of the lobsters (Homarus gammarus) and thus the age distribution and number of males and females. With that data, you can then obtain insight into the population with standard stock assessment models and methodologies. Usually, during lobster fishing, observations and measurements are made by professional observers on board. However, the budget was insufficient for that in this case. Furthermore, there is only limited space on board the small lobster vessels to measure each lobster caught, determine its gender and record all the data. We therefore invented and developed the CatchCam for objective automatic catch registration.”
Lobster length and gender determination
During the project, which ran from 1 July 2021 to 30 October 2023 and was funded by the European Maritime, Fisheries and Aquaculture Fund, six fishermen used five CatchCam systems. “The system is really just a standard visual 2D RGB camera in a metal housing with the lens pointed at a small platform. On that platform, you place any lobster caught or any lobster which needs to be automatically analysed. At the touch of a button, a colour photo is taken of each animal. The photo is then uploaded to a WMR server via the mobile network. There, an algorithm determines the length of the carapace, the hard shell that covers the lobster. Its length indicates the lobster’s age. And although fishermen determine the gender of a lobster based on gender characteristics on the underside of the animal, we can also do that using the CatchCam methodology based on a photo of the animal’s upper side. Together with the carapace length and gender, that gives insight into the sexual maturity and reproduction of lobsters, which is a good indication of population growth. In turn, the length distribution of the carapaces provides information about the composition of the lobster population.”
Direct feedback to the fisherman
To give fishermen insight into the length and gender of the Eastern Scheldt lobster it has just photographed, the CatchCam sends the data from the server back to the fishing boat immediately after a measurement. There, the length of the carapace and the gender are projected into the picture of the animal just taken on the camera screen. “That wireless communication is a deliberate choice to keep the on-board system as simple and robust as possible, but it could also potentially cause occasional connection and network problems,” Van Helmond points out. “For faster data processing and communication, we might use an on-board Jetson Nano computer in the future. That could also be a solution for measurements in places where mobile network coverage is problematic.”
LobStAR CatchCam used on a lobster fishery vessel during the research project
Challenges
“Beforehand, we were mainly concerned about the mobile network coverage on the Eastern Scheldt, but that turned out to be fine. Another challenge initially was the automatic length measurement of the carapace. And particularly the two annotation points at the front and back of the lobster. From WMR, we initially annotated the images ourselves to train the underlying algorithm. We can now correctly predict 76 percent of male lobsters and 70 percent of female lobsters based on CatchCam images and the algorithm. The keypoints are on average 6 px off compared to the annotation points indicated by humans.”
Much more versatile
Before developing the CatchCam, WMR investigated whether technologies were available elsewhere in the world in the lobster fishing industry that would be suitable for application in the LobStAR project. Edwin van Helmond: “In Wales, they use a similar system, while in Scotland a 3D scanner is used for crabs and lobsters. However, that scanner is much too big for on board a lobster boat. We therefore decided to develop our own system, also for cost reasons.”
The system and its reliability have since attracted the attention of other projects. “From 1 January 2025, we will be using the CatchCams in a PPP project studying mussel and lobster mortality funded by LVVN’s Top consortium for Knowledge and Innovation (TKI) Agri & Food.”
“I feel that the strength of the system really lies in its simplicity. In terms of both technology and use. You can easily apply it anywhere in the world and power it with a battery or solar panels if necessary. Whether that’s in Europe, Africa or the Caribbean. At the same time, some aspects of the system could be a bit more professional, so if there are any companies willing to help us with that, please do. The same applies to other researchers with good ideas for the technology and its applications. For the time being, it is an incredibly useful system for research purposes and, as a biologist, that is what I am mainly interested in.”
