Maintaining biodiversity with vision and robotics

Every two months, we introduce you to one of our specialists. We give an insight into the person, their research and their expectations. This time it’s researcher Janne Kool: “We need more nature knowledge. To avoid making decisions that could be disastrous for maintaining biodiversity.”

“Obviously, it’s amazing that the advent of robotics enables farmers to use fewer pesticides because they can hoe and weed in a more targeted way,” says research associate Janne Kool from the Vision+Robotics programme at Wageningen University & Research. “At the same time, it’s ironic that nesting farmland birds are at risk precisely because of that hoeing. These birds like to hide their nests under large weeds. Which means that algorithms don’t see them.”

Zwanenwater and NJN

Kool grew up in Callantsoog in the tip of North Holland. She has always loved being surrounded by nature. Near her house is the Zwanenwater nature reserve, with the two largest natural dune lakes in Western Europe. She’s never tired of walking in the reserve. “From late May to July, the most beautiful orchids grow there,” she says. “And you will always find a colourful palette of the loveliest flowers and plants.” At the age of 14, she joined the Netherlands Youth Association for Nature Studies (NJN). She became an active member of the plant working group. “I loved plants even then. That was partly because I have bad eyes. Birds fly off, plants don’t, so I could see them.”

The more unusual the plant, the more fun it is to identify it using the Plant Key. The shape of the leaves, how they are arranged, what the edges and veins look like: “I was very good at that.” Kool was destined for WUR but didn’t go there. She disliked biology and all those plant and animal sciences because she felt they were not precise enough. “You learn all the theory and then it turns out to be different. I couldn’t handle that uncertainty.” She chose maths. “Once you’ve proved a mathematical theorem, it’s just there, it’s a solid fact. I liked that.”

She graduated and obtained a PhD in geometry. Kool: “I went for the hardest, and most complicated.” But there was no denying: it was in her blood. Her PhD supervisor asked her: what are you going to do after this? “I might continue in academia,” she replied. “But doing ecology instead of mathematics.” She found a position as a postdoctoral researcher in Copenhagen. Applying algebraic geometry in biology. The best of both worlds. Had fun, learned a lot, but strangely the Danes didn’t allow her to talk to biologists. Biologists won’t understand you anyway, they told her. “Odd, when you’re doing applied biology.”

Technical camera to count insects

In 2017, after two years, she decided to do something else. She went to the Technical University of Denmark in Kongens Lyngby, just outside Copenhagen. Earlier, she asked friends working in ecology: how often is a flower visited by an insect? No idea, they said. Kool had found her subject: designing a technical camera to count insects. It allowed her to explore AI at the same time, to get more entrances to applied science. And: “I’d heard that in breeding, they like to know which bumblebees come to which flowers how often.”

After her Danish postdoctoral adventure, she returned to the Netherlands. Ideally, she wanted to go to Utrecht. With her husband. He found a job in Amsterdam (he now works on the Wageningen campus), but no, neither wanted to live there. A friend of hers worked at Biometris in Wageningen and lived in Bennekom, so they ended up near WUR. With her camera story in mind, she wrote an open job application to WUR and got a job in the Agrosystems department. Above all, she wanted to do useful work there, she says. To protect nature. “That has always been important to me. Particularly the loss of biodiversity. 30% of plant and animal species are facing extinction and will be lost forever. It really worries me. All that extinction sometimes scares me too.”

Maintaining biodiversity with vision and robotics

Her plea. Use vision and robotics to maintain biodiversity. With the development of an advanced cultivation system where farmers are less dependent on chemical agents and can still grow diverse and small-scale crops. With room for biodiversity and enough production to live on. Kool: “I want to contribute to projects that help make that happen.” One of these projects is a study by the Buijtenland van Rhoon regional cooperative near Rotterdam. Here they have introduced ‘nature-inclusive farming’ on a large scale, in other words agriculture that respects nature. No chemical pesticides to kill weeds, but regular weeding with camera-controlled hoeing machines. Disadvantages of mechanical hoeing. “It stirs everything up, leading to more disruption for soil life.”

And then there are the farmland birds. Kool: “Camera-controlled hoeing machines remove their nests, with the birds inside. And even fawns and many other species that grow and thrive there and don’t look like a crop.” In theory, AI-controlled hoeing is fine, but you need to fine-tune the technology better, Kool believes. To preserve those nests, calves and biodiversity. It’s not even necessary to remove all the weeds, because not all weeds are unwanted. “With good AI technology and data sets, it should be possible to weed selectively. Only remove weeds that you don’t want. So that you maintain a more covered soil and a more biodiverse arable landscape that also might be more productive.”

To prevent lots of farmland birds from being ‘hoed away’ or ‘mowed down’, volunteers have taken 600 photos of bird nests from the same distance that the hoeing and weeding machines have their cameras. Kool: “Lapwings make very little effort to build a nest. Just a few twigs, that’s all. Other birds do their best to hide their nests among the greenery. Like the lark and yellow wagtail.” At Sovon Bird Research, they are collecting even more photos. Taken all over the Netherlands between 1 April and mid June. After that, the crops are too high and nests are less likely to be found. “We can already detect 90 per cent of nests with datasets and algorithms.”

Feed camera-controlled hoeing machines with additional data

Time for the next step. We need a machine builder at the table. No need for an additional sensor for their weeding or hoeing machine. The hoeing machines are already camera-controlled. Kool: “You just need to feed that camera with additional data. So that it automatically thinks: ah, there’s a nest there, I won’t hoe there, I’ll just raise my blade.” The nests are only in a few spots in a field, you really don’t need to spare a large field. “Hoeing is done more often than spraying herbicides. If the machine remembers the location of a nest in a field, it can take that into account. Weeds keep returning and may eventually completely cover a nest so that you no longer see it.”

Many birds could be saved if nest-tracking algorithms were to be added to machines such as camera-controlled hoeing machines.

Birds lay their eggs in spring. Some birds have two nests if one is removed earlier. If a machine builder adds a nest-tracking algorithm to his machine, Kool could save many birds. A great additional selling point too. Especially if the algorithm counts how many bird nests it has saved, you might possibly get a subsidy. Kool: “I saved 7 redshank eggs and 8 skylark eggs.” That’s mad, she thinks. As an arable farmer, you have a duty of care for everything that happens on your land. You shouldn’t mow those nests away. “At the Bond Friese Vogelwachten, they use drones to track bird nests in fields. Meadow bird drones with thermal imaging and soon AI.”

The connection between techies and ecologists

Her bird research proves it: every technique you develop can have consequences you did not predict. But as a developer, you need to consider that beforehand. Just recently, she gave a talk on the project in Rhoon at WUR. She was asked a question that totally confused her. “Why don’t lapwings just nest in trees?” Hold on, she thought. You work at WUR and you think all birds build nests in trees? From a different angle: “Oh, so there’s flora and fauna in the field too.” “I thought you only found them in nature reserves.” Even techies at WUR can improve their nature knowledge.

The intermediate. That’s what Janne Kool wants to be. The connection between techies and ecologists. To safeguard the biodiversity of nature in our country. The two groups don’t understand each other very well, she says. Ecologists are always slightly overambitious with respect to ecological issues. Techies are primarily concerned with technical capabilities. Without considering the possible harm to nature. “Within Vision + Robotics, we need more nature knowledge. To prevent us making decisions for AI applications and other technologies that could be disastrous for maintaining biodiversity.” As humans, we are very good at destroying it. No one does that on purpose. But it happens. Out of ignorance.” Conversely, she believes that many ecologists are unaware of how many technological opportunities lie within their field. “Which is why I am trying to set up a biodiversity group within Vision+Robotics. To connect ecologists and techies here too.”

Herb-rich grasslands

Kool is also researching herb-rich grassland. Fields with red and white clover, sainfoins and other leguminous plants. Plus herbs like ribwort plantain, chicory, wild carrot or yarrow. A heaven for birds, butterflies and insects. For centuries, fields were full of them. All decimated and sacrificed to the laws of modern agriculture for miles of boring ryegrass. Which grows fast and has a high nutritional value for cows, so it’s easy. But not good for biodiversity. You can sow that with herbs, but that doesn’t work well. “The second year, almost everything has gone. Perhaps coming back afterwards, depending on species and soil.”

Opting for herb-rich grass is a long-term project. She wants to encourage farmers with a reward if they choose herb-rich. But you must be able to check it properly. Not by ticking a checklist on a form: yes, I have herb-rich grass. No, she looks for evidence through satellite data and photographs. With a team consisting of, amongst others, an ecologist and a remote-sensing specialist from Wageningen Environmental Research, they record the herbaceous richness of a piece of land and for example the NDVI Index, the Normalized Difference Vegetation Index. “From the sky, you measure from the colours of green. Based on that, you can see how often a plot is mowed. The more often it is mowed, the more intensive the cultivation.”

Everything in nature is interconnected

She looks enviously at England. There they catch aphids, the torment for potato and sugar beet growers. Because they transmit viruses. Kool: “They eat themselves fat and continue to reproduce until it gets too crowded around them. Then they get wings and fly away.” Sometimes they drift 80 kilometres in the air before whirling down to continue eating and reproducing. If they haven’t been caught in suction traps: 12-metre tall traps that suck in passing airborne insects. “They swirl down the funnels into a collection point where they can be monitored. The numbers, species and the viruses they carry.”

Thanks to these traps, they can accurately predict the number of aphids in England as early as April, with the viruses they carry. The harsher the winter, the fewer the number of aphids. And predict whether or not farmers should use neonicotinoids further into the season. “A harmful toxin that is actually banned but for which exceptions are sometimes made,” says Kool. In the Netherlands, we have a few of these suction traps – including on the site of the Dutch Potato Inspection Service. She would like to have them in more places. “Everything in nature is interconnected. It’s important that we all remember that.”

Researcher Vision+Robotics