“AI technologies and humans will integrate, just as scientific disciplines will merge”

Once every two months, we introduce you to a specialist. We give an insight into the person, their research and their expectations. This time it’s the turn of Vision+Robotics researcher Joseph Peller. He moves from spectral imaging to segmentation, phenotyping and working conditions in greenhouse horticulture: “As a research associate, you need to be able to take a more multidisciplinary approach.”

“You can’t be just a roboticist, botanist or agronomist in academia any more. The new generation of WUR academics combine computer science with their specific specialist knowledge. They know how to programme and understand why an organism grows the way it does.” Says WUR research associate Joseph Peller. You can’t stop that development. Data keep coming and are becoming increasingly accessible. Also for farmers, fishermen and growers who will soon be able to read and analyse data on the spot. “AI technologies and humans will integrate. Just as scientific disciplines will merge into each other,” says Peller. “There are so many interfaces. Technology that you learn in one discipline can sometimes be effortlessly applied in another. So, as a research associate, you need to be able to take a more interdisciplinary approach.”

Camel City

He himself is the perfect example of that. He is a physicist trained by an astronomer who works with spectral imaging and AI in agriculture and the fishing industry. Amazing, he laughs, for a boy who grew up in the US town of Winston Salem (population almost 186,000) in North Carolina. The town’s nickname: Camel City. For a reason. Winston Salem is home to the headquarters of the country’s largest tobacco producer. Peller particularly remembers the temperature. “It’s hot and humid there, up to 40 degrees in summer.”

Optics and cancer research

He obtained his bachelor’s degree from Appalachian State University in hilly Boone, North Carolina. Main subjects: history, physics and theology. Graduated in physics, specialisation optics. With those subjects, he could join the army (night vision) or ASML in Connecticut (lithography systems and machines) or opt for medical research. He decided on the latter. More specifically, he did a PhD on cancer research. His mentor was an astronomer. She says: “Like cancer, you can’t touch a star. If we want to trace that disease, we need the same techniques we use to find stars.” She introduced him to the world of spectroscopy and special imaging. For 200 dollars, he put together a spectral camera. “It’s a bit slow, but technically fine.”

For cancer research, you need people’s consent. At that moment, they are going through the worst time of their lives, he says, and then someone comes up to their bed and asks if they can put this camera in their body. Of course they say yes. Perhaps you’ll increase their chances of survival. Too often, he’d be told by a surgeon: we don’t have research data on this patient any more. We need to close this file. Peller: “I wasn’t mentally strong enough for this research.”

Unpronounceable village in the Netherlands

So, that was the situation. Six years ago. He knew what he didn’t want to do. But what did he want? Every week for four months, he sent eight job applications via LinkedIn, he says. Companies, institutes or research centres that had a vacancy in the area of optics. And always including a letter saying I was looking for “something fun, something to keep myself alive”. To organisations and institutions worldwide. In Budapest, Australia, Istanbul, New Zealand, England, and an unpronounceable village in the Netherlands.” And they responded. They were looking for someone to help with plant research.

Peller looked at his girlfriend. They didn’t have any children or a dog or a house, and they were longing to go somewhere in the world where it was less hot. So why not Wageningen? Peller: “We said, we’ll give it a try for a couple of years. In case it doesn’t work out. That was our grand European adventure.” At WUR, he went to work with Gerrit Polder. Who said: nice to meet you and that you mentioned my work in your PhD. “I thought, oh my god, I based half of it on his work.” Yes, that name, Polder, G. But he’d never looked up who that was. “So there I was, face to face with the main man of spectral photography.”

Joseph Peller at work with Gerrit Polder

Imaging technologies

As a research associate, he focused mainly on imaging technologies. In simple terms, they are cameras with a brain. Old photographic technology combined with state-of-the-art computer technologies. Like AI and machine learning. Peller: “You look at a picture, and at the same time you obtain all kinds of information about the things you can see and particularly those you can’t see directly.” Those cameras get smarter as technology gets smarter. Which is kind of fun, he says. “In America, I worked with a 200-dollar spectral camera I designed myself. Now I use super high-tech spectral systems that cost 40,000 euros. That’s quite a step-up.”

In the first year, Peller focused mainly on plant health. A sick person is very similar to a sick plant, he says. When you get skin cancer, diseased cells start multiplying under the skin. That produces changes in the surrounding tissue. “Collagen keeps your skin healthy, young and supple. That protein hardens to protect the diseased part. You can see this under a microscope if you take a skin sample and pulverise it. “Oh look, there’s more collagen here than there.” With spectral imaging, that pulverising isn’t necessary.

Chemistry without a microscope

Peller describes spectral imaging as chemistry without a microscope. “You take a picture and immediately see how the light is absorbed by that collagen. A dark spot means something is going on there. Because that area has hardened.” Spectral imaging also allows you to detect diseases in a plant. If it has suddenly yellowed because of a fungal infection. A fungus absorbs light in a different way; it looks darker. With the naked eye, you only see that yellow area. “I think they hired me at Wageningen on the basis of my cancer research.”

He is working on a project about tiny insects. Barely visible. You can follow how they chew through a plant, though. They do this in much the same way as mildew, Peller says. That penetrates cells with tiny fungal threads to drink all the chlorophyll from them. “In doing so, those mosquitoes leave small spots on the leaf. Once you have traced them and you can see how they spread over that same leaf, you can say, I think it’s this disease or pest.” Fish, plants, people: spectral photography can be applied to any living organism, says Peller, including fish. “A fish is nothing but a weird shaped fruit.”

Segmentation

To create spectral images and encode what you see, you need to be good at segmentation, says Peller. “You need to know that this round thing is a coin, while that round thing is a marble.” People can tell the difference. To a computer, all round things look the same. Also leaf shapes. Tricky when taking a photo in an orchard or greenhouse, because there are lots of leaves there. “In medicine, you don’t have that problem. You just take a picture of tissue that has been removed from the body. It’s already isolated.” When it comes to leaves, you need to train the robot to isolate leaf shapes. “Like people in other countries teach robots how to recognise furniture, cars, faces and so on in images.”

Phenotyping

Spectral imaging tells you all about the chemical composition of a plant. With segmentation, you study the physiology of that same plant such as stem length and leaf size. Through that research, Peller entered the world of phenotyping almost automatically. “In traditional phenotyping, researchers sit with a ruler between plants and measure the differences between leaf sizes and lengths,” says Peller. A pretty labour-intensive job. Some people are really very good at that. “By looking closely at a tomato, they can establish the value of wrinkling.” But it takes thirty years before you master that trick. So people asked him: “Aren’t there any digital tools that could help us do our work faster?”

In processing, it is less about spectral imaging and more about imaging technologies. “Spectral images typically have a lower resolution. For processing, we usually use very detailed colour images and apply AI and Machine Learning to segment the images based on forty traits on which you test whether a plant has a different phenotype. That ranges from flower petal and colour to leaf shape and position of the leaves.”

New phenotypes for climate change

WUR also employs phenotypers. Focused on climate change. They are trying to develop new genotypes or phenotypes that perform well in dry or saline soils. That performance is measured and captured in data. Peller: “Instead of saying, the leaves on this plant point down, you can use imaging technologies to establish that those leaves point 10 degrees down.” You can compare these data with other agriculturalists in Europe. “At European level, it’s been decided that they will measure in the same way.” However, it took a long time to reach that decision. Peller smiles: “I’d be sitting in big rooms with angry Europeans shouting at each other about harmonisation. And at the end of the day, they’d then agree that they were going to form a subcommittee.”

Labour issues

Peller helps phenotypers take the pictures, investigate whether a leaf does indeed point down at a 40-degree angle, and record and measure all these data. “How do you measure a leaf that is hanging at 40 degrees without touching the plant itself? And how do you teach a camera that the leaf in the picture is hanging at a 40-degree angle?” Here he increasingly focuses on the bigger picture in the greenhouses. On labour issues, which is actually where his heart lies, he confesses. Because things are happening in greenhouse horticulture that we really need to solve together. “However much I enjoy plant health and coding.”

It might sound strange that he is concerned with optimising the working conditions of labour immigrants in greenhouse horticulture. But it’s not, he emphasises. “Food production isn’t just about what grows and flourishes. It all starts with agriculturalists developing a seed for a crop and ends with the people who harvest, pack and sell it, along with everything else involved.” And yes, if you’re in the greenhouses anyway, why not talk to the people working there to see how you can use modern technology to optimise their tasks. He listens to them, talks to leaders in greenhouse horticulture and has discovered different ways of using technology to improve working conditions in Dutch greenhouses. “These are mainly set up for efficiency and plant growth. They are designed for plants not for the people working there.”

Cobots and ice packs

In the greenhouses, it’s really hot and humid. Vests with ice packs can then provide cooling. “Or sensors on your wrist that indicate when you need a break.” Peller also mentions exoskeletons or cobots. Tools that relieve people from physically demanding, dangerous or repetitive tasks, thus making greenhouse work more attractive. Cobots have already been introduced in production centres and warehouses. Trust is vital here: “You can’t load up employees with sensors and other devices and then spy on them in a control room. Checking whether they are working hard enough and not taking too many breaks.”

Technology is democratising

No, Peller doesn’t believe you will see robots everywhere in greenhouses in 10 years’ time. In the field, in the barn, in a greenhouse or on a boat: you will always need people. Along with robots that work better and more autonomously. That simultaneously makes them quite expensive and inaccessible. However, you do see small technology becoming more and more affordable and literally getting into the hands of horticulturalists, growers, fishermen and farmers. “They’ll soon be able to research things themselves on their phones. Like photographing organisms and forwarding their photos to the cloud. Before processing these data themselves in their own database.” The more they do this themselves, the more they manage their own data and can try new things. “That enables us to further innovate and think ahead.”

Wonderful world of WUR

Peller has evolved from someone working with plants to someone working for people who work with plants. A logical path, he thinks. “I started by looking at diseased plants. Now I’m looking at the bigger picture. Also to avoid people who work with plants getting sick.” So his work mainly has a social, ecological and economic side. That’s when a holistic view helps. Without a pronounced agricultural background. This means you can switch more easily, he says. From fish to agriculturalists, from wine growers to insects. “I start every day here as if it were my first day. Which is how I discover more and more of the wonderful world of WUR.”

Research associate