Can you picture it? A flexible robot that by itself assembles a packet of soup ingredients or a bag of bami vegetables by itself. WUR researchers from Vision + Robotics are taking up that challenge within the NXTGEN Hightech programme set up by the Dutch Government to boost the high-tech industry in the Netherlands over the next five years.
A tomato is not a spark plug
This agri-food project within NXTGEN Hightech connects nicely to ‘PicknPack’, an EU project led by WUR’s Vision + Robotics that ran from 2011 to 2016, in which robot modules were developed to pick fresh vegetables from harvest crates, measure their quality, sort them, and package them, among other tasks. It’s pretty tricky, according to Paul Goethals, business development manager of Vision + Robotics. You’re not talking about car parts that always have the same shape and weight; you’re dealing with ‘delicate’ products with slightly different shapes all the time. No two cucumbers, peppers or courgettes are exactly the same. And a tomato is not a mandarin. So, a robot gripper needs to have a good view of what it is getting its hands on. That is the big difference from the automotive industry, for example, where they have already been working with robotics for quite some time.
Machine learning for package robots
Goethals explains: “What we are doing now for example, is putting pieces of pepper in a salad container, in an arrangement that looks appetising as well.” Grabbing, weighing and distributing from the packing crate – that robotic arm has to be able to combine all this. The more it learns, the better it becomes. This is machine learning: it has learned to take decisions using a set of basic images. “It can keep refining its algorithm with new images and in new situations, work faster and faster, and make decisions more and more easily. Once it gets that right for pieces of sliced pepper, you can move on to pieces of carrot or anything similar.
Sensors and robotics in agri-food on the rise
Companies were in no rush to adopt PicknPack in 2011, recalls Goethals, because applications were not yet working well enough outside the laboratory and there were still plenty of ‘hands’ available. But interest in sensors and robotics within agri-food has grown enormously in recent years. This is due to the leap in technology development, but also has a lot to do with the now huge shortage of hands within the food processing industry. It is not a nice job either, working on a production line, shoulder to shoulder in a cold factory hall, sometimes wearing triple-layered white suits, coats and hats because of high hygiene standards. “Robots can take over the role of humans in more and more places there, especially if we succeed in collecting and packaging all the loose ingredients of salad meals, packets of soup ingredients, and nasi vegetables with a machine.”
Salad meal assembly
Together with partners from the business world, Vision + Robotics experts at WUR will see if they can make that happen. It is quite a challenge. For this you need a ‘self-thinking’ robotic arm equipped with numerous gripper tools. It has to pick up one salad meal ingredient from a harvest crate with one gripper, and another from another crate with yet another gripper. And it also has the job of putting both nicely in one and the same packaging box. What’s more, such a flexible robotic arm also has to be able to change tools quickly and automatically, and even more so at a time when batches of salad meals are getting smaller and smaller. Just look at the huge number of salads you can get from your supermarket. Goethals: “If you have to modify your production line for every variation, that’s unworkable. It means you suddenly have hundreds of people on that line standing around doing nothing for 20 minutes. With a flexible robot, you can respond to that trend of mass individualisation.”
Quality control is still largely a job for humans because obviously people are quick to see if fruit still looks okay, like whether a peach is rotten or overripe or has dark blemishes. And otherwise, they can smell that. But sensors might be able to take over those eyes and hands. Goethals: “We are also working on developing non-destructive sensing, quality measurements without having to cut open fruit or vegetables.” There is a whole department dedicated to that within Vision + Robotics, that works with hyperspectral cameras among others, looking into how to measure quality parameters, such as sugar content, moisture content, and ripeness or hardness. They also look to trace the early development of decay. “We increasingly understand which signals from such a camera relate to specific quality parameters in fruit and other food products. If you can build this kind of technology into sorting and packaging machines, for example, you can make better logistical decisions and thus not only reduce food waste, but also better meet consumer expectations,” concludes Goethals.