Sensor technology is like having superpower vision - VisionRobotics

Sensor technology is like having superpower vision

11 October 2023

Once every two months, we introduce one of our experts. We give an insight into the person, their field of research and their expectations. This time it’s Puneet Mishra, Senior Scientist in Advanced Sensing Technologies and Chemometrics. “Feeding people is a noble cause.”

The scent of marigolds came as a bit of a shock to Puneet Mishra. He hadn’t expected to encounter these flowers in Wageningen’s Belmonte Arboretum. He had to stop a moment and look, and there they were catching the sun, and he was immediately transported back to India, to his grandmother’s house. “She grew yellow marigolds in pots, ready to use them in October for diwali, the great Indian festival of lights. She played a big role in the choices I have made in my life. She passed away recently.”

Apple trees near the Himalayas

Mishra (31) is a Senior Scientist in Advanced Sensing Technologies and Chemometrics at Wageningen University & Research. He grew up in Nainital, a mountain resort town with 400,000 inhabitants in the state of Uttarakhand at the foothills of the Himalayas, almost 2,000 metres above sea level. It was a popular hill town in the days of the British, who flocked there to cool down during oppressively hot summers. You can still find a lot of British colonial heritage.

His parents worked for the government – his mother in social security, his father in forestry policy. At weekends and in summer, he would spend a lot of time with his grandmother. His family owned several orchards and plantations during the British Raj. His grandmother still lived on a plantation with mostly apple trees, in addition to peach, plum and pear trees. Her house was quite isolated, a five-hour walk from Nainital through the mountains. She would have pots of everything, he recalls. There were limes, chillies, aubergines, courgettes, and flowers. Mishra: “As an eleven-year-old boy, it took my breath away just looking at them. She put so much love and attention into caring for fruit and vegetables. It contrasted with the mechanisation used to maintain the orchards.”

Mangoes and Nokia

At weekends, his grandmother would sometimes take him to visit her sister, 40 kilometres away. She grew sugarcane, wheat, black chickpeas, and mangoes. He would make his own sugar water and climb the mango trees. Even back then, he knew that he did not want to become a fireman or a famous cricketer, but would end up in the world of food and agriculture. And he would do something to do with technology because that also fascinated him as a boy. Mishra: “I collected articles on the latest technological developments and stuck them in a notebook. Nokia, that smartphone brand, that was really something. You don’t want to know how many of those phones I took apart and put together to find out exactly how technology works.”

Inspired by Borlaug

Near to where he was born is Asia’s first agricultural university, G.B. of Agriculture and Technology, also known as Pantnagar University. It was established in 1960 in collaboration with several US agricultural universities to promote agricultural education in India at the time. Numerous seeds have been developed there that are suitable for drier environments, more disease-resistant and better for higher yields, including the renowned Pantnagar Seeds, a household name in rural India. Mishra: “Between 1965 and 1970, the wheat harvest in our country no less than doubled as a result, spurred on by modern farming techniques introduced by agronomist Norman Borlaug.” He is known as the father of the Green Revolution and received the Nobel Peace Prize in 1970 for his commitment to improved food production. “Borlaug described Pantnagar University as the harbinger of that green revolution, and yes, I wanted to be part of that.”

Postharvest technology

Mishra chose to major in agricultural engineering, specialising in the field of

postharvest technology – managing and transporting harvested crops. His thesis was about storing and drying mushrooms. He designed a solar dryer powered by the sun for this purpose. After graduating, he sought to broaden his horizon. He was awarded an Erasmus scholarship to study for two years, so in 2013 he departed for the agricultural institute of the Universidad Politécnica in Madrid as a 21-year-old. His objective was a master’s degree in postharvest sensors because sensors were the future, he was quick to realise, for detecting information from flowers, fruit, meat, and vegetables. He conducted exploratory research in the field of X-ray, ultrasound, and infrared spectroscopy. Mishra: “A particularly instructive period where I also learnt about modelling.” He was like a child in a sweet shop in Madrid. “I was very happy because I got to work a lot with machines and sensors.”

Peanut residues in biscuits

After Madrid, Mishra headed off as a visiting researcher to IRSTEA in Montpellier and to AgroParisTech, where he learnt even more about modelling in relation to agricultural technology. In Montpellier, he researched peanuts. “Many products, including biscuits, contain peanut residues that you can’t see, which is annoying for people with a peanut allergy. Using a special infrared imaging technique, I was able to trace peanut residues in a biscuit or flour, even if they were hidden behind or in other layers.” For that, you also need modelling to help you out, to extract information from those hidden peanut residues. Mishra: “Where exactly are those peanut traces hidden in a biscuit, and how much are we talking about? You need that data so that later you can identify them more easily and sooner before you start packaging the biscuits. That can prevent a lot of bother for people with a peanut allergy.”

Centre pivot irrigation in the desert

A new job awaited him, at the new King Abdullah University of Science and Technology (KAUST) in Thuwal, Saudi Arabia, where they conduct agricultural research into sustainable agriculture in the middle of the desert. Mishra: “You can see these places on Google Maps and from a plane – large circles, centre pivot irrigation, sometimes over 800 metres in diameter in the sand of the Tabuk region near Jeddah and the Red Sea.” The water sources are sometimes two hundred metres deep and contain groundwater that is no longer replenished, known as ‘fossil water’. It fell on the earth thousands of years ago as rain. “As a visiting researcher, I helped install the new sensors they had bought. I also designed an infrared detection tool that allows you to measure the moisture and salt content of the soil on the spot.” This was new. Mishra: “Before that, they researched this using a satellite, to see which plant varieties could survive in that soil, and how they responded to changes caused by flooding with salty seawater from the Red Sea.”

The greenhouse of Ghent

Oh yes, he could stay. He was also invited to do a PhD, but he found it far too hot to live there. Despite the lovely bungalow and all the other fine things, he could expect there, Mishra wanted to return to Europe. A great assignment lay in store for him at the University of Antwerp in collaboration with Ghent University. There, they wanted to know what maize to grow in arid countries. He found himself in greenhouses that were fully automated with sensors for monitoring plants. He left after a year. “I worked in Antwerp, but got the data from Ghent. I didn’t have to collect it myself.” And that’s what he loves doing most, doing the work himself, with robotics, sensors, and automation. “But there I was working with just data, not with the sensors themselves. I find that a bit boring.”

Tea and medicines

There was one more stepping stone before Mishra ended up at WUR. In 2016, he received a three-year Marie SkłodowskaCurie European PhD scholarship for the international Modlife programme with 15 like-minded people spread across six European universities and companies. He started in Scotland with a group working on process analysis and control technologies. In practice, this involved implementing sensors and data analysis methods to monitor product properties and processing, among other things. He found himself at the Unilever Research Centre, where he got to analyse tea growing in a simulated environment in greenhouses. He also ended up in the pharmaceutical industry such as at Bayer, Leverkusen, where he got to work with very high-end automation. Mishra: “Pharmaceutical companies use a lot of sensor technology to monitor the drug manufacturing process. Every product has to look the same.” You don’t check that by picking up every bottle or pill; no, they do that there with sensors. “Technically very advanced; yes, I enjoyed it.”

Crisps in Wageningen

Completing his PhD meant that he could work at the biggest companies. But he had a better idea. Mishra: “I wanted to use the things I had learnt about sensoring at pharmaceutical companies within the food industry because those applications did not yet exist within food processing at that time. That’s why I am here. Because food, just like medicine, needs to have the same high consistency of quality.” Every Lay’s crisp from one pack must have the same quality, bite, and flavour as a crisp from another bag. “For that, you shouldn’t need to inspect each crisp several times throughout the manufacturing process; sensors can do that really well.”

Sensors in the abattoir

Mishra has a current example. He is working on a project in the abattoir of a large pork producer that wants meat of a very high quality, of course. “We are placing sensors there to measure the iodine value, the unsaturated fatty acid, of back bacon.” Back bacon consists mainly of fat and is used in various meat products. A poor iodine value means lower meat quality. When you know that, you can optimise that value because it should not fluctuate too much. If you feed the pigs all the same good food, their meat will have a higher iodine value, so that the fat quality will be better, and the meat will fetch a higher price. “I advise on how to use the sensor properly, what the best way is to measure that iodine value and to model the data.” Mishra is also working on the Spectral Cabinet, a machine that can determine the sugar content of a crate or bowl of fruit in real time and on the spot. “That device can use a picture of everything from that crate or bowl to determine the sugar content of each individual apple, orange, pear, grape and so on. Normally you would need to go to the lab for that; now growers can just do it anywhere on their farm.

Finally, in Wageningen

So, here he is in Wageningen. And it’s a wonder that he ended up here at all. Picture this: he applied to WUR back in 2016, to work as a technology consultant for agricultural technology. But he didn’t get the job. In 2019, he applied again, and again he missed out. Mishra: “I think someone fished my CV out of the pile. They came up with a new position for me, specifically relating to sensing. And they asked me whether I wanted to apply. That topic had not really taken off here before. They had struggled to find someone with knowledge of agriculture, machinery, and data. So, finally I came along at the right time.”

Sensor course for farmers

Developments in sensoring are happening at lightning speed, making it difficult to pinpoint trends. He sees that sensoring always excites people in the field. “People with little to no technological knowledge are keen to start using sensor technology straightaway.” These include farmers, fishers, and growers. For the latter, for example, it is important to know when to pick the fruit from the trees. Mishra: “The right time to pick depends on the sugar content and moisture content. That determines 90% of the freshness, crispness and all those things. If you can measure that precisely, you can also estimate the rest of the ‘lifecycle’ of the fruit, which countries you can transport it to or how long you can store it. So, you want to do that at the right time.” Many farmers in the Netherlands and Belgium are already using this sensor technology with the help of their phones. And they are bound to grow in number; hence his new course in sensor technologies specifically for that target group. “To teach them about the benefits of this technology and how to use it.”

AI as feedback and feedforward

The advent of AI is acting as a booster, in modelling, and that is his second insight. Especially for research where you have stacks of data, says Mishra. “Modelling is based on mathematics and the like. With AI, you can automate a lot of data that you used to have to work out with your hands and your head.” It is easier for him because he did all that data processing with his head and hands first for years and years. “It gives me a good understanding of what I don’t have to use that for.” A third trend: robotics. “Self-learning automation systems with AI as feedback and feedforward.” And that does bother him a bit. For medical research, money in relation to sensoring always became available very quickly and easily. That’s not the case for agricultural research, even though you can’t separate health and nutrition. “The department where I used to work in Madrid is closing because they cannot get sufficient funding. Fortunately, government agencies are now also seeing the importance of sensoring within the agriculture and food processing industries.”

My mission

“I want to make an impact, across the world. And I can do that in Wageningen. That’s because this is where the whole world of agriculture comes together. Feeding people is a way to achieve peace, I once read. That is my personal mission: to feed people all over the world. I have seen a lot of poverty in my life. In India, you give food to people who cannot afford it. Feeding people is a noble cause. I have stayed very connected to my roots all these years.”

Puneet Mishra Vision Robotics

Dr. Puneet Mishra

Researcher, Post Harvest Technology

Contact Dr. Puneet Mishra