Apple Picking Robot: Learning from Demonstration
29 July 2024
Could farmers instead of programmers be training robots to perform certain tasks? Robert van den Ven investigated the use of Learning from Demonstration to teach a robot all the motions needed to harvest an apple.
There are already several apple-picking robots, I know the examples. But those, too, are prototypes that still require some tweaking before they can actually be brought to market. Hence my research.
One of the major challenges of such a robot is making the picking motion. Most prototypes pick an apple from the tree by rotating it around the middle of the core. This can damage an apple, which you don’t want. To prevent this, a picking robot should make a pull-and-twist motion that rotates around the stem, so it needs to grasp the apple there. However, programming this is technically more challenging, according to research. Manual programming takes time, but this isn’t necessary with Learning from Demonstration (LfD). It had not really been investigated before whether and how you could teach a robot arm these picking movements using LfD. We did, and yes, it can learn that rotation.
For this, we divided the entire picking process into four steps: approaching, grasping, picking, and placing. If I pick an apple with my hand and film it with a camera, no, a robot cannot learn from that. That image, that motion, is too complex for it. Therefore, you need to simplify that reality. Using Learning from Demonstration, we taught a UR3 robot arm the four steps. Initially using a cube instead of an apple in a lab setting. I used a glove with trackers that can translate a movement into movements for a robot in real-time. If I move my hand five centimeters to the left, the robot arm mimics me. The trackers on the glove ensure that all that data is sent to a computer, which then sends it to the robot.
The possibilities of LfD are vast. If it works, a farmer could eventually teach a robot something (extra) on the spot in the same way he teaches people. That would be revolutionary. To ensure the robot performs a task precisely as desired and can also take on new tasks like picking pears or pruning plants. To make the robot more multifunctional. The robot can store the new or adjusted task in its database and thus refine its internal model. Who knows, this might even be voice-controlled in the future.
But first, I’m going to take the robot to the orchard to test whether the examples from the lab also work in the orchard.
Read the full paper here.
