Bananas, cups and peelers: Robots learn how to handle curved objects like fruits and tools

 

It does not take much to confuse some robots. A machine might be great at handling a simple object like a box, yet when it tries to work with a more irregular shape like a banana, it often fails.

But help is at hand. Researchers from the Swiss Federal Technology Institute of Lausanne (EPFL) and Idiap Research Institute have developed a new approach that lets robots more reliably manipulate a variety of different shapes by teaching them to follow the unique geometry of any object they encounter.

Their work is detailed in a paper published in the journal Science Robotics.

Everyday tasks such as peeling a potato, slicing a cucumber, or washing dishes are simple and easy for most of us humans, requiring little thought. Yet for robots, they present a series of challenges.

When handling regular shapes, robots can follow preset paths and fixed directions. But objects such as cups, fruits, and tools come in all shapes and sizes, differing in their geometry, proportions, and surface curves. Although robots can be programmed for different shapes or sets of shapes, generalizing across many irregular objects remains difficult.

Mapping the geometry of everyday objects

The research team has developed a geometrically aware system that enables robots to handle some tasks that come naturally to us. They designed a way for robots to build a map of directions across irregular objects, so the machine can constantly determine how to move at each point on a surface.

The system uses a stereo camera to capture a 3D view of an object. It then processes these images to generate a cloud of coordinates that acts as a guide for a robot arm. Then, through a process called transfer, the robot can apply the same skill it learned on one object to a completely different shape without having to be retaught. "Our representation enables task-transfer across shapes addressing the immense shape variation of everyday objects," wrote the study authors in their paper.

Results and future development

In tests, the research team's robot successfully performed tasks like peeling, slicing, and cleaning across a wide range of objects it had never seen before, even when the camera data was incomplete. Because the system uses a mathematical process to smooth out the information it receives, the robot is less affected by small errors or missing parts of the 3D data.

"By conditioning diffusion processes on the target object's point cloud and keypoints, we developed a computationally efficient and robust method for computing these local frames online, even in the presence of partial and noisy sensor data."

While the results are good, the team admits there is room for improvement. For example, the system currently uses a few key points on an object that can be manually labeled before the robot can begin its task. The researchers hope to automate this step in the future. Additionally, they want to test their technology on more complex, soft objects that change shape when touched, such as sponges.