One of the hottest topics in robotics is the field of soft robots, which utilises squishy and flexible materials rather than traditional rigid materials. But soft robots have been limited due to their lack of good sensing. A good robotic gripper needs to feel what it is touching (tactile sensing), and it needs to sense the positions of its fingers (proprioception). Such sensing has been missing from most soft robots.

In a new pair of papers, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) came up with new tools to let robots better perceive what they’re interacting with: the ability to see and classify items, and a softer, delicate touch. 

Have sensorised skins

“We wish to enable seeing the world by feeling the world. Soft robot hands have sensorised skins that allow them to pick up a range of objects, from delicate, such as potato chips, to heavy, such as milk bottles,” says CSAIL director Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science and the deputy dean of research for the MIT Stephen A Schwarzman College of Computing. 

One paper builds off last year’s research from MIT and Harvard University, where a team developed a soft and strong robotic gripper in the form of a cone-shaped origami structure. It collapses in on objects much like a Venus' flytrap, to pick up items that are as much as 100 times its weight. 

Tactile sensors

To get that new-found versatility and adaptability even closer to that of a human hand, a new team came up with a sensible addition: tactile sensors, made from latex 'bladders' (balloons) connected to pressure transducers.

The new sensors let the gripper not only pick up objects as delicate as potato crisps, but it also classifies them – letting the robot better understand what it’s picking up, while also exhibiting that light touch. 

When classifying objects, the sensors correctly identified 10 objects with more than 90% accuracy, even when an object slipped out of grip.

“Unlike many other soft tactile sensors, ours can be rapidly fabricated, retrofitted into grippers, and show sensitivity and reliability,” says MIT postdoc Josie Hughes, the lead author on a new paper about the sensors.

“We hope they provide a new method of soft sensing that can be applied to a wide range of different applications in manufacturing settings, like packing and lifting.” 

Embedded cameras

In a second paper, a group of researchers created a soft robotic finger called 'GelFlex' that uses embedded cameras and deep learning to enable high-resolution tactile sensing and 'proprioception' (awareness of positions and movements of the body). 

The gripper, which looks much like a two-finger cup gripper you might see at a soft drinks machine, uses a tendon-driven mechanism to actuate the fingers. When tested on metal objects of various shapes, the system had more than 96% recognition accuracy. 

“Our soft finger can provide high accuracy on proprioception and accurately predict grasped objects, and also withstand considerable impact without harming the interacted environment and itself,” says Yu She, lead author on a new paper on GelFlex.

“By constraining soft fingers with a flexible exoskeleton, and performing high-resolution sensing with embedded cameras, we open up a large range of capabilities for soft manipulators.” 

Magic ball senses 

The magic ball gripper is made from a soft origami structure, encased by a soft balloon. When a vacuum is applied to the balloon, the origami structure closes around the object, and the gripper deforms to its structure. 

While this motion lets the gripper grasp a much wider range of objects than ever before, such as soup cans, hammers, wine glasses, drones, and even a single broccoli floret, the greater intricacies of delicacy and understanding were still out of reach – until they added the sensors.  

When the sensors experience force or strain, the internal pressure changes, and the team can measure this change in pressure to identify when it will feel that again. 

In addition to the latex sensor, the team also developed an algorithm which uses feedback to let the gripper possess a human-like duality of being both strong and precise – and 80% of the tested objects were successfully grasped without damage. 

The team tested the gripper-sensors on a variety of household items, ranging from heavy bottles to small, delicate objects, including cans, apples, a toothbrush, a water bottle, and a bag of cookies. 

Going forward, the team hopes to make the methodology scalable, using computational design and reconstruction methods to improve the resolution and coverage using this new sensor technology. Eventually, they imagine using the new sensors to create a fluidic sensing skin that shows scalability and sensitivity. 

Hughes co-wrote the new paper with Rus, which they will present virtually at the 2020 International Conference on Robotics and Automation. 

GelFlex

In the second paper, a CSAIL team looked at giving a soft robotic gripper more nuanced, human-like senses. Soft fingers allow a wide range of deformations, but to be used in a controlled way there must be rich tactile and proprioceptive sensing. The team used embedded cameras with wide-angle 'fisheye' lenses that capture the finger’s deformations in great detail.

To create GelFlex, the team used silicone material to fabricate the soft and transparent finger, and put one camera near the fingertip and the other in the middle of the finger.

Then, they painted reflective ink on the front and side surface of the finger, and added LED lights on the back. This allows the internal fish-eye camera to observe the status of the front and side surface of the finger. 

Trained neural networks

The team trained neural networks to extract key information from the internal cameras for feedback. One neural net was trained to predict the bending angle of GelFlex, and the other was trained to estimate the shape and size of the objects being grabbed. The gripper could then pick up a variety of items such as a Rubik’s cube, a DVD case, or a block of aluminum. 

During testing, the average positional error while gripping was less than 0.77 of a millimetre, which is better than that of a human finger. In a second set of tests, the gripper was challenged with grasping and recognising cylinders and boxes of various sizes. Out of 80 trials, only three were classified incorrectly. 

In the future, the team hopes to improve the proprioception and tactile sensing algorithms, and utilise vision-based sensors to estimate more complex finger configurations, such as twisting or lateral bending, which are challenging for common sensors, but should be attainable with embedded cameras.