Robots are learning to walk and work. While robot dogs are not yet man's best friend, real autonomy and reasoning will make them useful companions in industry, search and rescue and even space exploration. But you must walk before you can run and machines are learning lessons from biology for better walking robots. 

The first chords of the 1960s Motown song Do You Love Me, by the Contours, sound on the speakers as the robots start to dance. Several models, including a bipedal humanoid version, and a four-legged dog-like contraption, are seen dancing with each other. They shuffle, do pirouettes and swing. 

 

Released by the US robotics company Boston Dynamics, the viral video of robots with legs dancing created a stir at the end of 2020. Reactions ranged from people suggesting it was made using CGI, to fear that the robots were going to take over the world.

Yet for all the impressive engineering, the video also showed the limitations that legged robots face. Whereas for humans dancing is quite easy, for robots it's incredibly hard, and the three-minute video meant that every movement of the robots had to be manually scripted in detail.

"Today robots are still relatively stupid," says Marco Hutter, professor at ETH Zurich and expert in robotics. "A lot of the Boston Dynamics videos are hand-crafted movements for specific environments. They need human supervision. In terms of real autonomy and reasoning, we're still far away from humans, animals or what we expect from science fiction."

'In terms of real autonomy and reasoning, we're still far away from science-fiction.' Marco Hutter, LeMo

Yet these sorts of robots could be very helpful to humanity. They could help us when disasters strike, they could improve industrial operations and logistics and they could even help us explore outer space. But for that to happen we need to make legged robots better at basic tasks like walking and teach them how to do so without supervision. 

Virtual learning

The ERC-project LeMo is one of the investigations launched by European researchers to make robots move more autonomously. Their core premise is that legged locomotion isn't what it could be, and that machine learning techniques could improve it. LeMo is specifically focused on so-called reinforcement learning. 

"Reinforcement learning uses a simulation to generate massive data for training a neural network control policy," explains Hutter, who is also the project leader of LeMo. "The better the robot walks in the simulation, the higher reward it gets. If the robot falls over, or slips, it gets punished."

The robot they use in the project is a 50 kilogram, dog-like, four-legged robot. On top of it are several sensors and cameras that allow it to detect its environment.

This part has become pretty standard for legged robots, yet the advancement LeMo produces lies in the software. Instead of using a model-based approach, where the researchers program rules into the system, like "when there's a rock on the ground, lift up your feet higher", they 'train' an AI-system in a simulation.

Here the robot's system walks over and over through a virtual terrain simulation, and every time it performs well it receives a reward. Every time it fails it receives a punishment. By repeating this process millions of times, the robot learns how to walk through trial and error.

Robots with improved locomotive abilities can help in search and rescue operations and space exploration. © ETH Zurich

"LeMo is one of the first times reinforcement learning has been used on legged robots," says Hutter. "Because of this, the robot can now walk across challenging terrain, like slippery ground and inclined steps. We practically never fall any more."

Using this technology, the ETH Zurich team recently won a $2m Defense Advanced Research Projects Agency contest in which teams were challenged to deploy a fleet of robots to explore challenging underground areas by themselves.

"Legged robots are already used for industrial inspections and other observation tasks," says Hutter. "But there are also applications like search and rescue and even space exploration, where we need better locomotion. Using techniques like reinforcement learning we can accomplish this."

Natural inspiration

Another ERC-project, called M-Runners, is working on how to build legged robots that work in outer space. Today when we launch robots to places like the moon or Mars, they are generally wheeled robots. These need to land, and ride on, relatively flat pieces of terrain. 

"But the interesting things for geologists aren't generally located in the flatlands," says professor Alin Albu-Schäffer, of the TU Munich and the German Aerospace Center. "They are found in places like canyons, where rovers cannot easily go."

Which is why there's a strong interest in sending legged robots up into space. But before we can do that, more research needs to happen on making them work better. M-Runner here takes inspiration from nature. 

"Our hypothesis is that biology is more energy efficient," says Albu-Schäffer. "Our muscles and tendons have some elasticity. Animals, like a horse galloping, use this elasticity to store and release energy. Traditional robots on the other hand are rigid, and don't do that."

This means that legged robots are not as efficient as they could be. But really understanding these processes, and transferring them to robots, is quite a challenge. It requires a deep understanding of biology, but also of the mathematics behind how movements are made and repeated. 

'Our hypothesis is that biology is more energy efficient.' Alin Albu-Schäffer, M-Runners

The complex system of the limb, with a high amount of interdependent parts like muscles, tendons and bones, working together very closely to repeat movements like walking or running. "Modelling this mathematically is a scientifically unsolved question," says Albu-Schäffer.

Which is what the M-Runner project is trying to solve, and transfer to robots, a quest that's heavily interdisciplinary. "We work on biomechanics and biological systems," says Albu-Schäffer. "But also neuroscience, mathematics and physics. In turn we build tools that apply this to the actual robots."

So far the project has already built a prototype robot, a dog-sized variant, on which the researchers are testing different types of running and gaits. The eventual goal is to apply this theoretical research into a role such as space exploration. "We also think about low gravity in simulations," says Albu-Schäffer. "The robot here can do more spectacular jumps and stride farther."

Beyond this research, legged robots are already becoming integrated into our economy and society today. "These machines are already in use," says Hutter. "It's not a household item yet. But in industrial contexts it's getting more popular, and in China even household use cases are being investigated."

But their mass market appeal relies on these robots becoming better at walking and acting in the real world. Which is why more research is needed. "Legged robots aren't just about Boston Dynamics," says Albu-Schäffer. "In Europe cutting edge-research is also being done, and we're seeing real advances in the technology."