Over the past decade or so, robotics and computer scientists have tried to use reinforcement learning (RL) approaches to train robots to navigate effectively in their environment and perform many basic tasks. However, creating accessible robots that can support and manage research controls related to RL algorithms has so far proved to be quite a challenge.
Researchers from Aalto University and Ote Robotics recently created RealAnt, a low-cost four-legged robot that can be effectively used to test and implement RL algorithms. The new robotics platform, presented in a paper previously published on arXiv, is a minimalistic and accessible real-world version of the Ant robot simulation environment, which is often used in RL research.
"The initial inspiration for our work was RL research, which successfully demonstrated learning to walk from scratch on simulators of ant - like quadrupeds and humanoid robots," said Tech Xplore's Jussi Sainio, co-founder of Ote Robotics. "The basic premise of RL algorithms is that programming a robot to perform tasks becomes much easier and more 'natural' - you just need to determine the available sensor measurements, motor actions, then set a target and connect them all to reinforcement learning. an algorithm that calculates everything else ."
Initially, RL algorithms would only work well if they were trained on robot simulations for thousands of hours. More recently, however, computer scientists have been able to teach ant-inspired four-legged robots to walk using very little training data, achieving what is known as high sampling efficiency. This made it possible to directly train robots in the real world, eliminating the need for simulation-based training.
"We quickly realized that walking robots like RealAnt were not easily and inexpensively available, especially for training with reinforcement, which can easily damage the robot through mismanagement," Sainio explained. "There wasn't a complete combined software and hardware stack that could be taken and started with reinforcement learning in the real world, compared to simulator environments. So I started creating my own robot prototypes and interface software."
The main goal of the recent work of Sainio and his colleagues was to create a simple and inexpensive robotics platform based on existing basic RL solutions. Such a platform will allow more researchers to create and test Autonomous robots that can perform many basic tasks in the real world.
RealAnt, the four-legged robot they created, is versatile, minimalistic and inexpensive. Moreover, it can autonomously learn to walk by moving its legs in a coordinated manner, and can sense its position and orientation in a given environment. Using RL algorithms, Real Ant can be trained to perform many simple but valuable tasks.
"You can think of the RealAnt platform as a real version of the' Ant 'simulator environment, which is a popular benchmark for RL," Sainio said. "This is one of the easiest platforms to get started with reinforcement learning and real robots. The main advantage of the RealAnt platform is that it is easily and inexpensively available."
Building the RealAnt will cost about $ 410 in materials, and its individual components are easy to get hold of. Moreover, the robot can be assembled in less than an hour after its individual parts are prepared. Both its hardware and software are open source, and a fully assembled robot can also be easily purchased online on the Ote Robotic website.
The low cost of manufacturing and easy Assembly make RealAnt available to a huge number of people around the world. In addition, they are easier to deploy in large numbers than the more expensive and complex robots on the market today.
"The RealAnt platform includes the necessary robot hardware (motors, sensors) and software stack (communication, tracking) to interact with the robot, and our published basic reinforcement learning solution serves as an example of how it can be taught to walk from scratch," Sainio said. "The sample solution is simple and contains a small guide for each learning task - in terms of machine learning, we don't do much manual reward construction to shape learning effectiveness - which makes defining new tasks simple and straightforward."
One of the reasons why the RealAnt robot is more affordable than other existing quadrupeds that support RL is that its body moves using 8 inexpensive intelligent servomotors, rather than more expensive and complex motors. In addition, to track its position and orientation, the robot uses augmented reality tags that can be easily printed on paper, and an inexpensive external webcam.
"All parts of the robot's body are printed in three - dimensional space, and they are small enough to be printed on most consumer three-dimensional printers," Sainio said. "This makes it cheaper to manufacture and modify the platform than robots with components that are made using laser cutting or mechanical processing of metal or plastic sheets. Since the RealAnt design uses low-cost engines, we manage them carefully, limiting their maximum torque, and so they can withstand constant rough movements during random research and training."
Sainio and his colleagues evaluated RealAnt in both simulation and real-world experiments. The robot showed excellent results in all these tests, which showed great prospects for a wide range of applications.
So far, most machine learning and machine learning methods for robotics applications have mostly been trained in simulated environments. The researchers hope that RealAnt will open up exciting new opportunities in this area, as the robot can be trained and tested both in simulations and in the physical world.
"RealAnt can serve as a real - world robotic environment and a reference for RL, helping to match simulated environments to reality," Sainio explained. "Creating a real physical robot on its feet that can learn to walk or perform other tasks from scratch, without a simulator, is still a relatively new and rare feat. Real robotics is difficult to handle, so I think it's a good idea to create a completely new, minimal robot platform capable of RL."
The robotics platform created by Sainio and his colleagues may soon help other teams test their RL and ML algorithms on a real robot. The researchers hope that RealAnt will contribute to the development of a wide range of applications, such as in agricultural settings, where autonomously trained robots can be used to uproot weeds and harvest crops, which helps preserve biodiversity and possibly even reduce the use of pesticides.
"We now intend to Refine and expand the RealAnt platform to expand the hardware capabilities, such as providing the robot with more advanced detection capabilities and possibly manipulators, as well as running multiple robots simultaneously, building on the core platform that is now available on the Internet," Sainio said. "We are also exploring ways to make RealAnt walk or perform other more complex tasks even faster, which further reduces training time."