We translate it into understandable language: robots will be taught to speak humanly to each other

Russian specialists have created an operating system for robots that allows them to exchange information in natural language when performing common tasks. This is done so that a person can set tasks, monitor their actions and make adjustments to them. Right now, data exchange between machines is mostly digital. According to experts, the new feature can really make it easier for people to manage a team of robots. However, it is dangerous due to errors, slowing down the speed of robot interaction and will require additional energy, which is always limited in the battery.

Conversations between robots

Skoltech specialists have developed a new operating system for so-called cognitive robots. This type includes universal devices with artificial intelligence, capable of thinking and making decisions without human help. They don't have a strict program of action. They are able to work in a new environment and perform universal actions, they can receive any task and independently find a way to solve it.

The new software provides for the possibility of exchanging information between machines working in a team in a natural, that is, ordinary human language. This is done so that the user can control their actions and, if necessary, make adjustments to them. Right now, robots communicate data to each other mostly digitally via Wi-Fi.

Photo: IZVESTIA/Kristina Kormilitsyna

— With the help of our operating system, such devices begin to communicate in the same human language as we do. This is a universal system that allows machines and people to coexist together and communicate conveniently," Professor Dmitry Teteryukov, head of the Skoltech Laboratory of Intelligent Space Robotics, told Izvestia.

Orientation — server: robots have been taught to find their way according to the principle of human memory

How new technology will accelerate the adoption of autonomous cleaners and deliverymen

The function of natural language communication is provided in the CognitiveOS operating system. It is a modular software product that can be installed on almost any type of robotic device and combine several machines into one team or make them part of a swarm of robots. The program's component blocks can be combined depending on which device it is installed on and for what purposes.

For example, a robot dog acting as a guide and a robotic arm perform different functions. The former needs modules for space analysis, decision-making, navigation, and ethics, as it interacts with people. All the manipulator needs is a behavior block and a powerful video analytics system for recognizing the objects it works with. The new operating system will allow robots to perform tasks with complex scenarios together.

Photo: RIA Novosti/Evgeny Biyatov

— Suppose a user orders a drink. The robot dog takes the order, goes to the bar, where there is a manipulator, and passes the order to him. The robotic arm finds the right bottle, pours the contents into a glass, and passes it to the dog, who delivers it to the human. Both of these machines will have one of our systems, but its modules are different," Dmitry Teteryukov explained.

The use of the modular principle allows saving computing power and using only the most necessary functions of the robot, the scientists noted. This allows you to do all calculations directly on board the machine, rather than sending them to remote powerful computers. Due to this, the robot becomes more autonomous, works faster and is more reliable, they added.

The developers plan to expand the list of machines on which their software can be installed. For example, add humanoid robots, drones, AI agents, and digital doppelgangers with the ability to combine different programs and machines into large swarms. And also to create an operating system that meets the concept of "Industry 6.0". It provides for the creation of factories where robots of various types will be used, capable of acting without human intervention and controlled by artificial intelligence.

We also need gestures.

The ability of robots to communicate with each other in natural language can really make it easier for humans to control them. But for greater convenience, it is necessary that they understand not only words, but also each other's gestures, said Artemy Kotov, a leading researcher at the Laboratory of Cognitive Research on the Foundations of communication at Moscow State Linguistic University.

Photo: IZVESTIA/Anna Selina

— This is important for pack robots. But not all of them work in contact with people. It is more convenient for them to transmit information digitally, as it is faster and more reliable. But using natural language is useful for machines that are close to humans. In order to avoid a large number of errors, you can simply voice the data that they received digitally. If robots work with humans to solve some common task, such as building a house, then they also need to understand gestures and glances, as it happens with humans. This is called joint attention," he said.

Miscalculation: the inspection of microelectronics was assigned to a robot inspector with AI

How automatic complexes can help improve the quality of functions and the cost of final products

Communication of robots in natural language can significantly reduce the speed of their interaction compared to the transmission of digital data, says Eduard Klyshinsky, associate professor at the School of Linguistics at the Higher School of Economics.

— If this feature is implemented at the operating system level, it will slow down the operation somewhat. And if it is at the application software level, then this is a very convenient thing, but it will require a lot of energy. After all, the robot must be autonomous, dependence on the battery can undermine it. Such a gadget will simply discharge faster," the expert said.

Photo: RIA Novosti/Grigory Sysoev

Cognitive robots are intelligent systems capable of perceiving, analyzing, making decisions and adapting to the environment, with self—learning skills to work in difficult and unpredictable conditions, explained Roman Gorbachev, head of the Laboratory of Wave Processes and Control Systems at MIPT.

— The operating system proposed by Skoltech is a step forward towards creating flexible and reliable robotic autonomous systems that will help radically accelerate the creation of cognitive robots. The presented approaches pave the way for the rapid transfer of AI functions between different models," the expert told Izvestia.

From a scientific point of view, Skoltech's developments are of interest in the context of further research on multi-agent systems, human-machine interaction, and the use of AI models with a large context in physical object management tasks. With appropriate development, such approaches can serve as the basis for creating more autonomous and versatile AI systems, concluded Denis Kuznetsov, Deputy head of the Laboratory of Neural Systems and Deep Learning at the Institute of AI at MIPT.