Scientists have created a benchmark for evaluating the quality of chemical language AI models.

Russian scientists have developed a tool to evaluate the effectiveness of large language models in solving chemical problems. The new benchmark will accelerate the creation of artificial intelligence (AI) capable of generating new medicines. This was reported to TASS on November 1 by the press service of the AI Institute of Artificial Intelligence.

"Our test is designed to help language model developers better understand the architectural limitations of neural networks. It can become the basis for further research in the field of automation of chemical reasoning," the press service quotes Kuzma Khrabrov, a researcher at the Center for AI Drug Development (AIDD) at the AIRI Institute.

It is noted that in recent years, AI and machine learning have begun to be actively introduced into chemistry and related fields. One of the most famous examples is the AlphaFold protein structure determination algorithm, which won the Nobel Prize. Currently, researchers are developing new methods for solving chemical problems using AI language models.

However, solving chemical problems often requires not just extracting facts, but consistently addressing related issues. It is clarified that AI sometimes does not cope with this adequately. To work effectively in this area, it is necessary to create approaches that quickly find ways to improve AI's ability to solve complex problems.

To improve the work of chemical AI, scientists have developed a specialized test of basic chemical tasks, including determining reaction products, predicting their properties, and generating molecular descriptions using a formula. Artificial intelligence, in turn, must solve the problem, identify the reaction product and evaluate its important characteristics, such as bioactivity.

Based on the test, the researchers tested several large language models, including universal AI systems and specialized chemical models. These checks confirmed the possibility of using the "chemical AI" assessment method and identified problematic aspects in the operation of the models. It is noted that as a result, this should accelerate the development of artificial intelligence in chemical science.

How neural networks help to detect diseases and create medicines. Analysis

Candidate of Medical Sciences, clinical pharmacologist Andrey Kondrakhin, in an interview with Izvestia on May 24, spoke about reducing medical errors and speeding up the routine part of the work of specialists using AI in medicine and pharmacology. According to him, artificial intelligence is also used in radiology, histology, cancer research and other industries. The specialist also noted that the introduction of technology facilitates prescribing, filling out medical records and drawing up medical referrals.

All important news is on the Izvestia channel in the MAX messenger.