Vzglyad programs: AI will help the blind navigate the city without the Internet

Russian engineers have created a device for the orientation of the blind, which recognizes surrounding objects and suggests the direction of movement without the need for an Internet connection. Unlike analogues that need to communicate with a cloud server and complex, expensive systems, AI algorithms can develop an economical, compact computer. According to experts of the All-Russian Society of the Blind, the invention looks promising, but for an objective assessment it is necessary to test it in practice.

An AI algorithm without Internet access

St. Petersburg State University of Aerospace Instrumentation has developed a unique device capable of fully autonomously recognizing surrounding objects and helping blind users navigate in space. Its key feature is that the system does not require an internet connection or cloud services.: Artificial intelligence algorithms work directly on a compact embedded computer, which has enough computing power to process data in real time.

— Existing analogues on the market either belong to the premium price segment, or are rigidly linked to cloud servers. Sending a video stream to the cloud creates dangerous signal delays, which is unacceptable when a blind person is moving along a busy street. The scientific novelty of the project lies in the deep optimization of heavy AI models under the strict hardware limitations of a mobile microcomputer. We managed to solve a complex engineering problem — to launch parallel object detection based on the YOLO architecture and speech recognition completely offline," said Valentina Sinitsina, the author of the project, a student at GUAP.

All existing ways of orientation for the blind are imperfect, the developers said. A white cane provides safety only at a distance of one step, and the usual GPS navigators on smartphones are not able to notice dynamic obstacles, such as electric scooters left on the sidewalk, scaffolding or low-hanging road signs. In addition, mobile applications become useless in areas with unstable cellular coverage. The developed complex acts as a smart personal assistant that scans the environment in real time and warns of obstacles.

They can do anything: how do people with disabilities get a job

The human resource potential of disabled people in Russia is underestimated

The hardware basis of the device is a compact microcomputer with a portable camera, which is mounted on the user. The software part is divided into two independent computing circuits. The first is macro navigation: a local geoinformation module with support for offline maps. It recognizes the user's voice commands, finds the desired address, and plots the optimal walking route. The second is micronavigation: a computer vision system based on a lightweight neural network. It continuously analyzes the video stream from the camera, instantly recognizes infrastructure objects (poles, bins, benches, parked cars) and calculates the exact distance to them.

Photo: IZVESTIA/Polina Violet

The device processes the video stream in real time at a high frame rate. Its architecture implements a priority displacement system that provides instant response to potentially dangerous situations. For example, if an obstacle or other threat suddenly appears in the user's path, the micro navigation module automatically interrupts the operation of the standard voice navigator and immediately transmits an emergency warning.

Implementation of the development

The prototype of the device has already successfully passed a series of load tests. The tests confirmed the stability of the microcomputer's temperature regime and the complete absence of delays in real-time data processing.

— Any new development of this kind will not be superfluous. It is most convenient when it is embedded in the phone, then you do not need to carry an extra item. However, now we, like everyone else, face difficulties accessing the Internet and confusing navigation, so the device's ability to work without it is an advantage. In any case, we need to test the invention," said Lyubov Boitsova, a leading specialist at the Moscow city organization of the All—Russian Society for the Blind.

Life by Touch: how blind people find their way in the world

And who helps them adapt to the new realities?

According to Maria Vostrikova, head of the Department of Social Development of the Department of Social Rehabilitation of the Office of the All-Russian Society of the Blind, the development looks promising: it is based on the real needs of blind people, uses a modern approach that allows it to function completely offline. There is also already a working prototype of the device and load tests have been carried out. If it is possible to bring the invention to an industrial version and keep an affordable price, it has a good chance to occupy its niche in the market of such equipment.

Photo: IZVESTIA/Polina Violet

— Similar solutions already exist on the global market, but many of them are either focused only on individual use cases, or to some extent depend on external infrastructure and belong to a fairly expensive segment. If the developers of GUAP really managed to ensure stable offline operation of computer vision, speech recognition and navigation on a compact computing platform in real time, then this represents a serious engineering achievement," says Yaroslav Seliverstov, a leading expert in the field of AI at University 2035.

The chosen direction for solving orientation problems seems promising. However, such devices often have high power consumption with continuous local video processing, poor stability of the monocam when shaking while walking, and complete blindness to obstacles under the user's feet, said Maxim Kolyasnikov, a leading researcher at the Faculty of Biomedical Engineering at Bauman Moscow State Technical University, associate professor at the UrFU Institute of Economics and Management.

It also seems to be a dangerous scenario in which the user navigates the street by a continuous voice stream, which will inevitably block out the natural noises of the city and create cognitive overload. Without the use of specialized headphones and the integration of distance sensors like ultrasound, this device is unlikely to be applicable in practice, the expert believes.