Mobile apps for deaf people. Self-voicing: programs for reading text by voice For the deaf, a tablet or notepad for communication

The main problem with such applications is that this niche is underdeveloped. While searching, I looked at more than 30 applications, and almost all of them were designed for an English-speaking audience or people living in Europe or the USA. However, the six applications below are an exception and, I hope, will be useful to someone.

Miracle Mode

Miracle Modus is an app created by an autistic developer. Judging by his words and comments in Play market the app really works. The principle of operation is simple and complex at the same time: Miracle Modus draws various lines and patterns and plays relaxing sounds that have a calming effect on people with autism.

Diabetes Log Book

An application that helps people with type 1 and type 2 diabetes. Here you can enter all the information about the food that you ate during the day. The full report can then be exported to Excel to show to your doctor.

migraine buddy

Allows migraine sufferers to analyze their lifestyle and understand what factors lead to headaches. The app also tracks your sleep so you can see the correlation between migraines and sleep quality.

Bumps

Bumps helps blind people learn Braille. The app only supports English language, so it is suitable only for those who want to learn how to read not only in Russian. In addition to learning the alphabet, Bumps lets you enter different words and see how they are written in Braille.

There is a similar app for Android called Braile Guide.

Spread Signs

Spread Signs has a huge library of gestures to help deaf and dumb people communicate. The application base includes more than a dozen languages, including Russian, Ukrainian and English. You can learn both gestures denoting individual letters, and whole words.

Two students from Beijing's Beihai University have developed an app to help deaf people in China. The application was called Shouyin, which literally means "voice of hands." It allows real-time translation of signs of the language of the deaf and dumb into speech.

The application works in conjunction with a smart bracelet and on this moment can recognize more than 200 characters, each character has been recorded a thousand times by different people to improve the accuracy of the translation. Now the recognition rate is 95%.

Young Nana Wang, who is one of the developers, said she was inspired by a deaf and mute friend, with whom she could only communicate through messages on mobile phone. In China, approximately 5% of the population or 70 million people are deaf. Undoubtedly, the application will be a great success. The official release of the application is scheduled for the end of May. Similar programs already exist in other parts of the world. For example, Uni from MotionSavvy and the deaf-and-dumb translator from Google Chats.

Technology today is developing at a rapid pace, including in social spheres and in medicine. They help not only to cope with various kinds of ailments, but also to better understand your neighbor, even if his perception of the world is different from yours. Talking to a deaf person will help you special application, while VR technology can help, for example, with understanding autism, which means bringing you closer to those who suffer from this disease.

Various studies have shown that over 99% of people have heard of autism, but only 16% really understand what this mental condition means. And according to a survey conducted in the UK, more than 25% of people with autism have ever asked to leave a public place such as a restaurant or shop when they saw symptoms of their illness.

To deal with this problem of misinformation, the British "National Autistic Society" decided to create VR movie, which would help all people to know what it is like to live with such a diagnosis. Gmbox wrote about it.

This virtual video takes you into the body of a young boy with autism and shows the world through his eyes. In the movie, you walk into a mall and experience the enormous sensory load that it feels when it walks through this crowded place. The noise, the abundance of various objects, colors, the flickering of people, their voices - all this causes the boy to panic, which is very difficult for him to cope with.

Another Alzheimer's community has also released a VR movie that puts you in the shoes of someone with dementia. The film A Walk Through Dementia will let you understand how some simple daily activities, like making yourself a cup of tea, can become a real problem for the owner of this disease.

Idea

Hello! My name is Vitaly, I am a developer and co-founder of a startup BeWarned. Since childhood, I have very poor hearing, and my wife does not hear at all, for a long time I wanted to get into application development and software for the deaf and hard of hearing. All the problems they face are familiar to me.

Me and my wife Natalia

Previously, I was developing software for civil airlines of Ukraine, I created the SIMO augmented reality application. I have always been interested in everything related to technology. And a couple of years ago, I met Sergey Malyukov, now our CEO, through a teacher at the Kyiv Polytechnic Institute, with whom I once studied.

The idea of ​​creating an application for deaf and hard of hearing people came to Sergey in 2013. One evening he was walking along the street in the evening, and a car began to honk in his back, Sergey quickly jumped aside, but the couple walking nearby did not budge. He called after them to move away, but they remained in the same place where they had stood. Soon Sergei realized that it was a pair of deaf people, they communicated with each other in sign language. Sergey was sure that there are applications that help to recognize dangerous sounds. After analyzing existing applications and gadgets for deaf people and not finding a single solution that helps to recognize dangerous sounds, he set out to make such an application. In the process, several partners joined him and now BeWarned is self-invested by the creators of the startup.

As soon as I heard about the project and saw the developments, I was instantly carried away by the idea and soon became a co-founder. At the moment I am engaged in the technical development of a startup.

Implementation

In 2014, the first prototype of the BeWarned app was released with the Sound Monitor feature, which allowed real-time recognition of dangerous sounds. It was the first small success. But that was only the beginning.

Since childhood, I liked technology, I often assembled or disassembled something, I wanted to see what would come of it.

From hundreds of letters and reviews, it was clear that deaf people needed something more than an app that recognized dangerous sounds. The deaf had a thousand unresolved issues that could be resolved. Based on these letters and feedback, the BeWarned platform was created, which brought together 4 technical assistants for the deaf: Sound Monitor, Connect, Emergency Call and Dance.

Sound Monitor solves the problem of security by identifying dangerous sounds (scream, car horn, dog barking, siren). In case of danger, the application warns the user by vibrating and flashing the flash on the smartphone. Connect helps the deaf to communicate with others - it converts text to speech and vice versa. Dance allows you to feel music tracks by converting them into vibration, light signals and ripple visual EQ. Emergency Call helps the user call hearing loved ones for help when they are in a dangerous situation.

You can find more videos on our channel

In the summer of 2016, we beta tested the platform in the US. As a result, 88.9% of users reported that they would recommend the app to friends.

The most difficult part of developing BeWarned was the Sound Monitor, which we spent several years developing. The dangerous sounds recognition program is based on neural networks. We tried two types of neural networks: classical multilayer perceptron and convolutional neural network. But then we decided to try the Hidden Markov Model, it was the best fit for our tasks.

In order to create a database on which the algorithm will be based, we ourselves recorded many variants of one type of dangerous sound, for example, a car signal, on the basis of which the algorithm will recognize incoming sounds and determine them as belonging to a certain type. However, in the course of work, we had a problem that the recognition algorithm attributed all incoming sounds only to existing categories: a car signal, a scream, the sound of a police siren, and a barking dog. For example, if someone laughed, then the program recognized it as belonging to one of these categories. To solve this problem, we had to segment the sounds, for example, by frequencies and other characteristics. As a result, it allowed to introduce an additional indefinite type of sound.

Some time after the release of the BeWarned platform, we decided to make important changes in order to improve usability: we radically simplified everything and adapted it to the perception of deaf people. That's why we re-released the apps individually.

In December, BW Dance (Android, iOS) was released, which turned from a function of the BeWarned platform into an independent application. BW Dance allows deaf and hard of hearing people to dance to their favorite songs. The user adds favorite songs to the playlist and the app transforms the music into vibrations, flashes and animations on the screen. The user can also feel the music through the visual player. BW Dance was positively received by users both in Ukraine and abroad.

Since then, we've released Connect by Bewarned (Android, iOS), an app that allows deaf and hard of hearing people to communicate with hearing people. In order to start chatting with one or more people in the application, you just need to press the START button and let the interlocutor speak, after which the application will convert speech into a text message that a deaf person can read. The application also converts text to speech, a deaf person needs to write what he wants to say. In addition, we have developed a set of templates that simulate simple, often recurring situations, such as ordering coffee in a coffee shop. With these patterns, a deaf person can quickly communicate something to a hearing person. We also provided the ability to create additional templates for the personal purposes of each deaf person.

results

In Ukraine, journalists began to actively write to us, stories were filmed about us by STB, 1 + 1, Channel 5 and others. We saw that people care about the non-hearing, and also made sure that the concern for simplicity and usability was a step in the right direction. We realized that we are on the right track.

My wife and I communicate with journalists of Channel 5

We launched Connect by Bewarned on Product Hunt on January 30th and entered the TOP 5 of the day, as well as getting into the official mailing list. In the comments on our product on Product Hunt, we received offers to help develop localizations in French and even Farsi. Together with the already existing four localizations - Spanish, Russian, Ukrainian, and English, we will be able to get closer to our global goal: to make life easier for deaf people.

I am convinced that IT people should use the skills given to them for the benefit of other people. Our small but successful experience shows that new applications can significantly change the lives of other people. That is why we are going to continue to work on the idea embodied in Connect by Bewarned: overcoming barriers in communication between the deaf and the hearing, there are about 400 million of them in the world.

The BeWarned team from left to right: Me, Evgeny, Evgeniya, Ivan, Yulia, Sergey, Andrey, Ilona, ​​Ira, Anton, Andrey (now there are even more of us than in this photo :))

People like me often have problems when they need to communicate with a hearing person: a policeman, a doctor, a banker. Legislatively, we are guaranteed equal rights and access to services and services on an equal basis with those who hear. But in reality, our possibilities are significantly limited. That is why we decided to take on a truly global task: the creation of a global electronic infrastructure, which will facilitate the access of deaf people to all services that are available to hearing people: banks, hospitals, taxis, etc. We are currently working on this issue.

Such simple steps things like talking on the phone, walking down stairs, or even determining the expiration date of milk can become challenging for people with disabilities. However, some of these problems can be successfully dealt with using a smartphone.

Introducing the selection mobile applications that make life easier for people with disabilities.

Applications that recognize speech

Yandex.Conversation

Yandex recently released an app that helps deaf and hard of hearing people communicate with friends and strangers by translating spoken language into text and vice versa.

The application is able to listen and recognize speech - everything that is said to you is displayed on the smartphone screen as text, as well as pronounce printed words aloud.

The app was designed and developed by a group of MIPT students and is powered by Yandex SpeechKit speech recognition technology.

RogerVoice

RogerVoice is an application with which deaf people can talk to their interlocutors on the phone. The app uses speech recognition technology to convert voice to text so deaf people can read what the other person is saying.

The app came about thanks to successful Kickstarter campaign – the app is currently in beta and can be downloaded from the RogerVoice website.

An app that helps you hear

HearYouNow

With this application, hearing impaired users can adjust the sound according to the situation they are in - in a public place, in a meeting, etc. To do this, headphones are connected to the smartphone, through which sound is supplied with the appropriate settings to optimize the background or foreground.

The app is developed by ExSilent, a Dutch hearing aid manufacturer, for people who can use hearing aids but are not yet ready for it.

Hello!

"Bread nourishes the body, but the book nourishes the mind"...

Books are one of the most valuable wealth of modern man. Books appeared in ancient times and were very expensive (one book could be exchanged for a herd of cows!). In today's world, books are available to everyone! By reading them, we become more literate, our horizons and ingenuity develop. And in general, they have not yet come up with a more perfect source of knowledge for transferring to each other!

With development computer technology(especially in the last 10 years) - it has become possible not only to read books, but also to listen to them (that is, you will read them special program, male or female voice). I would like to tell you about software tools for voice acting.

Possible Recording Problems

Before proceeding to the list of programs, I would like to dwell on a common problem and consider cases when a program cannot read text.

The fact is that there are voice engines, they can be of different standards: SAPI 4, SAPI 5 or Microsoft Speech Platform (most text playback programs have a choice of this tool). So, it is logical that in addition to the program for reading by voice, you need an engine (it will depend on what language you will be read in, what voice: male or female, etc.).

speech engines

Engines can be free and commercial (naturally, best quality sound reproduction is provided by commercial engines).

SAPI 4. Obsolete versions of tools. Not recommended for modern PCs. outdated versions. It is better to look at SAPI 5 or Microsoft Speech Platform.

SAPI 5. Modern speech engines, both free and paid. You can find dozens of SAPI 5 speech engines on the Internet (with both female and male voices).

One of the most the best programs to read text. Allows your PC to read more than just simple files in txt format, but also news, RSS, any web pages on the Internet, email etc.

In addition, it allows you to convert text into an mp3 file (which you can then download to any phone or mp3 player and listen on the road, for example). Those. you can create audio books by yourself!

The voices of the IVONA program are very similar to the real ones, the pronunciation is not bad enough, they do not stutter. By the way, the program can be useful for those who study foreign language. Thanks to it, you can listen to the correct pronunciation of certain words, turns.

Supports SAPI5, plus it cooperates well with external applications (for example, Apple iTunes, Skype).

Example (recording of one of my recent articles)

Of the minuses: he reads some unfamiliar words with the wrong stress and intonation. In general, it’s not bad enough to listen, for example, to a paragraph from a history book while you go to a lecture / class - even more than that!

Balabolka

An excellent program for working with books: reading, cataloging, searching for the right one, etc. In addition to standard documents that other programs can read (TXT-HTML, HTML-TXT, TXT-DOC, DOC-TXT, PDB-TXT, LIT-TXT , FB2-TXT, etc.) ICE Book Reader supports .LIT, .CHM, and .ePub file formats.

In addition, ICE Book Reader allows you not only to read, but also an excellent desktop library:

• allows you to store, process, catalog books (up to 250,000 copies!);
• automatic ordering of your collection;
• a quick search for a book from your "dump" (especially true if you have a lot of non-catalogued literature);
• ICE Book Reader's database engine is superior to most programs of its kind.

The program also allows you to voice texts.

To do this, go to the program settings and configure two tabs: “Mode” (select reading by voice) and “Speech synthesis mode” (select the speech engine itself directly).

Talker

With Sakrament Talker you can turn your computer into a "talking" audio book! Sakrament Talker supports RTF formats and TXT, it can automatically recognize the file encoding (probably, sometimes you noticed that some programs open a file with “cracks” instead of text, but this is impossible in Sakrament Talker!).

In addition, Sakrament Talker allows you to play large enough files, quickly find certain files. The voiced text can not only be listened to on a computer, but also saved to an mp3 file (which can later be copied to any player or phone and listened away from the PC).

In general, a rather good program that supports all popular voice engines.

That's all for today. Despite the fact that today's programs still cannot fully (100% qualitatively) read the text so that a person cannot determine who is reading it: a program or a person ... But I think that someday programs will reach this: the power of computers grow, engines grow in volume (including more and more even the most complex turns of speech) - which means that soon enough the sound from the program will be indistinguishable from ordinary human speech?!