AI enters the era of accessibility: What does the application of sign language recognition and translation mean? 【full text】

Security Exhibition Network Technology News believes that in the impression of people, AI is a technology category that belongs to the "most". The so-called majority means that the amount of related data is large and easy to accumulate, which is suitable for the characteristics that AI is extremely dependent on massive data; the second means that the application scenario is wide, it is easy to realize the cost recovery, and it is suitable for the high threshold investment of AI research and development.

The face recognition, voice interaction, etc. that often appear around us at present are all in line with these characteristics. But this does not mean that the AI ​​belonging to the "minority" is blank. Now, we might as well turn our attention to technological developments that can eliminate the difference between "minority and majority", such as sign language recognition for the hearing impaired, which is a typical example.

Sign language that you and I do n’t understand, why is AI difficult to read?

Sign language is the use of gestures to measure movements, according to the changes in gestures to simulate images or syllables to form certain meanings or words, is a unique communication method for the hearing impaired. However, although this type of communication can allow hearing-impaired people to communicate with each other or ordinary people who understand sign language to communicate with them, it still cannot meet the communication needs between hearing-impaired people and the general public.

This means that in some social public spaces, such as government affairs scenes or service trade scenes, people with hearing impairment may encounter some inconveniences. And AI just happened to be a solution.

In some software, we have already begun to apply AI's gesture recognition capabilities, such as "taking care" when taking pictures to trigger some AR. Corresponding to the capture of gestures and the semantics of gestures, can't the translation and generation of sign language be realized?

This logic is correct, but from logic correct to application feasible, but there is still a short distance.

First of all, the expression of sign language has certain characteristics, it is not easy to capture. We know that there is no degree of "sign gesture", and the expression of some sign language vocabulary is very close, and the sign language expression is usually in units of sentences, and there will be no obvious gap between words. In the past, the way of using the front camera to capture gesture recognition is basically not feasible.

So the solutions given by many technologies and teams are to add peripherals. For example, the University of Science and Technology of China and Microsoft have launched a sign language translation system based on Kinect. However, these peripherals are either low in portability or expensive, making it difficult to promote them.

At the same time, sign language expression also has national and regional characteristics, and there are difficulties in the universality of the model. There are two concepts of "grammatical sign language" and "natural sign language" in sign language. Grammatical sign language is a common Mandarin. As for natural sign language, it is like a dialect. This has also caused the sign language data collection and labeling to be a high-cost, heavy workload.

For example, Amazon once proposed that transforming the smart speaker Alex can make it translate some simple signals. However, because of the lack of large-scale training data sets, this function can only recognize some simple American sign language and stay in the laboratory stage.

There is no secret way to solve problems in sign language, the AI ​​industry should have equal rights

Although exploration is difficult, technology companies and academia continue to achieve results in sign language AI. For example, the "Youtu AI Sign Language Translator" launched by Tencent Youtu Lab, the AI ​​sign language anchor launched by iQiyi, etc., have made great progress in the application of sign language AI.

The breakthrough of sign language AI can be divided into two routes, on the one hand is the progress of the technology of sign language AI itself, on the other hand is the breakthrough of application scenarios.

In the sign language AI technology itself, it can be divided into two solutions: recognition model and data set. On the data set, you can build a sign language recognition data set through contact with social institutions and hearing impaired people like Youtu, and expand the diversity of expression habits and speed for the local differences in sign language expression.

As for the recognition model, the industry also has a new concept of algorithm construction, such as extracting static and dynamic information in gestures through 2D convolutional neural network and 3D convolutional neural network, and improving the video recognition effect through integrated processing to get rid of the shackles of other sensors . At the same time, for the phenomenon of whole sentence expression in sign language, word-level information mining is added after the video frame, and the information proposed by the feature extractor is verified to further determine the boundary of the expression of the gesture to the word expression. In addition to improving recognition, it can also Improve the ability to summarize the regional expressions in natural sign language. On this basis, the ability of context understanding can also be introduced into the algorithm model in order to face the more complex sign language recognition and translation needs.

However, although the technology has been improved, the application scenarios will still be subject to some restrictions. For example, sign language recognition may rely on more powerful computing power, which is difficult to be portable and consumerized in a short time. However, through cooperation with the government, sign language recognition can enter some public service scenarios. Or, like iQiyi, starting with sign language generation can also help the hearing impaired.

In fact, it is not difficult to find, but if we look at the technology, we can find that the progress of sign language AI is not because of a sudden breakthrough in a basic science, but because there are more and more companies. He and the scholars have consistently invested in R & D and data accumulation for a long time in order to be free from the plight of the previous sign language corpus and continue to iterate on algorithms.

In other words, companies and scholars have invested the energy and financial resources that almost match the "most" AI technology in the "minority" AI technology. For the AI ​​industry, this is undoubtedly an affirmative spirit.

From people-oriented to science and technology: why should we be more active in guiding the current of technology?

Tencent Youtu's investment in sign language AI is actually a direction of current in the AI ​​industry that is faintly emerging.

A few days ago, at the Digital China Summit in Fuzhou, Ma Huateng mentioned the concept of "technology is good" and put forward "we believe that technology can benefit mankind; humans should make good use of technology, avoid abuse, and prevent evil use; technology should work hard To solve the social problems brought about by their own development. "

Coincidentally, after returning to Stanford, Li Feifei set up the HAI Research Institute (People-oriented Stanford Artificial Intelligence Research Institute) and began to serve as the director this year. HAI's research goal is to promote the development of AI technology in the direction of benefiting mankind and predict the practical impact of AI on human life.

Technology giants and academics have set their sights in the same direction, because people have begun to gradually discover the power of AI, 5G, industrial digitalization and other technical forces, and have reached the point where they have to be guided or even constrained.

As mentioned above, technology companies have played a great role in promoting this wave of technological development, and profit-seeking is naturally the instinct and nature of the company, so companies will devote themselves to those who satisfy most people and application scenarios. Extensive technology with relatively low R & D costs.

This behavior itself is understandable, but the efficiency improvement brought by new technologies such as AI is too significant. Whether people will squeeze or even marginalize those areas and groups that are temporarily unable to access new technologies, many people are thinking. one question.

For example, as the machine translation capabilities of mainstream languages ​​such as English, Japanese, French, and Russian continue to increase, will those small languages ​​with insufficient corpus and fewer users be further marginalized because they are not technically empowered?

Similarly, when public affairs management is increasingly replaced by AI technologies such as voice interaction and image recognition, will the hearing impaired and visually impaired groups encounter more troubles in obtaining services?

A similar situation has actually happened: at the end of 2018, the United Nations released a report on the digital results of the British government. The result is that the data shows that in England, the number of homeless people has increased by 60% since 2010. 1.2 million people, the demand for food banks for the relief of the poor has nearly quadrupled-because many poor people do not know how to apply for poverty subsidies on the Internet, and even their families have no way to connect to the Internet, they can only end up in poverty Deeper.

In many cases, even non-malicious technologies may have unpredictable trends. We should be more proactive in guiding science and technology towards good.

Conclusion

Later, we might as well look at such a set of numbers:

According to estimated data from the Beijing Hearing Association in 2017, the number of hearing-impaired people in China is approximately 72 million. Globally, data released by the World Health Organization show that a total of about 466 million people worldwide suffer from disability hearing loss.

——You see, the so-called “majority” and “minority” in this world are originally a relative concept, and there is no clear distinction between black and white. Especially for AI, a technology that is good at imitating human capabilities, its existence could have pushed down the air walls that hindered the communication between various groups, rather than making this trend intensify. Our goal of using technology to build a better world, since we can leave no one behind, we should not leave anyone behind.

Fortunately, from the sign language recognition and translation services for the hearing impaired, we can already see this trend-the brain that is good at computing is not a simulated object of AI, but also the human heart. We believe that under the guidance of academia and giants, more and more companies will pay attention to the development of barrier-free AI technology in the future and continue to break through various barriers.

Although love is silent, there is also an echo.

Although AI is silent, there is also an echo.