Sub-Degree YPEC 2020

SD09 – SIGNCHAT – an AI Powered Sign Language Translator

There are around 18% of people who are suffering from Hearing Impairment (Hearing loss). Hearing Impairment can highly affect their daily life in work environment, family or social. Only loneliness and silent accompany them for their whole life…… This project is a real-time sign language translation application, SignChat. The project aims at helping sign language user to improve their quality of life. It will help them to communicate with other people, avoid misunderstanding and help them to “speak up” in a direct way. Technically, the application consists of hand tracking and sign language translation features. Its real-time, 3-dimensions hand tracking feature is powered by Google’s mediapipe framework. Then, the tracking data is further processed into textual message with the AI model built by the team using LSTM neural network. The application provides both AI training and AI translation features. That means the sign language database could be even extended for new signs. Now, the team designed and developed an application that translate sign language instantly. The team wishes to help them to break through themselves by technology. The team hopes these applications can break the barriers between sign language and words.