Sign Language to Text and Voice Conversion for Speaking Impaired People Using Convolutional Neural Network

Abstract

Sign language, a visual-gestural form of communication predominantly utilised by individuals with hearing impairments, has distinct problems in routine encounters with non-signers. To solve this problem with communication, we suggest a complete Sign Language Detection and Translation System (SLDTS) that uses Convolutional Neural Networks (CNN) to recognise sign language and turn it into audio. It also uses keyword recognition to make communication more efficient. The SLDTS is built on deep learning methods, notably CNN-based models, that can accurately find and recognise sign language motions. These models learn quickly how to recognise different hand shapes, movements, and gestures by being trained on massive, annotated datasets of sign language gestures. By analysing input video streams or images in real time, the CNN-based sign language detection component of the SLDTS can efficiently recognise and understand sign language motions, creating the foundation for effective communication between sign language users and non-signers. When the SLDTS sees sign language gestures, it uses text-to-speech (TTS) synthesis to turn them into spoken words. This translation procedure takes the recognised signs and turns them into text, then combines them into audio output. The SLDTS uses powerful TTS technology to make sure that the translated audio is clear and sounds natural, so that those who don't know sign language may understand what the signer is trying to say. This feature lets people with and without hearing problems talk to each other in real time, which helps them understand each other better and interact more.