Online sign language interpreter
- OpenCV
- Python
AI algorithm that converts video of a person using sign language into a text transcript
Latest achievements of CV specialists give us an opportunity to let people with hearing or speech disabilities communicate seamlessly and share knowledge.
Recognition of words of the sign language is carried out according to the algorithm:
Information preparation. Primary information allows you to define words, but due to the huge variability of movements in three-dimensional space, this is only possible for a very limited number of words. The result of performing transformations on the primary information is a vector containing the maximum amount of useful information (for one video frame) with a minimum space usage.
Recognition of words of the sign language is carried out by a set of vectors.
Extended with text-to-speech and speech-to-text methods, this algorithm is able to translate:
The sign language lacks punctuation marks and any other components that could indicate that one gesture ended and another began. Therefore, in order to recognize words, it was necessary to solve the problem of finding the beginning and the end of a word on a continuous time series. To do this, a sliding window algorithm (Windowing method) was developed, which, based on a set of many previous words, can predict the next one.
The algorithm is implemented for 2 languages: Russian and English.
Similar Projects
Virtual try-on tool for makeup products
The system consists of a face detection and segmentation model and an algorithm that allows recoloring objects without losing their original texture.
Influencers search API
The API we've developed can help a company to find a best matching influencer. Want to promote your new music album in Instagram stories? Just send this to the API and get a list of accounts which will increase your revenue.
Cardiac coherence training app
Mobile application with a set of exercises to practice breathing techniques.