Authors : Sohan Maurya; Sparsh Doshi; Harsh Jaiswar; Sahil Karale; Sneha Burnase; Dr. Poonam. N. Sonar

Volume/Issue : Volume 9 - 2024, Issue 5 - May

Google Scholar : https://tinyurl.com/wnav297v

Scribd : https://tinyurl.com/ym5srjwu

DOI : https://doi.org/10.38124/ijisrt/IJISRT24MAY500

Abstract : Individuals with hearing impairments communicate mostly through sign language. Our goal was to create an American Sign Language recognition dataset and utilize it in a neural network-based machine learning model that can interpret hand gestures and positions into natural language. In our study, we incorporated the SVM, CNN and Resnet-18 models to enhance predictability when interpreting ASL signs through this new dataset, which includes provisions such as lighting and distance limitations. Our research also features comparison results between all the other models implemented under invariant conditions versus those using our proposed CNN model. As demonstrated by its high levels of precision at 95.10% despite changes encountered during testing procedures like varying data sets or scene configurations where losses are minimal (0.545), there exists great potential for future applications in image recognition systems requiring deep learning techniques. Furthermore, these advancements may lead to significant improvements within various fields related explicitly to speech-language therapy sessions designed specifically around helping people overcome challenges associated with deafness while building bridges towards improved social integration opportunities.

Keywords : Image Recognition, Image Classification, Feature Extraction, Deep Learning, Convolutional Neural Network (CNN), Sign Language Translation, American Sign Language (ASL), Real-Time Recognition.

References :

1. Wadhawan, Ankita., & Kumar, Parteek. (2020). Deep learning-based sign language recognition system for static signs. Neural Computing and Applications , 32 , 7957 - 7968. [doi.org/10.1007/s00521-019-04691-y]
2. Masood, S.., Srivastava, Adhyan., Thuwal, H.., & Ahmad, Musheer. (2018). Real-Time Sign Language Gesture (Word) Recognition from Video Sequences Using CNN and RNN. , 623-632. [doi.org/10.1007/978-981-10-7566-7_63]
3. Rastgoo, R.., Kiani, K.., & Escalera, Sergio. (2020). Video-based isolated hand sign language recognition using a deep cascaded model. Multimedia Tools and Applications, 79, 22965 - 22987. [doi.org/10.1007/s11042-020-09048-5]
4. Koller, Oscar., Zargaran, Sepehr., Ney, H.., & Bowden, R.. (2016). Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition. [doi.org/10.5244/C.30.136]
5. Koller, Oscar., Zargaran, Sepehr., Ney, H.., & Bowden, R.. (2018). Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs. International Journal of Computer Vision, 126, 1311-1325. [doi.org/10.1007/s11263-018-1121-3]
6. Katoch, Shagun., Singh, Varsha., & Tiwary, U.. (2022). American Sign Language recognition system using SURF with SVM and CNN. Array , 14 , 100141. [doi.org/10.1016/j.array.2022.100141]
7. Barbhuiya, Abul Abbas., Karsh, R.., & Jain, Rahul. (2020). CNN based feature extraction and classification for sign language. Multimedia Tools and Applications , 80 , 3051 – 3069. [doi.org/10.1007/s11042-020-09829-y]
8. Huang, Jie., Zhou, Wen-gang., Li, Houqiang., & Li, Weiping. (2019). Attention-Based 3D-CNNs for Large-Vocabulary Sign Language Recognition. IEEE Transactions on Circuits and Systems for Video Technology , 29 , 2822-2832. [doi.org/10.1109/TCSVT.2018.2870740]
9. Sasikala, N., Swathipriya, V., Ashwini, M., Preethi, V., Pranavi, A., and Ranjith, M. Feature extraction of real-time image using sift algorithm. European Journal of Electrical Engineering and Computer Science 4, 3 (2020). [doi.org/10.24018/ejece.2020.4.3.206.]
10. Dalal, N., and Triggs, B. Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05) (2005), vol. 1, Ieee, pp. 886893. [doi 10.1109/CVPR.2005.177]
11. Rekha, J., Bhattacharya, J., and Majumder, S. Shape, texture and local movement hand gesture features for Indian sign language recognition. In 3rd international conference on trends in information sciences & computing (TISC2011) (2011), IEEE, pp. 3035. [dx.doi.org/10.1109/tisc.2011.6169079]
12. Ram, P., and Padmavathi, S. Analysis of harris corner detection for color images. In 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES) (2016), IEEE, pp. 405410. [doi: 10.1109/SCOPES.2016.7955862]
13. Chang, F., and Chen, C.-J. A component labelling algorithm using contour tracing technique. In Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. (2003), vol. 3, Citeseer, pp. 741741.  [doi:10.1109/ICDAR.2003.1227760] https://miro.medium.com/v2/resize:fit:1400/format:webp/1*Ha7EfcfB5mY2RIKsXaTRkA.png