Authors : Vijay Kumar; Vishal Bansal; Mekhala; Adnan Shoeb

Volume/Issue : Volume 10 - 2025, Issue 5 - May

Google Scholar : https://tinyurl.com/muhdx2j2

DOI : https://doi.org/10.38124/ijisrt/25may1721

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : Gesture Talk is a real-time sign language-to-text conversion to create a communication environment for the deaf and able-bodied, with a touch of inclusivity across many other domains. The whole idea is to further develop a cheap and efficient solution to translate American Sign Language (ASL) gestures into text without the necessity for an interpreter. The process uses convolutional neural networks (CNNs) with a dual-layer classification algorithm, applying Gaussian blur and adaptive thresholding on the pre-processed webcam video frames and incorporating an auto-correct feature to enhance word prediction. The system was trained and tested on a labeled ASL gestures dataset prepared out of pre-processed 128×128 grayscale images obtained from RGB video inputs. Gesture Talk achieves recognition of 98.0% accuracy for ASL gestures, which surpasses many existing systems, and provides a user-friendly interface supporting all platforms, enabling deployment on desktops, mobiles, and web applications for much greater accessibility for deaf individuals.

References :

1. S. Thakar, S. Shah, B. Shah, and A. V. Nimkar, “Sign language to text conversion in real time using transfer learning,” in Proc. IEEE Int. Conf. Adv. Comput., Commun. Control Netw. (ICACCCN), 2022, pp. 1–6. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9971953
2. J. P, P. R. K. S. Bhama, and M. B, “Sign language recognition using deep CNN with normalised keyframe extraction and prediction using LSTM,” J. Sci. Ind. Res., 2023. [Online]. Available: https://or.niscpr.res.in/index.php/JSIR/article/view/2375
3. P. Dong, Y. Li, S. Chen, J. T. Grafstein, I. Khan, and S. Yao, “Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning,” Mater. Horizons, 2023. [Online]. Available: https://pubs.rsc.org/en/content/articlelanding/2023/mh/d3mh01062g/unauth
4. M. B. Munir, F. R. Alam, S. Ishrak, S. Hussain, and M. Shalahuddin, “A machine learning based sign language interpretation system for communication with deaf-mute people,” in Proc. ACM Int. Conf. Adv. Comput. Commun. Syst. (ICACCS), 2021, pp. 91–96. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/3471391.3471422
5. J.-H. Hsu, M.-H. Su, C.-H. Wu, and Y.-H. Chen, “Speech emotion recognition considering nonverbal vocalization in affective conversations,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2021, pp. 7438–7442. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9420285
6. M. Mahzabin, M. Hasan, S. Nahar, and M. U. Ahmed, “Automated hand gesture recognition using machine learning,” in Proc. IEEE Int. Conf. Robot., Autom., Artif. Intell. (RAAI), 2021, pp. 1–6. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9689817
7. R. K, P. A, P. K. S, S. Sasikala, and S. A. Kumar, “Two-way sign language conversion for assisting deaf-mutes using neural network,” in Proc. IEEE Int. Conf. Electron., Commun. Aerosp. Technol. (ICECA), 2022, pp. 1–5. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9785057
8. S. N. Koyineni, G. K. Sai, K. Anvesh, and T. Anjali, “Silent expressions unveiled: Deep learning for British and American sign language,” Procedia Comput. Sci., 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050924005751
9. S. Küçükdermenci, “Sign language voice convertor design using Raspberry Pi for impaired individuals,” in Proc. Int. Conf. Recent Innov. Res. Eng. Technol. (ICRIRET), 2023. [Online]. Available: https://asproceeding.com/index.php/icriret/article/view/1395
10. H. ZainEldin, S. A. Gamel, F. M. Talaat, M. Aljohani, N. A. Baghdadi, A. Malki, M. Badawy, and M. A. Elhosseini, “Silent no more: A comprehensive review of artificial intelligence, deep learning, and machine learning in facilitating deaf and mute communication,” Artif. Intell. Rev., 2024. [Online]. Available: https://link.springer.com/article/10.1007/s10462-024-10816-0
11. M. M. Chandra, S. Rajkumar, and L. S. Kumar, “Sign languages to speech conversion prototype using the SVM classifier,” in Proc. IEEE Int. Conf. Electron., Comput. Commun. Technol. (CONECCT), 2019, pp. 1–5. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8929356
12. Gayathri and A. S. Kumar, “Sign language recognition for deaf and dumb people using Android environment,” SSRN Electron. J., 2017. [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3374759
13. S. C. M. Murillo, M. C. A. E. Villanueva, K. I. M. Tamayo, M. J. V. Apolinario, M. J. D. Lopez, and Edd, “Speak the sign: A real-time sign language to text converter application for basic Filipino words and phrases,” Cent. Asian J. Math. Theory Comput. Sci., 2021. [Online]. Available: https://cajmtcs.centralasianstudies.org/index.php/CAJMTCS/article/view/92
14. P. Loke, J. Paranjpe, S. Bhabal, and K. Kanere, “Indian sign language converter system using an Android app,” in Proc. IEEE Int. Conf. Electron., Commun. Aerosp. Technol. (ICECA), 2017, pp. 436–440. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8212852
15. Singh and A. K. Bansal, “An integrated model for automated identification and learning of conversational gestures in human-robot interaction,” in Proc. Int. Conf. Comput. Vis. Robot., 2023, pp. 25–36. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-031-44127-1_3
16. Researchgate.net. Retrieved May 7, 2025, from https://www.researchgate.net/profile/Rupesh-Jaiswal/publication/371831685/figure/fig1/AS:11431281170202362@1687639425044/Hand-gesture-recognition-process-of-the-proposed-model.ppm