Authors : Swetha Tarigoppula; Tharala Sandeep; Panjala Saivani; Mothukuri Karthik; Puspati Sravani

Volume/Issue : Volume 11 - 2026, Issue 3 - March

Google Scholar : https://tinyurl.com/4r22cx3h

Scribd : https://tinyurl.com/4542ytte

DOI : https://doi.org/10.38124/ijisrt/26mar1909

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : Individuals with speech and hearing impairments face persistent barriers to independent communication, particularly in clinical settings where trained interpreters are seldom available. This paper presents SilentChat, a fully offline gesture-to-speech system that converts real-time hand gestures into spoken and on-screen text using a standard webcam. MediaPipe Hands extracts 21 three-dimensional landmarks per frame; wrist-origin translation and scale normalisation produce a 63-element feature vector classified by a Random Forest (RF) model. The system supports nine clinically relevant gestures and delivers consistent recognition performance across diverse users. Bidirectional communication is realised through offline text-to-speech (pyttsx3) and offline speech-to-text transcription (OpenAI Whisper). An emergency alert module, a picture-based communication gallery, and a custom gesture trainer extend communicative scope. All functional test cases were validated, confirming suitability for hospital and community deployment.

Keywords : Gesture Recognition, Assistive Technology, MediaPipe, Random Forest, Speech Synthesis, Offline Communication

References :

1. Achara, P., Sriram, P., Prabhu, S., Bhatt, A.: Assistive hand gesture glove for hearing and speech impaired using 1D-CNN on Android. In: Proc. IEEE ICCCA, pp. 1–5 (2020). https://ieeexplore.ieee.org/document/9143031/
2. Kamble, M., Patil, P.: Hand gesture recognition using MediaPipe Holistic and LSTM. In: Proc. IEEE ICDC, pp. 1–6 (2023). https://ieeexplore.ieee.org/document/10318885/
3. Jha, S., Pandey, A., Srivastava, A.: ISL recognition and translation using MediaPipe and LSTM. In: Proc. IEEE ICICC, pp. 1–6 (2023). https://ieeexplore.ieee.org/document/10235113/
4. Cruz, J.D., Bernal, L.C.A., Palaoag, D.: Real-time hand gesture recognition using MediaPipe Holistic and LSTM with MLP. In: Proc. IEEE HNICEM, pp. 1–6 (2022). https://ieeexplore.ieee.org/document/10001800/
5. Lugaresi, C., et al.: MediaPipe: A framework for perceiving and processing reality. In: Workshop on Perception and Interactive Applications, IEEE CVPR (2019). https://arxiv.org/abs/1906.08172
6. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011). https://jmlr.org/papers/v12/pedregosa11a.html
7. Radford, A., et al.: Robust speech recognition via large-scale weak supervision. arXiv:2212.04356 (2022). https://arxiv.org/abs/2212.04356
8. Srihari, H., Nayana, R.B., Suhas, S.: Real-time hand gesture recognition for assistive technologies. In: Proc. IEEE ICOEI, pp. 1–5 (2019). https://ieeexplore.ieee.org/document/8697363/
9. Rai, K.S., Shrestha, P., Chitrakar, S., Banerjee, A.K.: A real-time sign language recognition system using MediaPipe and Random Forest with text-to-speech. In: Proc. IEEE AISP (2025). https://ieeexplore.ieee.org/document/10986900
10. Prakash, Y., Sriram, D., Varma, R.: Real-time sign language recognition and translation using MediaPipe and Random Forests. In: Proc. IEEE ICACCS (2024). https://ieeexplore.ieee.org/document/10932602/
11. Sharma, R., Kumar, T., Jain, A.: Hand gesture recognition using MediaPipe and CNN for ISL with regional language TTS. In: Proc. IEEE ICACTA, pp. 1–6 (2023). https://ieeexplore.ieee.org/document/10334218/
12. Mariappan, H.M., Gomathi, V.: Real-time recognition of Indian Sign Language. In: Proc. IEEE ICCIDS, pp. 1–6 (2019). https://ieeexplore.ieee.org/document/8862125/
13. Sajanraj, A., Beena, M.: Real-time ISL recognition using grid-based features. In: Proc. IEEE ICOEI, pp. 1–6 (2018). https://ieeexplore.ieee.org/document/8493808/
14. Wadhawan, S., Kumar, P.: Hand landmark distance-based sign language recognition using MediaPipe. In: Proc. IEEE IC3A, pp. 1–5 (2023). https://ieeexplore.ieee.org/document/10100061/
15. Renimol, J.M., Thomas, B.L.: Indian sign language to voice using ESP32-Cam and MediaPipe. In: Proc. IEEE ICECT (2025). https://ieeexplore.ieee.org/document/11135993/