Computer Vision-Powered Indian Sign Language Recognition System


Authors : Dr. V. Sathiyasuntharam; Harsh Nagar; Divek Jain; Harmanpreet Kaur

Volume/Issue : Volume 10 - 2025, Issue 11 - November


Google Scholar : https://tinyurl.com/5ysx2bw7

Scribd : https://tinyurl.com/msktacp8

DOI : https://doi.org/10.38124/ijisrt/25nov517

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 30 to 40 days to display the article.


Abstract : This systematic review analyzes eight pioneering ISL recognition studies from 2012 to 2024 for tracking the transition from conventional machine learning to advanced deep learning for helping the deaf community in India. Early methods, such as SVM with Hu Moments, attained up to 97.5% accuracy but relied on a small dataset and heavy preprocessing. The latest deep learning models, especially CNN-based, have achieved more than 99% accuracy for the recognition of the static alphabet. The system that represents the current state-of-the-art is a hybrid MF-DNet, which includes VGG-19, MediaPipe, and BiLSTM. MF-DNet recognizes dynamic words with an accuracy of 96.88% for 263 classes, addressing occlusion. However, severe challenges persist: the critical lack of standardized large-scale ISL datasets, extremely minimal studies regarding continuous sentence interpretation, and the absence of real-time deployment on mobile platforms. The future research focus will fall on the creation of the ISL-100K benchmark, application of Vision Transformers, and lightweight end-to-end bidirectional translation systems towards the ultimate goal of achieving more than 95% accuracy in real-time ISL sentence recognition on mobile.

References :

  1. Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. Time Series Analysis: Forecasting and Control. Wiley, 2015.
  2. Fan, S., & Hyndman, R. J. “Short-Term Load Forecasting Based on a Semi-Parametric Additive Model.” IEEE Transactions on Power Systems, vol. 27, no. 1, pp. 134–141, 2012.
  3. Hochreiter, S., & Schmidhuber, J. “Long Short-Term Memory.” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  4. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation.” arXiv preprint arXiv:1406.1078, 2014.
  5. Zhang, G., Eddy Patuwo, B., & Hu, M. Y. “Forecasting with Artificial Neural Networks: The State of the Art.” International Journal of Forecasting, vol. 14, no. 1, pp. 35–62, 1998.
  6. UCI Machine Learning Repository. “Individual Household Electric Power Consumption Dataset.” University of California, Irvine. Available: https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption.
  7. Weron, R. “Electricity Price Forecasting: A Review of the State-of-the-Art with a Look into the Future.” International Journal of Forecasting, vol. 30, no. 4, pp. 1030–1081, 2014.
  8. Marino, D. L., Amarasinghe, K., & Manic, M. “Building Energy Load Forecasting Using Deep Neural Networks.” 2016 International Joint Conference on Neural Networks (IJCNN), pp. 4394–4401.
  9. Wang, Y., Chen, Q., Kang, C., Xia, Q., & Zhang, M. “Load Profiling and Its Application to Demand Response: A Review.” IEEE Transactions on Smart Grid, vol. 10, no. 3, pp. 3125–3143, 2019.
  10. Amasyali, K., & El-Gohary, N. M. “A Review of Data-Driven Building Energy Consumption Prediction Studies.” Renewable and Sustainable Energy Reviews, vol. 81, pp. 1192–1205, 2018.
  11. Smyl, S. “A Hybrid Method of Exponential Smoothing and Recurrent Neural Networks for Time Series Forecasting.” International Journal of Forecasting, vol. 36, no. 1, pp. 75–85, 2020.
  12. Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y., & Zhang, Y. “Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network.” IEEE Transactions on Smart Grid, vol. 10, no. 1, pp. 841–851, 2019.
  13. Ahmad, T., & Chen, H. “A Review on Machine Learning Forecasting Growth Trends and Their Real-Time Applications in Different Energy Systems.” Sustainable Cities and Society, vol. 54, p. 102010, 2020.
  14. Li, C., Ding, Z., Zhao, D., Yi, J., & Zhang, G. “Building Energy Consumption Prediction: An Extreme Deep Learning Approach.” Energies, vol. 10, no. 10, p. 1525, 2017.
  15. Mocanu, E., Nguyen, P. H., Gibescu, M., & Kling, W. L. “Deep Learning for Estimating Building Energy Consumption.” Sustainable Energy, Grids and Networks, vol. 6, pp. 91–99, 2016.

This systematic review analyzes eight pioneering ISL recognition studies from 2012 to 2024 for tracking the transition from conventional machine learning to advanced deep learning for helping the deaf community in India. Early methods, such as SVM with Hu Moments, attained up to 97.5% accuracy but relied on a small dataset and heavy preprocessing. The latest deep learning models, especially CNN-based, have achieved more than 99% accuracy for the recognition of the static alphabet. The system that represents the current state-of-the-art is a hybrid MF-DNet, which includes VGG-19, MediaPipe, and BiLSTM. MF-DNet recognizes dynamic words with an accuracy of 96.88% for 263 classes, addressing occlusion. However, severe challenges persist: the critical lack of standardized large-scale ISL datasets, extremely minimal studies regarding continuous sentence interpretation, and the absence of real-time deployment on mobile platforms. The future research focus will fall on the creation of the ISL-100K benchmark, application of Vision Transformers, and lightweight end-to-end bidirectional translation systems towards the ultimate goal of achieving more than 95% accuracy in real-time ISL sentence recognition on mobile.

CALL FOR PAPERS


Paper Submission Last Date
30 - November - 2025

Video Explanation for Published paper

Never miss an update from Papermashup

Get notified about the latest tutorials and downloads.

Subscribe by Email

Get alerts directly into your inbox after each post and stay updated.
Subscribe
OR

Subscribe by RSS

Add our RSS to your feedreader to get regular updates from us.
Subscribe