Authors : Ayush Fulsundar; Deep; Sujal; Lavishka; Vijaya S. Patil

Volume/Issue : Volume 11 - 2026, Issue 4 - April

Google Scholar : https://tinyurl.com/57b8hu6f

Scribd : https://tinyurl.com/bdhh8jp4

DOI : https://doi.org/10.38124/ijisrt/26apr1909

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : EyeVox is a secure multimodal human–computer interaction system designed to enable hands-free desktop interaction using eye gaze and voice input. The system utilizes realtime iris and facial landmark detection through MediaPipe to control mouse cursor movement based solely on eye movements, eliminating the dependency on conventional input devices such as keyboards and mice. Additionally, a voice assistant module allows users to execute system commands and perform authentication using a predefined voice phrase. To enhance security, EyeVox integrates dual-biometric authentication combining gaze behavior and voice recognition. Advanced security mechanisms including role-based access control, continuous authentication, intelligent threat detection, shoulder surfing resistance, and immutable audit logging are incorporated to protect against unauthorized access, spoofing attacks, and session hijacking. The system is specifically designed with accessibility in mind, making it suitable for users with physical or motor impairments. Experimental evaluation on consumer-grade hardware demonstrates low latency, high authentication accuracy, and stable real-time performance. EyeVox offers a scalable, cost-effective, and secure solution for assistive computing and modern secure desktop environments.

Keywords : Gaze Tracking, Voice Assistant, Multimodal Biometrics, Human–Computer Interaction, Continuous Authentication, Shoulder Surfing Protection, Secure Systems, Accessibility.

References :

1. M. Parisay, C. Poullis, and M. Kersten, “Eyetap: A novel technique using voice inputs to address the midas touch problem for gazebased interactions,” arXiv preprint arXiv:2002.08455, 2020. [Online]. Available: https://arxiv.org/abs/2002.08455
2. M. Paing, J. A., and P. C., “Design and development of an assistive system based on eye tracking,” Electronics, vol. 11, no. 4, p. 535, 2022. [Online]. Available: https://www.mdpi.com/2079-9292/11/4/535
3. P. Tangade, S. Musale, G. Pasalkar, M. Umale, and S. Awate, “A review paper on mouse pointer movement using eye tracking system and voice recognition,” International Journal of Emerging Engineering Research and Technology, vol. 2, no. 8, pp. 135–138, 2014. [Online]. Available: https://ijeert.ijrsset.org/pdf/v2-i8/20.pdf
4. S. Zhai, C. Morimoto, and S. Ihde, “Manual and gaze input cascaded (magic) pointing,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 246–253, 1999.
5. P. Qvarfordt, D. Beymer, and S. Zhai, “Realtourist: A system for exploring tourist attractions using eye gaze and speech,” Human-Computer Interaction - INTERACT 2005, pp. 1–14, 2005.
6. F. Abbaas and G. Serpen, “Evaluation of biometric user authentication using an ensemble classifier with face and voice recognition,” arXiv preprint arXiv:2006.00548, 2020. [Online]. Available: https: //arxiv.org/abs/2006.00548
7. R. Ramachandra, M. Stokkenes, A. Mohammadi, S. Venkatesh, K. Raja, P. Wasnik, E. Poiret, S. Marcel, and C. Busch, “Smartphone multi-modal biometric authentication: Database and evaluation,” arXiv preprint arXiv:1912.02487, 2019. [Online]. Available: https: //arxiv.org/abs/1912.02487
8. M. Abuhamad, A. Abusnaina, D. Nyang, and D. Mohaisen, “Sensorbased continuous authentication of smartphones’ users using behavioral biometrics: A contemporary survey,” arXiv preprint arXiv:2001.08578, 2020. [Online]. Available: https://arxiv.org/abs/2001.08578
9. M. Khamis, A. Khamis, M. Abusnaina, D. Nyang, and D. Mohaisen, “Gazetouchpin: Protecting sensitive data on mobile devices using secure multimodal authentication,” Proceedings of the 19th ACM International Conference on Multimodal Interaction, pp. 1–9, 2017. [Online]. Available: https://www.mkhamis.com/data/papers/khamis2017icmi.pdf
10. ——, “Gazetouchpass: Multimodal authentication using gaze and touch on mobile devices,” Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1–8, 2016. [Online]. Available: https://www.mkhamis.com/data/papers/ khamis2016chi.pdf
11. S. Krishna, P. Lopes, and P. Maes, “Multimodal biometric authentication for vr/ar using eeg and eye tracking,” in Proceedings of the 21st ACM International Conference on Multimodal Interaction, 2019, pp. 43–52. [Online]. Available: https://dl.acm.org/doi/10.1145/3340555.3353736
12. F. Alt, E. Katsini, and M. Khamis, “The role of eye gaze in security and privacy applications,” CHI Conference on Human Factors in Computing Systems, pp. 1–13, 2020. [Online]. Available: https://florian-alt.org/unibw/wp-content/publications/katsini2020chi.pdf
13. J. Doe and J. Smith, “Pre-attentivegaze: Gaze-based authentication dataset with pre-attentive processing,” Scientific Data, vol. 12, no. 1, pp. 1–10, 2025. [Online]. Available: https://www.nature.com/articles/ s41597-025-04538-3
14. S. Holland and S. Komogortsev, “Gaze trajectory as a biometric modality,” in Proceedings of the 2011 Workshop on Eye Gaze in Intelligent Human Machine Interaction, 2011, pp. 1–6. [Online]. Available: https://www.researchgate.net/publication/221334850 Gaze Trajectory as a Biometric Modality
15. Y. Wang and H. Zhao, “Gaze analysis: A survey on its applications,” Image and Vision Computing, vol. 140, p. 104731, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0262885624000647