Keywords
Dactyl alphabet
Deep learning
Hand gestures recognition
Honduran sign language
How to Cite
Abstract
Introduction The critical problem of hearing and speech difficulties among thousands of Hondurans is a latent need, and in search of tools that allow the inclusion of the population with this condition is sought through the introduction of a convolutional neural network (CNN) designed for real-time detection and classification of the dactyl alphabet of the Honduran sign language (LESHO). This study represents an important step forward in promoting accessibility and inclusion of the Honduran deaf community, which faces few technological solutions adapted to their needs. Methods A proprietary dataset comprising more than 8,000 images with various angles and gestures was meticulously constructed, ensuring robust training and evaluation of the model. Spiral research methodology was employed to iteratively refine network performance, with an emphasis on accuracy and real-time deployment capabilities. Results The final model showed exceptional results during the testing, achieving a mean average precision (mAP) of 98.8%, a precision of 97.4%, and a recall of 97.7%. These metrics underscore the reliability of the CNN in recognizing both static and dynamic gestures with minimal errors. Conclusion The model’s capacity to generalize indicates its potential for further applications, such as full sign language interpretation and expanded vocabulary training
References
Amin, M., Rizvi, S., Mazzei, A., & Anselma, L. (2023). Assistive Data Glove for Isolated Static Postures Recognition in American Sign Language Using Neural Network. Electronics 2023, 12, 1904. doi: https://doi.org/10.3390/electronics12081904
Boehm, B. (1995). A Spiral Model of Software Development and Enhancement. Interactive Technologies, 281-292.
Cortés, C. A. B., & Duke, A. M. R. (2023, November). A Comparative Study of CNN Trainings for the Detection of E. Coli, P. Aeruginosa and S. Aureus: A Raspberry Pi-Based Prototype. In 2023 IEEE 41st Central America and Panama Convention (CONCAPAN XLI) (pp. 1-6). https://doi.org/10.1109/CONCAPANXLI59599.2023.10517539
Daniels, S., Suciati, N., & Fathichah, C. (2021, February). Indonesian sign language recognition using yolo method. In IOP Conference Series: Materials Science and Engineering (Vol. 1077, No. 1, p. 012029). https://iopscience.iop.org/article/10.1088/1757-899X/1077/1/012029
Düntsch, I., & Gediga, G. (2019, May). Confusion matrices and rough set data analysis. In Journal of Physics: Conference Series (Vol. 1229, No. 1, p. 012055). IOP Publishing. http://dx.doi.org/10.1088/1742-6596/1229/1/012055
Fernandez, O. A., Ordóñez-Ávila, J. L., & Magomedov, I. A. (2021, December). Evaluation of parameters in a neural network for detection of red ring pest in oil palm. In AIP Conference Proceedings (Vol. 2442, No. 1). https://doi.org/10.1063/5.0076481
Fuentes, M. S., Zelaya, N. A. L., & Avila, J. L. O. (2020, April). Coffee fruit recognition using artificial vision and neural networks. In 2020 5th International Conference on Control and Robotics Engineering (ICCRE) (pp. 224-228). https://doi.org/10.1109/ICCRE49379.2020.9096441
Garcia, B., & Viesca, S. A. (2016). Real-time American sign language recognition with convolutional neural networks. Convolutional Neural Networks for Visual Recognition, 2(225-232), 8.
Grif, M. G., & Kondratenko, Y. K. (2021, October). Development of a software module for recognizing the fingerspelling of the Russian Sign Language based on LSTM. In Journal of Physics: Conference Series (Vol. 2032, No. 1, p. 012024). IOP Publishing. https://iopscience.iop.org/article/10.1088/1742-6596/2032/1/012024
Interiano, A. A. A., Palma, M. A. M., & Leiva, K. M. R. (2023, December). Prediction of Spinal Abnormalities in Neuroradiology Images Applying Deep Transfer Learning. In 2023 IEEE International Conference on Machine Learning and Applied Network Technologies (ICMLANT) (pp. 1-7). https://doi.org/10.1109/ICMLANT59547.2023.10372991
Jimenez-Nixon, D. A., Corrales, J. F. M., & Reyes-Duke, A. M. (2022, December). Coral detection using artificial neural networks based on blurry images for reef protection in cayo blanco, honduras. In 2022 IEEE International Conference on Machine Learning and Applied Network Technologies (ICMLANT) (pp. 1-6). https://doi.org/10.1109/ICMLANT56191.2022.9996481
Köpüklü, O., Gunduz, A., Kose, N., & Rigoll, G. (2019, May). Real-time hand gesture detection and classification using convolutional neural networks. In 2019 14th IEEE international conference on automatic face & gesture recognition (FG 2019) (pp. 1-8). https://doi.org/10.1109/FG.2019.8756576
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems, 33(12), 6999-7019. https://doi.org/10.1109/TNNLS.2021.3084827
Poder Legislativo de Honduras. (2014, May 22). Decreto 321/2013. Ley de la Lengua de Señas Hondureña (LESHO). Diario Oficial de la República de Honduras "La Gaceta".
Terven, J., Cordova-Esparza, D. M., Ramirez-Pedraza, A., & Chavez-Urbiola, E. A. (2023). Loss functions and metrics in deep learning. A review. arXiv preprint arXiv:2307.02694.
Williams, H. (2010). A Sociolinguistic survey of the Honduran Deaf community. p. 24. Retrieved from https://www.sil.org/resources/publications/entry/9181
World Health Organization. (2021). World report on hearing. Retrieved april 27, 2024, from https://www.who.int/publications/i/item/9789240020481
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright (c) 2024 Jorge Alejandro Ramos, Alicia María Reyes-Duke, Alberto Carrasco Bardales