Authors : Azizi Wasike; Ismail Kamukama; Yusuf Abass Aleshinloye; Adeleke Raheem Ajiboye; Jamir Ssebadduka

Volume/Issue : Volume 9 - 2024, Issue 10 - October

Google Scholar : https://tinyurl.com/mvaepn9m

Scribd : https://tinyurl.com/8e8kbrde

DOI : https://doi.org/10.38124/ijisrt/IJISRT24OCT410

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : This review explores recent advancements in Natural Language Understanding-driven Machine Translation (NLU-MT) with a focus on English and the low-resource dialectal Lusoga. A Low-resource language, such as Lusoga, faces significant challenges in Machine Translation (MT) due to the scarcity of high-quality parallel corpora, the complex morphology inherent in Bantu languages, and the dialectal variations within Lusoga itself, particularly between Lutenga and Lupakoyo. This paper examines the role of NLU-based MT systems in overcoming these challenges by shifting from word-for-word mapping to meaning-based translations, enabling better handling of these dialectal differences. We highlight the success of leveraging linguistic similarities between Lusoga and related languages, such as Luganda, to improve translation performance through multilingual transfer learning techniques. Key advancements include the use of transformer-based architectures such as Multilingual Bidirectional and Auto-Regressive Transformer (mBART) and Multilingual Text-To-Text Transfer Transformer (mT5), specifically selected for their effectiveness in NLU-driven contexts, which have shown promise in enhancing translation accuracy for African low-resource languages. However, the review also identifies ongoing obstacles, including historical low demand and the lack of well-developed corpora, which hinder scalability. The paper concludes by emphasizing the potential of hybrid approaches that combine community-driven corpus-building initiatives with improved model architectures to drive further progress in low-resource MT. Ultimately, NLU-MT is positioned as a crucial tool not only for bridging communication gaps but also for preserving linguistic diversity and cultural heritage.

Keywords : Natural Language Understanding; Machine Translation; Low-Resource Languages; Lusoga, Dialectal Variations; Transfer Learning; Community-driven Corpus Building; mBART; mT5;mBERT.

References :

1. A. B. Olani, A. B. Olani, T. B. Muleta, and D. H. Rikitu, “Impacts of language barriers on healthcare access and quality among Afaan Oromoo ‑ speaking patients in Addis Ababa ,” BMC Health Serv. Res., pp. 1–12, 2023, doi: 10.1186/s12913-023-09036-z.
2. C. W. Gulere, “Standardised Language-Based Orthographies,” in LITERACY: A BRIDGE TO EQUITY, 2019.
3. J. Dong, “Transfer Learning-Based Neural Machine Translation for Low-Resource Languages,” ACM Trans. Asian Low-Resource Lang. Inf. Process., 2023, doi: 10.1145/3618111.
4. K. Ogueji, Y. Zhu, and J. Lin, “Small Data ? No Problem ! Exploring the Viability of Pretrained Multilingual Language Models for Low-Resource Languages,” in Proceedings ofthe 1st Workshop on Multilingual Representation Learning, 2021, pp. 116–126.
5. A. Nzeyimana, “Low-resource neural machine translation with morphological modeling,” Find. Assoc. Comput. Linguist. NAACL 2024 - Find., pp. 182–195, 2024.
6. A. Hernández, R. M. Ortega-mendoza, E. Villatoro-tello, C. J. Camacho-bello, and O. Pérez-cortés, “Natural Language Understanding for Navigation of Service Robots in Low-Resource Domains and Languages : Scenarios in Spanish and Nahuatl,” Mathematics, vol. 12, no. 8, 2024, doi: https://doi.org/10.3390/math12081136.
7. S. Ghosh, “Natural Language Processing: Basics, Challenges, and Clustering Applications,” in A Handbook of Computational Linguistics: Artificial Intelligence in Natural Language Processing, 2024. doi: http://dx.doi.org/10.2174/9789815238488124020006.
8. M. M. I. Alam, S. Ahmadi, and A. Anastasopoulos, “CODET: A Benchmark for Contrastive Dialectal Evaluation of Machine Translation,” EACL 2024 - 18th Conf. Eur. Chapter Assoc. Comput. Linguist. Find. EACL 2024, pp. 1790–1859, 2024.
9. P. Prasada, M. Vishwanatha, and P. Rao, “Reinforcement of low-resource language translation with neural machine translation and backtranslation synergies,” Int. J. Artif. Intell., vol. 13, no. 3, pp. 3478–3488, 2024, doi: 10.11591/ijai.v13.i3.pp3478-3488.
10. T. Sumanth, “Deep Learning for Natural Language Processing,” Int. J. Adv. Res. Eng. Technol., vol. 190, no. 5, pp. 523–533, 2021, doi: 10.1007/978-981-16-0882-7_45.
11. C. C. Emezue and B. F. P. Dossou, “MMTAfrica: Multilingual Machine Translation for African Languages,” WMT 2021 - 6th Conf. Mach. Transl. Proc., pp. 398–411, 2021.
12. B. Akera et al., “Machine translation for african languages: commutity creation of datasets and models in Uganda,” 3rd Work. African Nat. Lang. Process. 2022., no. 61733011, pp. 1–13, 2022, [Online]. Available: https://openreview.net/forum?id=BK-z5qzEU-9
13. A. Vaswani, “Attention Is All You Need,” in Conference on Neural Information Processing Systems (NIPS 2017), 2017.
14. Y. Zhao, J. Zhang, and C. Zong, “Transformer: A General Framework from Machine Translation to Others,” Mach. Intell. Res., vol. 20, no. 4, pp. 514–538, 2023, doi: 10.1007/s11633-022-1393-5.
15. M. C. Kenton, L. Kristina, and J. Devlin, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” 2019.
16. M. Rahman, A. I. Shiplu, Y. Watanobe, and A. Alam, “RoBERTa-BiLSTM : A Context-Aware Hybrid Model for Sentiment Analysis,” 2021. [Online]. Available: https://arxiv.org/pdf/2406.00367
17. M. Joshi, D. Chen, Y. Liu, D. S. Weld, L. Zettlemoyer, and O. Levy, “SpanBERT: Improving Pre-training by Representing and Predicting Spans,” 2020.
18. D. Jurafsky and C. D. Manning, “Mini But Mighty : Efficient Multilingual Pretraining with Linguistically-Informed Data Selection,” in Findings of the Association for Computational Linguistics: EACL 2023, 2023, pp. 1251–1266.
19. N. R. Robinson, P. Ogayo, D. R. Mortensen, and G. Neubig, “ChatGPT MT : Competitive for High- ( but not Low- ) Resource Languages,” 2023. [Online]. Available: https://aclanthology.org/2023.wmt-1.40.pdf
20. T. B. Brown et al., “Language Models are Few-Shot Learners,” 2022.
21. Y. Tang et al., “Multilingual Translation with Extensible Multilingual Pretraining and Finetuning,” 2020.
22. C. Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, pp. 1–67, 2020.
23. A. Oladipo et al., “Better Quality Pretraining Data and T5 Models for African Languages,” 2023. [Online]. Available: https://aclanthology.org/2023.emnlp-main.11.pdf
24. R. Kimera, D. N. Rim, and H. Choi, “Building a Parallel Corpus and Training Translation Models Between Luganda and English,” J. KIISE, vol. 49, no. 11, pp. 1009–1016, 2022, doi: 10.5626/jok.2022.49.11.1009.
25. M. Nabirye, G. M. De Schryver, and J. Verhoeven, “Lusoga (Lutenga),” J. Int. Phon. Assoc., vol. 46, no. 2, pp. 219–228, 2016, doi: 10.1017/S0025100315000249.
26. F. Śanchez-Martinez et al., “An English-Swahili parallel corpus and its use for neural machine translation in the news domain,” Proc. 22nd Annu. Conf. Eur. Assoc. Mach. Transl. EAMT 2020, pp. 299–308, 2020.
27. A. L. Tonja, O. Kolesnikova, A. Gelbukh, and G. Sidorov, “Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data,” Appl. Sci., vol. 13, no. 2, 2023, doi: 10.3390/app13021201.
28. H. Wan-hua and U. Kruschwitz, “Investigating Neural Machine Translation for Low-Resource Languages : Using Bavarian as a Case Study,” 2024. [Online]. Available: https://aclanthology.org/2024.sigul-1.20.pdf
29. T. Ngo, P. Nguyen, V. V. Nguyen, T. Ha, and L. Nguyen, “An Efficient Method for Generating Synthetic Data for Low-Resource Machine Translation An Efficient Method for Generating Synthetic Data for Low-Resource Machine Translation,” Appl. Artif. Intell., vol. 36, no. 1, 2022, doi: 10.1080/08839514.2022.2101755.
30. E. S. A. Lee et al., “Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2022, pp. 58–67. doi: 10.18653/v1/2022.findings-acl.6.
31. E. Nyoni and B. A. Bassett, “Low-Resource Neural Machine Translation for Southern African Languages,” arXiv:2104.00366. Accessed: Apr. 09, 2023. [Online]. Available: http://arxiv.org/abs/2104.00366
32. F. N. Al-Wesabi, H. J. Alshahrani, A. E. Osman, and E. S. Abd Elhameed, “Low-Resource Language Processing Using Improved Deep Learning with Hunter–Prey Optimization Algorithm,” Mathematics, vol. 11, no. 21, 2023, doi: 10.3390/math11214493.
33. C. Gulere, An Introduction to Lusoga Orthography, 2nd ed. Mpolyabigere RC – RICED Center Ltd. Plot, 2012. [Online]. Available: https://shorturl.at/3lJl8
34. M. Nabirye, “Compiling the first monolingual Lusoga dictionary,” Lexikos, vol. 19, pp. 177–196, 2009, doi: 10.4314/lex.v19i1.49125.
35. M. Nabirye and G. M. De Schryver, “Digitizing the Monolingual lusoga dictionary: Challenges and prospects,” Lexikos, vol. 23, pp. 297–322, 2013, doi: 10.5788/23-1-1217.
36. G. M. De Schryver and M. Nabirye, “Corpus-driven Bantu Lexicography Part 2: Lemmatisation and rulers for Lusoga,” Lexikos, vol. 28, pp. 79–111, 2018, doi: 10.5788/28-1-1458.
37. G. M. De Schryver and M. Nabirye, “Corpus-driven Bantu Lexicography Part 3: Mapping meaning onto use in Lusoga,” Lexikos, vol. 28, pp. 112–151, 2018, doi: 10.5788/28-1-1459.
38. M. R. Marlo, M. Nabirye, and G. M. de Schryver, “Reduplication in Lusoga,” Africana Linguist., vol. 28, pp. 147–197, 2022.
39. G. M. De Schryver and M. Nabirye, “A quantitative analysis of the morphology, morphophonology and semantic import of the Lusoga noun,” Africana Linguist., vol. 16, pp. 97–153, 2010, doi: 10.3406/aflin.2010.989.
40. J. Dodge et al., “Documenting Large Webtext Corpora : A Case Study on the Colossal Clean Crawled Corpus,” 2020.
41. L. Gao, S. Biderman, and S. Black, “The Pile: An 800GB Dataset of Diverse Text for Language Modeling,” 2020.
42. N. Aspert, V. Miz, B. Ricaud, P. Vandergheynst, and I. R. Mar, “A graph-structured dataset for Wikipedia research,” 2013.
43. B. Thompson, “subs2vec : Word embeddings from subtitles in 55 languages,” Behav. Res. Methods, vol. 53, pp. 629–655, 2021.
44. S. Mesham, L. Hayward, J. Shapiro, and J. Buys, “Low-Resource Language Modelling of South African Languages,” 2021. [Online]. Available: http://arxiv.org/abs/2104.00772
45. N.-N. Joyce et al., “Applied AI Letters - 2024 - Nakatumba‐Nabende - Building Text and Speech Benchmark Datasets and Models for Low‐Resourced.pdf,” Appl. AI Lett., p. 18, 2024.
46. P. Nabende, “Towards Data-Driven Machine Translation for Lumasaaba BT - Digital Science,” in The 2018 International Conference on Digital Science, T. Antipova and A. Rocha, Eds., Cham: Springer International Publishing, 2019, pp. 3–11.
47. J. Omona and N. Groce, “Translation and research outcomes of the Bridging the Gap project: A case of the Luo language, spoken in northern Uganda,” Transl. Stud., vol. 14, no. 3, pp. 282–297, 2021, doi: 10.1080/14781700.2021.1888784.
48. E. P. Wairagala, J. Mukiibi, J. F. Tusubira, C. Babirye, and J. Nakatumba-Nabende, “Gender Bias Evaluation in Luganda-English Machine Translation,” Zenodo. [Online]. Available: https://zenodo.org/records/5864560
49. M. Mager, E. Mager, K. Kann, and N. T. Vu, “Ethical Considerations for Machine Translation of Indigenous Languages: Giving a Voice to the Speakers,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2023, pp. 4871–4897. doi: 10.18653/v1/2023.acl-long.268.
50. Z. Zhou, J. Chen, N. Wang, L. Li, and D. Wang, “Adversarial Data Augmentation for Robust Speaker Verification,” in Proceedings of the 2023 9th International Conference on Communication and Information Processing, in ICCIP ’23. New York, NY, USA: Association for Computing Machinery, 2024, pp. 226–230. doi: 10.1145/3638884.3638917.
51. E. M. Provost, “Best Practices for Noise-Based Augmentation to Improve the Performance of Deployable Speech-Based Emotion Recognition Systems,” 2023.
52. L. Pandey et al., “Towards scalable efficient on-device ASR with transfer learning,” 2024.
53. S. Cahyawijaya et al., “NusaWrites : Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages,” vol. 1, pp. 921–945, 2023.
54. T. Glushkova, C. Zerva, and A. F. T. Martins, “BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation,” in Proceedings of the 24th Annual Conference of the European Association for Machine Translation, EAMT 2023, 2023, pp. 47–58.
55. S. M. Lakew, A. Erofeeva, M. Federico, F. B. Kessler, and M. M. T. Srl, “Neural Machine Translation into Language Varieties,” in Proceedings ofthe Third Conference on Machine Translation (WMT), 2018, pp. 156–164.
56. P. Helm, G. Bella, G. Koch, and F. Giunchiglia, “Diversity and language technology : how language modeling bias causes epistemic injustice,” Ethics Inf. Technol., vol. 26, no. 1, pp. 1–15, 2024, doi: 10.1007/s10676-023-09742-6.
57. T. Kunz and T. Gummer, “Understanding Respondents’ Attitudes Toward Web Paradata Use,” Soc. Sci. Comput. Rev., vol. 38, no. 6, pp. 739–753, Feb. 2019, doi: 10.1177/0894439319826904.
58. M. Nurminen and M. Koponen, “Machine translation and fair access to information,” Transl. Spaces, vol. 9, no. 1, pp. 150–169, 2020, doi: 10.1075/ts.00025.nur.
59. T. Kocmi and O. Bojar, “Trivial Transfer Learning for Low-Resource Neural Machine Translation,” in Proceedings ofthe Third Conference on Machine Translation (WMT), 2018, pp. 244–252.
60. S. M. Lakew, A. Erofeeva, M. Negri, M. Federico, and M. Turchi, “Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary,” in International Workshop on Spoken Language Translation, 2018.
61. C.-K. Wu, C.-C. Shih, Y.-C. Wang, and R. T.-H. Tsai, “Improving low-resource machine transliteration by using 3-way transfer learning,” Comput. Speech Lang., vol. 72, p. 101283, 2022, doi: https://doi.org/10.1016/j.csl.2021.101283.
62. R. Dabre and A. Fujita, “Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation,” in Conference on Empirical Methods in Natural Language Processing, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:208163390
63. X. Han et al., “Pre-trained models : Past , present and future,” AI Open, vol. 2, no. August 2021, pp. 225–250, 2023, doi: 10.1016/j.aiopen.2021.08.002.
64. P. Gao, L. Zhang, Z. He, H. Wu, and H. Wang, “Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization,” Find. ofthe Assoc. Comput. Linguist., no. 10, pp. 12103–12119, 2023.
65. X. Chen and C. Zhang, “Language-agnostic Zero-Shot Machine Translation with Language-specific Modeling,” in 2024 International Joint Conference on Neural Networks (IJCNN), 2024, pp. 1–8. doi: 10.1109/IJCNN60899.2024.10649983.
66. B. Zheng et al., “Consistency Regularization for Cross-Lingual Fine-Tuning,” 2020.
67. N. Vieillard et al., “Leverage the Average : an Analysis of KL Regularization in Reinforcement Learning,” in 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
68. S. Gu and Y. Feng, “Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappings,” in Findings ofthe Association for Computational Linguistics, 2022, pp. 6492–6504.
69. D. Liu, J. Niehues, J. Cross, and F. Guzm, “Improving Zero-Shot Translation by Disentangling Positional Information,” in Proceedings ofthe 59th Annual Meeting ofthe Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021, pp. 1259–1273.
70. Z. Sun, Y. Liu, F. Meng, J. Xu, Y. Chen, and J. Zhou, “LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translation,” 2021.
71. R. Taye et al., “Language As A Barrier In Health Care Communication-A Comparative Study On Rural And Urban hospitals,” J. Pharm. Negat. Results ¦, vol. 14, no. February, p. 2023, 2023, doi: 10.47750/pnr.2023.14.S02.271.
72. M. Phiri, C. C. Thelma, and N. H. Mwanapabu, “The Effect of Using Local Languages as A Medium of Instruction on Academic Performance of Learners : A Case of Selected Primary Schools in Solwezi District of North- Western Province , Zambia,” Int. J. Nov. Res. Humanit. Soc. Sci., vol. 11, no. 3, 2024, doi: 10.5281/zenodo.11178057.
73. F. Thonon et al., “Electronic tools to bridge the language gap in health care for people who have migrated: Systematic review,” J. Med. Internet Res., vol. 23, no. 5, pp. 1–14, 2021, doi: 10.2196/25131.
74. H. Gao, “Research on Automatic Business English Text Translation Technology Based on Intelligent Computing,” Appl. Math. Nonlinear Sci., vol. 9, no. 1, pp. 1–15, 2024.
75. S. Mumtaz, S. P. Chandio, and D. A. K. Malokani, “The correlation between English language proficiency and perceived career opportunities. Empirical Analysis,” Remit. Rev., vol. 8, no. 4, pp. 4818–4827, 2023, doi: 10.33182/rr.v8i4.310.
76. R. Mlambo and M. Matfunjwa, “The use of technology to preserve indigenous languages of South Africa,” J. Lit. Crit. Comp. Linguist. Lit. Stud., no. Etim, pp. 1–8, 2024.
77. I. Jibreel, “Online Machine Translation Efficiency in Translating Fixed Expressions Between English and Arabic ( Proverbs as a Case-in-Point ),” Theory Pract. Lang. Stud., vol. 13, no. 5, pp. 1148–1158, 2023.
78. N. Rupčić, “Intergenerational Learning and Knowledge Transfer BT  - Managing Learning Enterprises: Challenges, Controversies and Opportunities,” N. Rupčić, Ed., Cham: Springer Nature Switzerland, 2024, pp. 201–211. doi: 10.1007/978-3-031-57704-8_13.
79. S. Nisioi, A. S. Uban, and L. P. Dinu, “Identifying Source-Language Dialects in Translation,” Mathematics, vol. 10, no. 9, 2022, doi: 10.3390/math10091431.
80. S. Luger, M. Leventhal, C. M. Homan, M. Zampieri, and M. Zampieri, “Towards a Crowdsourcing Platform for Low Resource Languages – A Semi-Supervised Approach,” in Conference on Human Computation and Crowdsourcing (HCOMP), 2020, pp. 1–3.
81. W. Nekoto et al., “Participatory Research for Low-resourced Machine Translation : A Case Study in African Languages,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 2144–2160.