Authors :
Shahidur Rahoman Sohag; Syed Murtoza Mushrul Pasha
Volume/Issue :
Volume 9 - 2024, Issue 12 - December
Google Scholar :
https://tinyurl.com/5anxuf56
Scribd :
https://tinyurl.com/mryn7ken
DOI :
https://doi.org/10.5281/zenodo.14603421
Abstract :
The extraction of causal relationships from
biomedical literature, focusing on overcoming the unique
challenges presented by the complexity of biomedical
language, implicit causalities, and the scarcity of large
annotated datasets. The research offers an extensive
review of various methods, ranging from rule-based
systems to classical machine learning models such as
SVMs, to the cutting-edge deep learning techniques
including LSTM, CNN, and BioBERT, which have
significantly improved the identification of both explicit
and implicit causal relationships. A major contribution of
this work lies in addressing the limitations posed by small
datasets through the incorporation of semi-supervised
learning and data augmentation techniques. The paper
also emphasize the importance of capturing temporal
dependencies to enhance the understanding of event
sequences, crucial for recognizing causality in biomedical
studies. Furthermore, the research underscores the
significance of domain adaptation, fine-tuning general-
purpose datasets like SemEval for the specific needs of
biomedical literature, which often contains domain-
specific terms and complex structures. By tackling these
challenges and proposing innovative solutions, this paper
advances the field of biomedical text mining, offering
valuable insights for future research and practical
applications in clinical decision support, drug safety
monitoring, and biomedical knowledge discovery.
Keywords :
Biomedical Causal Discovery; Biomedical Causality Mining; Causal Relationship Mining; Biomedical Text Analysis; NLP in Biomedical Research.
References :
- W. Ali, W. Zuo, R. Ali, X. Zuo, and G. Rahman, “Causality mining in natural languages using machine and deep learning techniques: A survey,” Applied Sciences, vol. 11, no. 21, 2021. [Online]. Available: https://www. mdpi.com/2076-3417/11/21/10064
- S. R. Sohag, S. Zhang, M. Xian, S. Sun, F. Xu, and Z. Ma, “Causality extraction from nuclear licensee event reports using a hybrid framework,” 2024. [Online]. Available: https://arxiv.org/abs/2404.05656
- N. Asghar, “Automatic extraction of causal relations from natural language texts: A comprehensive survey,” 2016. [Online]. Available: https://arxiv.org/abs/1605.07895
- K. Singh, G. Gupta, L. Vig, G. Shroff, and P. Agarwal, “Deep convolutional neural networks for pairwise causality,” 2017. [Online]. Available: https://arxiv.org/abs/1701.00597
- Yin and F. Corradi, “Never reset again: A mathematical framework for continual inference in recurrent neural networks,” 2024. [Online]. Available: https://arxiv.org/abs/ 2412.15983
- Graves and A. Graves, “Long short-term memory,” Supervised sequence labelling with recurrent neural networks, pp. 37–45, 2012.
- O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, “Convolutional neural networks for speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 22, no. 10, pp. 1533–1545, 2014.
- Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 2016. [Online]. Available: https://arxiv.org/abs/1409.0473
- J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
- M. Jullien, M. Valentino, and A. Freitas, “Semeval-2024 task 2: Safe biomedical natural language inference for clinical trials,” arXiv preprint arXiv:2404.04963, 2024.
- W. Ali, W. Zuo, R. Ali, G. Rahman, X. Zuo, and I. Ullah, “Towards improving causality mining using bert with multi-level feature networks.” KSII Transactions on Internet & Information Systems, vol. 16, no. 10, 2022.
- J. Berant, V. Srikumar, P.-C. Chen, A. Vander Linden,B. Harding, B. Huang, P. Clark, and C. D. Manning, “Modeling biological processes for reading comprehension,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1499– 1510.
- T. Caselli and P. Vossen, “The event storyline corpus: A new benchmark for causal and temporal relation extraction,” in Proceedings of the Events and Stories in the News Workshop, 2017, pp. 77–86.
- Hashimoto, K. Torisawa, J. Kloetzer, M. Sano, I. Varga, J.-H. Oh, and Y. Kidawara, “Toward future scenario generation: Extracting event causality exploiting semantic relation, context, and association features,” in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2014, pp. 987–997.
- V. Khetan, M. I. H. Rizvi, J. Huber, P. Bartusiak, B. Sacaleanu, and A. Fano, “Mimicause: Representation and automatic extraction of causal relation types from clinical notes,” arXiv preprint arXiv:2110.07090, 2021.
- Sorgente, G. Vettigli, and F. Mele, “Automatic extraction of cause-effect relations in natural language text.” DART@ AI* IA, vol. 2013, pp. 37–48, 2013.
- T. N. De Silva, X. Zhibo, Z. Rui, and M. Kezhi, “Causal relation identification using convolutional neural networks and knowledge based features,” International Journal of Computer and Systems Engineering, vol. 11, no. 6, pp. 696–701, 2017.
- S. Zhao, T. Liu, S. Zhao, Y. Chen, and J.-Y. Nie, “Event causality extraction based on connectives analysis,” Neuro-computing, vol. 173, pp. 1943–1950, 2016.
- M. Riaz and R. Girju, “Recognizing causality in verb-noun pairs via noun and verb semantics,” in Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), 2014, pp. 48–57.
- H. Kayesh, M. S. Islam, J. Wang, A. Kayes, and P. A. Watters, “A deep learning model for mining and detecting causally related events in tweets,” Concurrency and Computation: Practice and Experience, vol. 34, no. 2, p. e5938, 2022.
- Z. Li, X. Ding, T. Liu, J. E. Hu, and B. Van Durme, “Guided generation of cause and effect,” arXiv preprint arXiv:2107.09846, 2021.
- R. Girju, D. I. Moldovan et al., “Text mining for causal relations.” in FLAIRS, vol. 2, 2002, pp. 360–364.
- R. Girju, P. Nakov, V. Nastase, S. Szpakowicz, P. Turney, and D. Yuret, “Classification of semantic relations between nominals,” Language Resources and Evaluation, vol. 43, pp. 105–121, 2009.
- Li, M. Zhang, G. Fu, and D. Ji, “A neural joint model for entity and relation extraction from biomedical text,” BMC bioinformatics, vol. 18, pp. 1–11, 2017.
- Beltagy, K. Lo, and A. Cohan, “Scibert: A pre-trained language model for scientific text,” arXiv preprint arXiv:1903.10676, 2019.
- J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
- T. Ruan and S. Zhang, “Towards understanding how attention mechanism works in deep learning,” 2024. [Online]. Available: https://arxiv.org/abs/2412.18288
- T. Dasgupta, R. Saha, L. Dey, and A. Naskar, “Automatic extraction of causal relations from text using linguistically informed deep neural networks,” in Proceedings of the 19th annual SIGdial meeting on discourse and dialogue, 2018, pp. 306–316.
- N. Kyriacou, P. Greenland, and M. A. Mansournia, “Using causal diagrams for biomedical research,” Annals of emergency medicine, vol. 81, no. 5, pp. 606–613, 2023.
- S. Hochreiter, “Long short-term memory,” Neural Computation MIT-Press, 1997.
- Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017.
- Y. Chen, W. Wan, J. Hu, Y. Wang, and B. Huang, “Complex causal extraction of fusion of entity location sensing and graph attention networks,” Information, vol. 13, no. 8, p. 364, 2022.
- L. Deng, “A tutorial survey of architectures, algorithms, and applications for deep learning,” APSIPA transactions on Signal and Information Processing, vol. 3, p. e2, 2014.
- S. C. Fanni, M. Febi, G. Aghakhanyan, and E. Neri, “Natural language processing,” in Introduction to Artificial Intelligence. Springer, 2023, pp. 87–99.
- S. Pyysalo, F. Ginter, J. Heimonen, J. Bjo¨rne, J. Boberg, J. Ja¨rvinen, and T. Salakoski, “Bioinfer: a corpus for information extraction in the biomedical domain,” BMC bioinformatics, vol. 8, pp. 1–24, 2007.
- Fan, W. Fan, C. Smith et al., “Adverse drug event detection and extraction from open data: A deep learning approach,” Information Processing & Management, vol. 57, no. 1, p. 102131, 2020.
- H. Gurulingappa, A. M. Rajput, A. Roberts, J. Fluck, M. Hofmann-Apitius, and L. Toldo, “Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports,” Journal of biomedical informatics, vol. 45, no. 5, pp. 885– 892, 2012.
- Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. O. Se´aghdha, S. Pado´, M. Pennacchiotti, L. Romano, and S. Szpakowicz, “Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals,” arXiv preprint arXiv:1911.10422, 2019.
- Webber, R. Prasad, and A. Joshi, “Reflections on the penn discourse treebank and its relatives,” Computational Linguistics, 2024.
- R. Prasad, E. Miltsakaki, N. Dinesh, A. Lee, A. Joshi, L. Robaldo, and B. Webber, “The penn discourse treebank 2.0 annotation manual,” December, vol. 17, p. 2007, 2007.
- J.-D. Kim, T. Ohta, S. Pyysalo, Y. Kano, and J. Tsujii, “Overview of bionlp’09 shared task on event extraction,” in Proceedings of the BioNLP 2009 workshop companion volume for shared task, 2009, pp. 1–9.
- P. Mirza, R. Sprugnoli, S. Tonelli, and M. Speranza, “Annotating causality in the tempeval-3 corpus,” in Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), 2014, pp. 10–19.
- M. Krallinger, O. Rabal, S. Akhondi, M. Perez, J. Santamaria, G. Rodr´ıguez, G. Tsatsaronis, A. Intxaurrondo, J. Lo´pez, U. Nandal et al., “Overview of the biocreative vi chemical-protein interaction track. semantic scholar,” 2021.
- T. Ohta, Y. Tateisi, J.-D. Kim, H. Mima, and J. Tsujii, “The genia corpus: An annotated research abstract corpus in molecular biology domain,” in Proceedings of the human language technology conference. Citeseer, 2002, pp. 73–77.
- Christopoulou, M. Miwa, and S. Ananiadou, “Connecting the dots: Document-level neural relation extraction with edge-oriented graphs,” arXiv preprint arXiv:1909.00228, 2019.
- Pisner and D. M. Schnyer, “Support vector machine,” in Machine learning. Elsevier, 2020, pp. 101–121.
- S. Selva Birunda and R. Kanniga Devi, “A review on word embedding techniques for text classification,” Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, pp. 267–281, 2021.
- H. Sarker, H. Janicke, M. A. Ferrag, and A. Abuadbba, “Multi-aspect rule-based ai: Methods, taxonomy, challenges and directions toward automation, intelligence and transparent cybersecurity modeling for critical infrastructures,” Internet of Things, p. 101110, 2024.
- Pisner and D. M. Schnyer, “Support vector machine,” in Machine learning. Elsevier, 2020, pp. 101–121.
- S. Selva Birunda and R. Kanniga Devi, “A review on word embedding techniques for text classification,” Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, pp. 267–281, 2021.
- H. Sarker, H. Janicke, M. A. Ferrag, and A. Abuadbba, “Multi-aspect rule-based ai: Methods, taxonomy, challenges and directions toward automation, intelligence and transparent cybersecurity modeling for critical infrastructures,” Internet of Things, p. 101110, 2024.
- Christopoulou, M. Miwa, and S. Ananiadou, “A walk-based model on entity graphs for relation extraction,” arXiv preprint arXiv:1902.07023, 2019.
- Barik, E. Marsi, and P. Øzturk, “Event causality extraction from natural science literature,” 2016.
- D.-S. Chang and K.-S. Choi, “Incremental cue phrase learning and bootstrapping method for causality extraction using cue phrase and word pair probabilities,” Information processing & management, vol. 42, no. 3, pp. 662–678, 2006.
- Airola, S. Pyysalo, J. Bjo¨rne, T. Pahikkala, F. Ginter, and T. Salakoski, “All-paths graph kernel for protein-protein interaction extraction with evaluation of cross-corpus learning,” BMC bioinformatics, vol. 9, pp. 1–12, 2008.
- Balashankar, S. Chakraborty, S. Fraiberger, and L. Subramanian, “Identifying predictive causal factors from news streams,” in Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp. 2338–2348.
- S. Bethard and J. H. Martin, “Learning semantic links from a corpus of parallel temporal and causal relations,” in Proceedings of ACL-08: HLT, Short Papers, 2008, pp. 177– 180.
- Blanco, N. Castell, and D. I. Moldovan, “Causal relation extraction.” in Lrec, vol. 66, 2008, p. 74.
- N. Panday, D. Sigdel, I. Adam, J. Ramirez, A. Verma, A. N. Eranki, W. Wang, D. Wang, and P. Ping, “Data-driven insights into the association between oxidative stress and calcium-regulating proteins in cardiovascular disease,” Antioxidants, vol. 13, no. 11, p. 1420, 2024.
- Q.-C. Bui, B. O´ . Nualla´in, C. A. Boucher, and P. M. Sloot, “Extracting causal relations on hiv drug resistance from literature,” BMC bioinformatics, vol. 11, pp. 1–11, 2010.
- Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” arXiv preprint arXiv:1903.10676, 2019.
- L. Yao, C. Mao, and Y. Luo, “Graph convolutional networks for text classification,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 7370–737.
The extraction of causal relationships from
biomedical literature, focusing on overcoming the unique
challenges presented by the complexity of biomedical
language, implicit causalities, and the scarcity of large
annotated datasets. The research offers an extensive
review of various methods, ranging from rule-based
systems to classical machine learning models such as
SVMs, to the cutting-edge deep learning techniques
including LSTM, CNN, and BioBERT, which have
significantly improved the identification of both explicit
and implicit causal relationships. A major contribution of
this work lies in addressing the limitations posed by small
datasets through the incorporation of semi-supervised
learning and data augmentation techniques. The paper
also emphasize the importance of capturing temporal
dependencies to enhance the understanding of event
sequences, crucial for recognizing causality in biomedical
studies. Furthermore, the research underscores the
significance of domain adaptation, fine-tuning general-
purpose datasets like SemEval for the specific needs of
biomedical literature, which often contains domain-
specific terms and complex structures. By tackling these
challenges and proposing innovative solutions, this paper
advances the field of biomedical text mining, offering
valuable insights for future research and practical
applications in clinical decision support, drug safety
monitoring, and biomedical knowledge discovery.
Keywords :
Biomedical Causal Discovery; Biomedical Causality Mining; Causal Relationship Mining; Biomedical Text Analysis; NLP in Biomedical Research.