Research PaperNo Access

Evolving a Pipeline Approach for Abstract Meaning Representation Parsing Towards Dynamic Neural Networks

Florin Macicasan

Knowledge Engineering Research Group, Technical University of Cluj-Napoca, Cluj-Napoca 400027, Romania

E-mail Address: florin.macicasan@cs.utcluj.ro

Corresponding author.

Search for more papers by this author

Alexandru Frasie

Knowledge Engineering Research Group, Technical University of Cluj-Napoca, Cluj-Napoca 400027, Romania

Search for more papers by this author

Nicoleta-Teodora Vezan

Knowledge Engineering Research Group, Technical University of Cluj-Napoca, Cluj-Napoca 400027, Romania

Search for more papers by this author

Camelia Lemnaru

Knowledge Engineering Research Group, Technical University of Cluj-Napoca, Cluj-Napoca 400027, Romania

E-mail Address: camelia.lemnaru@cs.utcluj.ro

Corresponding author.

Search for more papers by this author

, and

Rodica Potolea

Knowledge Engineering Research Group, Technical University of Cluj-Napoca, Cluj-Napoca 400027, Romania

Search for more papers by this author

https://doi.org/10.1142/S0129065723500405Cited by:1 (Source: Crossref)

This article is part of the issue:

Special Issue on Neural Networks and Machine Learning
Guest Editor: Lazaros Iliadis

Abstract

Abstract Meaning Representation parsing aims to represent a sentence as a structured, Directed, Acyclic Graph (DAG), in an attempt to extract meaning from text. This paper extends an existing 2-stage pipeline AMR parser with state-of-the-art techniques in dependency parsing. First, Pointer-Generator Networks are used for out-of-vocabulary words in the concept identification stage, with an improved initialization via the use of word-and character-level embeddings. Second, the performance of the Relation Identification module is improved by jointly training the Heads Selection and the Arcs Labeling components. Last, we underline the difficulty of end-to-end training with recurrent modules in a static deep neural network construction approach and explore a dynamic construction implementation, which continuously adapts the computation graph, thus potentially enabling end-to-end training in the proposed pipeline solution.

Keywords:

References

1. Meaning and machine intelligence: What are you talking about?, The Economist Technological Quarterly: Finding a Voice (2017). Google Scholar
2. The world’s most valuable resource is no longer oil, but data, The Economist Leaders: Regulating the Internet Giants (2017). Google Scholar
3. L. Banarescu, C. Bonial, S. Cai, M. Georgescu, K. Griffitt, U. Hermjakob, K. Knight, M. Palmer and N. Schneider , Abstract meaning representation for sembanking, in 7th Linguistic Annotation Workshop and Interoperability with Discourse (Association for Computational Linguistics, 2013), pp. 178–186. Google Scholar
4. R. Pop, F. Macicasan and C. Lemnaru , A two stage approach for amr parsing using the concept inference order, in 2020 IEEE 16th Int. Conf. Intelligent Computer Communication and Processing, 2020, pp. 91–98. Crossref, Google Scholar
5. A. Frasie, N.-T. Vezan, G. Marian, F. Macicasan and C. Lemnaru , Enhancements on a pipeline approach for abstract meaning representation parsing, in Engineering Applications of Neural Networks, eds. L. Iliadis, C. Jayne, A. Tefas and E. Pimenidis (Springer International Publishing, Cham, 2022), pp. 505–516. Crossref, Google Scholar
6. G. Neubig, C. Dyer, Y. Goldberg, A. Matthews, W. Ammar, A. Anastasopoulos, M. Ballesteros, D. Chiang, D. Clothiaux, T. Cohn, K. Duh, M. Faruqui, C. Gan, D. Garrette, Y. Ji, L. Kong, A. Kuncoro, G. Kumar, C. Malaviya, P. Michel, Y. Oda, M. Richardson, N. Saphra, S. Swayamdipta and P. Yin, Dynet: The dynamic neural network toolkit, arXiv: 1701.03980. Google Scholar
7. O. B. Batiz, R. P. Helmer, R. Pop, F. Macicasan and C. Lemnaru , Concept identification with sequence-to-sequence models in abstract meaning representation parsing, in 2020 IEEE 16th Int. Conf. Intelligent Computer Communication and Processing, 2020, pp. 83–90. Crossref, Google Scholar
8. R. McDonald and J. Nivre, Recent advances in dependency parsing Tutorial at EACL (2014). Google Scholar
9. S. Cimpian, A. Lazar, F. Macicasan and C. Lemnaru , A transition-based approach for amr parsing using lstm networks, in 13th IEEE Int. Conf. Intelligent Computer Communication and Processing, 2017, pp. 103–110. Crossref, Google Scholar
10. J. Flanigan, S. Thomson, J. G. Carbonell, C. Dyer and N. A. Smith , A discriminative graph-based parser for the abstract meaning representation, in Proc. ACL 2014, June 22–27, Baltimore, USA, pp. 1426–1436. Google Scholar
11. F. Zamora-Martinez and M. J. Castro-Bleda , Efficient embedded decoding of neural network language models in a machine translation system, Int. J. Neural Syst. 28(9) (2018) 1850007. Link, Web of Science, Google Scholar
12. P. Jiménez and R. Corchuelo , An experimental study of neural approaches to multi-hop inference in question answering, Int. J. Neural Syst. 32(4) (2022) 2250011. Link, Web of Science, Google Scholar
13. C. Lyu and I. Titov, AMR parsing as graph prediction with latent alignment, in Proc. 56th Annual Meeting of the ACL (Volume 1: Long Papers) (Association for Computational Linguistics, Melbourne, Australia, 2018), pp. 397–407. Google Scholar
14. S. Zhang, X. Ma, K. Duh and B. Van Durme , AMR parsing as sequence-to-graph transduction, in Proc. 57th Annual Meeting of the ACL (Association for Computational Linguistics, Florence, Italy, 2019), pp. 80–94. Crossref, Google Scholar
15. D. Cai and W. Lam , AMR parsing via graph-sequence iterative inference, in Proc. 58th Annual Meeting of the ACL (Association for Computational Linguistics, Online, 2020), pp. 1290–1301. Crossref, Google Scholar
16. C. Wang, N. Xue and S. Pradhan , A transition-based algorithm for AMR parsing, in Proc. 2015 Conference of the North American Chapter of the ACL: Human Language Technologies (Association for Computational Linguistics, Denver, Colorado, May–June 2015), pp. 366–375. Google Scholar
17. J. Zhou, T. Naseem, R. Fernandez Astudillo and R. Florian , AMR parsing with action-pointer transformer, in Proc. 2021 Conf. North American Chapter of the ACL: Human Language Technologies (Association for Computational Linguistics, Online, 2021), pp. 5585–5598. Crossref, Google Scholar
18. R. Fernandez Astudillo, M. Ballesteros, T. Naseem, A. Blodgett and R. Florian , Transition-based parsing with stack-transformers, Findings of the ACL: EMNLP 2020 (Association for Computational Linguistics, Online, 2020), pp. 1001–1007. Crossref, Google Scholar
19. A. Drozdov, J. Zhou, R. Florian, A. McCallum, T. Naseem, Y. Kim and R. Astudillo , Inducing and using alignments for transition-based AMR parsing, in Proc. 2022 Conf. North American Chapter of the ACL: Human Language Technologies (Association for Computational Linguistics, Seattle, United States, 2022), pp. 1086–1098. Crossref, Google Scholar
20. J. Zhou, T. Naseem, R. Fernandez Astudillo, Y.-S. Lee, R. Florian and S. Roukos , Structure-aware fine-tuning of sequence-to-sequence transformers for transition-based AMR parsing, in Proc. 2021 Conf. Empirical Methods in NLP (Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021), pp. 6279–6290. Crossref, Google Scholar
21. I. Konstas, S. Iyer, M. Yatskar, Y. Choi and L. Zettlemoyer, Neural AMR: Sequence-to-sequence models for parsing and generation, in Proc. 55th Annual Meeting of the ACL (Volume 1: Long Papers) (Association for Computational Linguistics, Vancouver, Canada, 2017), pp. 146–157. Google Scholar
22. D. Xu, J. Li, M. Zhu, M. Zhang and G. Zhou , Improving AMR parsing with sequence-to-sequence pre-training, in Proc. 2020 Conf. Empirical Methods in NLP (EMNLP) (Association for Computational Linguistics, Online, 2020), pp. 2501–2511. Crossref, Google Scholar
23. C. Yu and D. Gildea, Sequence-to-sequence AMR parsing with ancestor information, in Proc. 60th Annual Meeting of the ACL (Volume 2: Short Papers) (Association for Computational Linguistics, Dublin, Ireland, 2022), pp. 571–577. Google Scholar
24. K.-H. Huang, V. Iyer, A. Kumar, S. Venkatapathy, K.-W. Chang and A. Galstyan , Unsupervised syntactically controlled paraphrase generation with abstract meaning representations, Findings of the ACL: EMNLP 2022 (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022), pp. 1547–1554. Crossref, Google Scholar
25. A. Jangra, P. Nema and A. Raghuveer , T-STAR: Truthful style transfer using AMR graph as intermediate representation, in Proc. 2022 Conf. Empirical Methods in NLP (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022), pp. 8805–8825. Crossref, Google Scholar
26. M. Inácio and T. Pardo , Semantic-based opinion summarization, in Proc. Int. Conf. Recent Advances in NLP (RANLP 2021) (INCOMA, Held Online, 2021), pp. 619–628. Crossref, Google Scholar
27. C. N. Bonial, L. Donatelli, J. Ervin and C. R. Voss , Abstract meaning representation for human-robot dialogue, in Proc. Society for Computation in Linguistics, 2019, pp. 236–246. Crossref, Google Scholar
28. N. Schrack, R. Cui, H. López and D. Hershcovich , Can AMR assist legal and logical reasoning? Findings of the ACL: EMNLP 2022 (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022), pp. 1555–1568. Crossref, Google Scholar
29. D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv:1409.0473. Google Scholar
30. A. See, P. J. Liu and C. D. Manning , Get to the point: Summarization with pointer-generator networks, in Proc. 55th Annual Meeting of the ACL, Vol. 1 (2017) 1073–1083. Crossref, Google Scholar
31. J. Pennington, R. Socher and C. Manning , GloVe: Global vectors for word representation, in Proc. 2014 Conf. Empirical Methods in NLP (Association for Computational Linguistics, Doha, Qatar, 2014), pp. 1532–1543. Crossref, Google Scholar
32. R. van Noord, A. Toral and J. Bos , Character-level representations improve DRS-based semantic parsing even in the age of bert, in Proc. 2020 Conf. Empirical Methods in NLP (2020), 4587–4603. Crossref, Google Scholar
33. X. Zhang, J. Cheng and M. Lapata, Dependency parsing as head selection, in Proc. 15th Conf. European Chapter of the ACL: Volume 1, Long Papers (Association for Computational Linguistics, Valencia, Spain, 2017), pp. 665–676. Google Scholar
34. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai and S. Chintala , Pytorch: An imperative style, high-performance deep learning library, Adv. Neural Inf. Process. Syst. 32 (2019) 8024–8035. Google Scholar
35. J. May , Semeval-2016 task 8: Meaning representation parsing, in Proc. 10th Int. Workshop Semantic Evaluation (SemEval-2016) (Association for Computational Linguistics, San Diego, California, 2016), pp. 1063–1073. Crossref, Google Scholar
36. S. Cai and K. Knight, Smatch: An evaluation metric for semantic feature structures, in Proc. 51st Annual Meeting of the ACL (Volume 2: Short Papers) (Association for Computational Linguistics, Sofia, Bulgaria, 2013), pp. 748–752. Google Scholar