Recently, graph convolutional networks (GCNs) for text classification have received considerable attention in natural language processing. However, most current methods just use original documents and words in the corpus to construct the topology of graph which may lose some effective information. In this paper, we propose a Multi-Stream Graph Convolutional Network (MS-GCN) for text classification via Representative-Word Document (RWD) mining, which is implemented in PyTorch. In the proposed method, we first introduce temporary labels and mine the RWDs which are treated as additional documents in the corpus. Then, we build a heterogeneous graph based on relations among a Group of RWDs (GRWDs), words and original documents. Furthermore, we construct the MS-GCN based on multiple heterogeneous graphs according to different GRWDs. Finally, we optimize our MS-GCN model through updated mechanism of GRWDs. We evaluate the proposed approach on six text classification datasets, 20NG, R8, R52, Ohsumed, MR and Pheme. Extensive experiments on these datasets show that our proposed approach outperforms state-of-the-art methods for text classification.

Keywords:

Remember to check out the Most Cited Articles!
Check out these titles in artificial intelligence!

References

1. Y. Kim, Convolutional neural networks for sentence classification, Master’s thesis, University of Waterloo (2014). Google Scholar
2. S. Lai, L. Xu, K. Liu and J. Zhao, Recurrent convolutional neural networks for text classification, in Proc. 29th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2015), pp. 2267–2273. Crossref, Google Scholar
3. T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks (2016), arXiv:1609.02907. Google Scholar
4. L. Yao, C. Mao and Y. Luo, Graph convolutional networks for text classification, in 33rd AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2019), pp. 7370–7377. Crossref, Google Scholar
5. X. Liu, X. You, X. Zhang, J. Wu and P. Lv, Tensor graph convolutional networks for text classification, in 34th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2020), pp. 8409–8416. Crossref, Google Scholar
6. C. Tan, Y. Ren and C. Wang, An adaptive convolution with label embedding for text classification, Appl. Intell. (2022), https://doi.org/10.1007/s10489-021-02702-x. Google Scholar
7. S. Zeng, Y. Ma, X. Zhang and X. Du, Term-based pooling in convolutional neural networks for text classification, China Commun. 17 (2020) 109–124. Crossref, Google Scholar
8. P. Liu, X. Qiu and X. Huang, Recurrent neural network for text classification with multi-task learning (2016), arXiv:1605.05101. Google Scholar
9. B. Behera and G. Kumaravelan, Text document classification using fuzzy rough set based on robust nearest neighbor (FRS-RNN), Soft Comput. 25 (2021) 9915–9923. Crossref, Google Scholar
10. J. Xie, Y. Hou, Y. Wang, Q. Wang, B. Li, S. Lv and Y. I. Vorotnitsky, Chinese text classification based on attention mechanism and feature-enhanced fusion neural network, Computing 102 (2020) 683–700. Crossref, Google Scholar
11. A. Joulin, E. Grave, P. Bojanowski and T. Mikolov, Bag of tricks for efficient text classification (2016), arXiv:1607.01759. Google Scholar
12. R. Johnson and T. Zhang, Deep pyramid convolutional neural networks for text categorization, in Proc. 55th Annual Meeting of the Association for Computational Linguistics (ACL, Stroudsburg, 2017), pp. 562–570. Crossref, Google Scholar
13. S. Wang, M. Huang and Z. Deng, Densely connected CNN with multi-scale feature attention for text classification, in Proc. Twenty-Seventh Int. Joint Conf. Artificial Intelligence, Int. Joint Conf. Artificial Intelligence, Stockholm, Sweden, 13–19 July, 2018. Crossref, Google Scholar
14. X. Li, Z. Li, H. Xie and Q. Li, Merging statistical feature via adaptive gate for improved text classification, in 35th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2021), pp. 13288–13296. Crossref, Google Scholar
15. J.-H. Lee, S.-K. Ko and Y.-S. Han, Salnet: Semi-supervised few-shot text classification with attention-based lexicon construction, in 35th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2021), pp. 13189–13197. Crossref, Google Scholar
16. J. Du, Y. Huang and K. Moilanen, Knowledge-aware leap-LSTM: Integrating prior knowledge into leap-LSTM towards faster long text classification, in 35th AAAI Conf. Artificial Intelligence (AAAI, Palo Alto, 2021), pp. 12768–12775. Crossref, Google Scholar
17. P. Li, P. Zhong, K. Mao, D. Wang, X. Yang, Y. Liu, J. Yin and S. See, Act: An attentive convolutional transformer for efficient text classification, in 35th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2021), pp. 13261–13269. Crossref, Google Scholar
18. H. Ding, Z. Wu and L. Zhao, Whale optimization algorithm based on nonlinear convergence factor and chaotic inertial weight, Concurr. Comput. Pract. Exp. 32 (2020) e5949. Crossref, Google Scholar
19. J. Bastings, I. Titov, W. Aziz, D. Marcheggiani and K. Sima’an, Graph convolutional encoders for syntax-aware neural machine translation (2017), arXiv:1704.04675. Google Scholar
20. Q. Li, Z. Han and X.-M. Wu, Deeper insights into graph convolutional networks for semi-supervised learning, in 32nd AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2018), pp. 3538–3545. Crossref, Google Scholar
21. M. Tu, G. Wang, J. Huang, Y. Tang, X. He and B. Zhou, Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs, in 57th Annual Meeting of the Association-for-Computational-Linguistics (ACL Stroudsburg, 2019), pp. 2704–2713. Crossref, Google Scholar
22. T. Yang, L. Hu, C. Shi, H. Ji, X. Li and L. Nie, HGAT: Heterogeneous graph attention networks for semi-supervised short text classification, ACM Trans. Inf. Syst. 39 (2021) 1–29. Crossref, Google Scholar
23. Y. Zhang, X. Yu, Z. Cui, S. Wu, Z. Wen and L. Wang, Every document owns its structure: Inductive text classification via graph neural networks, in 58th Annual Meeting of the Association-for-Computational-Linguistics (ACL, Stroudsburg, 2020), pp. 334–339. Crossref, Google Scholar
24. K. Ding, J. Wang, J. Li, D. Li and H. Liu, Be more with less: Hypergraph attention networks for inductive text classification (2020), arXiv:2011.00387. Google Scholar
25. Y. Wu, J. Li, Y. Yuan, A. Qin, Q.-G. Miao and M.-G. Gong, Commonality autoencoder: Learning common features for change detection from heterogeneous images, IEEE Trans. Neural Netw. Learn. Syst. 33 (2021) 4257–4270. Crossref, Google Scholar
26. K. Church and P. Hanks, Word association norms, mutual information, and lexicography, Comput. Linguist. 16 (1990) 22–29. Google Scholar
27. Q. Le and T. Mikolov, Distributed representations of sentences and documents, in Int. Conf. Machine Learning, Vol. 32 (PMLR, 2014), pp. 1188–1196. Google Scholar
28. J. Tang, M. Qu and Q. Mei, PTE: Predictive text embedding through large-scale heterogeneous text networks, in Proc. 21th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (ACM, New York, 2015), pp. 1165–1174. Crossref, Google Scholar
29. D. Shen, G. Wang, W. Wang, M. R. Min, Q. Su, Y. Zhang, C. Li, R. Henao and L. Carin, Baseline needs more love: On simple word-embedding-based models and associated pooling mechanisms, in Proc. 56th Annual Meeting of the Association-for-Computational-Linguistics (ACL, Stroudsburg, 2018), pp. 440–450. Crossref, Google Scholar
30. G. Wang, C. Li, W. Wang, Y. Zhang, D. Shen, X. Zhang, R. Henao and L. Carin, Joint embedding of words and labels for text classification, in Proc. 56th Annual Meeting of the Association-for-Computational-Linguistics (ACL, Stroudsburg, 2018), pp. 2321–2331. Crossref, Google Scholar
31. G. Liu and J. Guo, Bidirectional LSTM with attention mechanism and convolutional layer for text classification, Neurocomputing 337 (2019) 325–338. Crossref, Google Scholar
32. M. Defferrard, X. Bresson and P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral filtering, Adv. Neural Inf. Process. Syst. 29 (2016) 3844–3852. Google Scholar
33. J. Bruna, W. Zaremba, A. Szlam and Y. LeCun, Spectral networks and locally connected networks on graphs (2013), arXiv:1312.6203. Google Scholar
34. M. Henaff, J. Bruna and Y. LeCun, Deep convolutional networks on graph-structured data (2015), arXiv:1506.05163. Google Scholar
35. J. Ma, W. Gao and K. F. Wong, Rumor detection on twitter with tree-structured recursive neural networks, in Association for Computational Linguistics (2018), pp. 1980–1989. Crossref, Google Scholar
36. J. Ma, W. Gao and K. F. Wong, Detect rumors on twitter by promoting information campaigns with generative adversarial learning, in The World Wide Web Conf. (2019), pp. 3049–3055. Crossref, Google Scholar
37. T. Bian, X. Xiao, T. Xu, P. Zhao, W. Huang, Y. Rong and J. Huang, Rumor detection on social media with bi-directional graph convolutional networks, in 34th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, 2020), pp. 549–556. Crossref, Google Scholar
38. Y. J. Lu and C. T. Li, GCAN: Graph-aware co-attention networks for explainable fake news detection on social media, in Proc. 58th Annual Meeting of the Association for Computational Linguistics (ACL, Stroudsburg, 2020), pp. 505–514. Crossref, Google Scholar
39. Y. Z. Song, Y. S. Chen, Y. T. Chang, S. Y. Weng and H. H. Shuai, Adversary-aware rumor detection, in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (ACL, Stroudsburg, 2021), pp. 1371–1382. Crossref, Google Scholar
40. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization (2014), arXiv:1412.6980. Google Scholar