Machine LearningNo Access

MLCE: A Multi-Label Crotch Ensemble Method for Multi-Label Classification

Yuan Yao

Department of Computer Science, Harbin Institute of Technology, Shenzhen 518055, P. R. China

E-mail Address: yaoyuan@stu.hit.edu.cn

Search for more papers by this author

Yan Li

School of Computer Engineering, Shenzhen Polytechnic, Shenzhen 518055, P. R. China

E-mail Address: liyan@szpt.edu.cn

Corresponding author.

Search for more papers by this author

Yunming Ye

Department of Computer Science, Harbin Institute of Technology, Shenzhen 518055, P. R. China

E-mail Address: yeyunming@hit.edu.cn

Search for more papers by this author

, and

Xutao Li

Department of Computer Science, Harbin Institute of Technology, Shenzhen 518055, P. R. China

E-mail Address: lixutao@hit.edu.cn

Search for more papers by this author

https://doi.org/10.1142/S021800142151006XCited by:4 (Source: Crossref)

Abstract

Multi-label classification addresses the problem that each instance is associated with multiple labels simultaneously. In this paper, we propose a multi-label crotch ensemble (MLCE) model for multi-label classification, which takes label correlations into consideration. In MLCE, a multi-label cluster tree is first constructed. Then, we incorporate all multi-label crotch predictors of the tree into a classifier, where the multi-label crotch predictor is the crotch formed by an inner node of the tree and its children. Finally, a flexible weighted voting scheme is designed to produce the classification output. We perform experiments on 11 benchmark datasets. Experimental results clearly demonstrate the MLCE significantly outperforms six well-established multi-label classification approaches, in terms of the widely used evaluation metrics.

Keywords:

References

1. R. Cabral, F. D. l. Torre, J. P. Costeira and A. Bernardino, Matrix completion for weakly-supervised multi-label image classification, TPAMI 37(1) (2015) 121–135. Crossref, Web of Science, Google Scholar
2. T. Chen, M. Xu, X. Hui, H. Wu and L. Lin, Learning semantic-specific graph representation for multi-label image recognition, in ICCV (2019). Crossref, Google Scholar
3. W. Cheng and E. Hüllermeier, Combining instance-based learning and logistic regression for multilabel classification, Mach. Learn. 76(2) (2009) 211–225. Crossref, Web of Science, Google Scholar
4. A. Clare and R. D. King, Knowledge discovery in multi-label phenotype data, in PKDD (2001) pp. 42–53. Crossref, Google Scholar
5. K. Dembczyński, W. Waegeman, W. Cheng and E. Hüllermeier, On label dependence and loss minimization in multi-label classification, Mach. Learn. 88(1) (2012) 5–45. Crossref, Web of Science, Google Scholar
6. J. Demšar, Statistical comparisons of classifiers over multiple data sets, JMLR 7 (2006) 1–30. Google Scholar
7. H. Elghazel, A. Aussem, O. Gharroudi and W. Saadaoui, Ensemble multi-label text categorization based on rotation forest and latent semantic indexing, Expert Syst. Appl. 57 (2016) 1–11. Crossref, Web of Science, Google Scholar
8. A. Elisseeff and J. Weston, A kernel method for multi-labelled classification, in NeurIPS (2001) pp. 681–687. Google Scholar
9. J. Fürnkranz, E. Hüllermeier, E. LozaMencía and K. Brinker, Multilabel classification via calibrated label ranking, Mach. Learn. 73(2) (2008) 133–153. Crossref, Web of Science, Google Scholar
10. T. Gonçalves and P. Quaresma, A preliminary approach to the multilabel classification problem of portuguese juridical documents, in EPIA (2003), pp. 435–444. Crossref, Google Scholar
11. C. Li, C. Liu, L. Duan, P. Gao and K. Zheng, Reconstruction regularized deep metric learning for multi-label image classification, TNNLS (2019) 1–10. Google Scholar
12. P. Li, H. Li and M. Wu, Multi-label ensemble based on variable pairwise constraint projection, Inform. Sci. 222 (2013) 269–281. Crossref, Web of Science, Google Scholar
13. Q. Li, M. Qiao, W. Bian and D. Tao, Conditional graphical lasso for multi-label image classification, in CVPR (2016). Crossref, Google Scholar
14. Y. Li, E. Hung and K. Chung, A subspace decision cluster classifier for text classification, Expert Syst. Appl. 38(10) (2011) 12475–12482. Crossref, Web of Science, Google Scholar
15. Y. Li, E. Hung, K. Chung and J. Huang, Building a decision cluster classification model for high dimensional data by a variable weighting k-means method, in AI 2008: Advances in Artificial Intelligence (2008), pp. 337–347. Crossref, Google Scholar
16. Y. Li, Y. Ye, Z. Sun, E. Hung, J. Huang and Y. Li, An ensemble of decision cluster crotches for classification of high dimensional data, Knowl.-Based Syst. 43 (2013) 63–73. Crossref, Web of Science, Google Scholar
17. W. Lin and D. Xu, Imbalanced multi-label learning for identifying antimicrobial peptides and their functional types, Bioinformatics (2016). Crossref, Web of Science, Google Scholar
18. B. Liu, Y. Li and Z. Xu, Manifold regularized matrix completion for multi-label learning with admm, Neural Netw. (2018) 57–67. Crossref, Web of Science, Google Scholar
19. S. M. Liu and J.-H. Chen, A multi-label classification based approach for sentiment classification, Expert Syst. Appl. 42(3) (2015) 1083–1093. Crossref, Web of Science, Google Scholar
20. Y. Liu, L. Sheng, J. Shao, J. Yan, S. Xiang and C. Pan, Multi-label image classification via knowledge distillation from weakly-supervised detection, in ACM MM (2018), p. 700–708. Crossref, Google Scholar
21. G. Madjarov, D. Kocev, D. Gjorgjevikj and S. Droski, An extensive experimental comparison of methods for multi-label learning, Pattern Recogn. 45(9) (2012) 3084–3104. Crossref, Web of Science, Google Scholar
22. J. Nam, J. Kim, E. Loza Mencía, I. Gurevych and J. Fürnkranz, Large-scale multi-label text classification — revisiting neural networks, in ECML-PKDD (2014), pp. 437–452. Crossref, Google Scholar
23. J. Read, L. Martino, P. M. Olmos and D. Luengo, Scalable multi-output label prediction: From classifier chains to classifier trellises, Pattern Recogn. 48(6) (2015) 2096–2109. Crossref, Web of Science, Google Scholar
24. J. Read, B. Pfahringer, G. Holmes and E. Frank, Classifier chains for multi-label classification, Mach. Learn. 85(3) (2011) 333–359. Crossref, Web of Science, Google Scholar
25. G. Song and Y. Ye, A new ensemble method for multi-label data stream classification in non-stationary environment, in IJCNN (2014), pp. 1776–1783. Crossref, Google Scholar
26. E. Spyromitros, G. Tsoumakas and I. Vlahavas, An empirical study of lazy multilabel classification algorithms, in Artificial Intelligence: Theories, Models and Applications (2008), pp. 401–406. Crossref, Google Scholar
27. G. Tsoumakas, I. Katakis and I. Vlahavas, Random k-labelsets for multilabel classification, TKDE 23(7) (2011) 1079–1089. Web of Science, Google Scholar
28. G. Tsoumakas, I. Katakis and I. Vlahavas, Mining Multi-label Data (Springer, US, 2010), pp. 667–685. Google Scholar
29. G. Tsoumakas, E. Spyromitros-Xioufis, J. Vilcek and I. Vlahavas, Mulan: A Java library for multi-label learning, JMLR 12 (2011) 2411–2414. Google Scholar
30. S. Vluymans, C. Cornelis, F. Herrera and Y. Saeys, Multi-label classification using a fuzzy rough neighborhood consensus, Inform. Sci. 433–434 (2017). Web of Science, Google Scholar
31. X. Wang, W. Zhang, Q. Zhang and G.-Z. Li, Multip-schlo: Multi-label protein subchloroplast localization prediction with chous pseudo amino acid composition and a novel multi-label classifier, Bioinformatics 31(16) (2015) 2639–2645. Crossref, Web of Science, Google Scholar
32. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn., Morgan Kaufmann Series in Data Management Systems (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2005). Google Scholar
33. Q. Wu, M. Tan, H. Song, J. Chen and M. K. Ng, Ml-forest: A multi-label tree ensemble method for multi-label classification, TKDE 28(10) (2016) 2665–2680. Web of Science, Google Scholar
34. Q. Wu, Y. Ye, H. Zhang, T. W. S. Chow and S. Ho, ML-TREE: A tree-structure-based approach to multilabel learning, TNNLS 26(3) (2015) 430–443. Crossref, Web of Science, Google Scholar
35. M. Xu, Y. Li and Z. Zhou, Robust multi-label learning with pro loss, TKDE (2019) 1–1. Google Scholar
36. C.-K. Yeh, W.-C. Wu, W.-J. Ko and Y.-C. F. Wang, Learning deep latent space for multi-label classification, in AAAI (2017). Crossref, Google Scholar
37. M. L. Zhang and L. Wu, Lift: Multi-label learning with label-specific features, TPAMI 37(1) (2015) 107–120. Crossref, Web of Science, Google Scholar
38. M.-L. Zhang and Z.-H. Zhou, Multilabel neural networks with applications to functional genomics and text categorization, TKDE 18(10) (2006) 1338–1351. Web of Science, Google Scholar
39. M.-L. Zhang and Z.-H. Zhou, Ml-knn: A lazy learning approach to multi-label learning, Pattern Recogn. 40(7) (2007) 2038–2048. Crossref, Web of Science, Google Scholar
40. Y. Zhu, K. M. Ting and Z. Zhou, Multi-label learning with emerging new labels, TKDE 30(10) (2018) 1901–1914. Web of Science, Google Scholar