Machine LearningNo Access

COST-SENSITIVE MULTI-VIEW LEARNING MACHINE

Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

Shanghai Key Laboratory of Intelligent Information Processing, Shanghai 200433, P. R. China

Corresponding author.

Search for more papers by this author

MINGZHE LU

Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

Search for more papers by this author

ZENGXIN NIU

Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

Search for more papers by this author

XIANGYANG XUE

School of Computer Science, Fudan University, Shanghai 200433, P. R. China

Search for more papers by this author

, and

DAQI GAO

Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

Search for more papers by this author

https://doi.org/10.1142/S0218001414510045Cited by:0 (Source: Crossref)

Abstract

Multi-view learning aims to effectively learn from data represented by multiple independent sets of attributes, where each set is taken as one view of the original data. In real-world application, each view should be acquired in unequal cost. Taking web-page classification for example, it is cheaper to get the words on itself (view one) than to get the words contained in anchor texts of inbound hyper-links (view two). However, almost all the existing multi-view learning does not consider the cost of acquiring the views or the cost of evaluating them. In this paper, we support that different views should adopt different representations and lead to different acquisition cost. Thus we develop a new view-dependent cost different from the existing both class-dependent cost and example-dependent cost. To this end, we generalize the framework of multi-view learning with the cost-sensitive technique and further propose a Cost-sensitive Multi-View Learning Machine named CMVLM for short. In implementation, we take into account and measure both the acquisition cost and the discriminant scatter of each view. Then through eliminating the useless views with a predefined threshold, we use the reserved views to train the final classifier. The experimental results on a broad range of data sets including the benchmark UCI, image, and bioinformatics data sets validate that the proposed algorithm can effectively reduce the total cost and have a competitive even better classification performance. The contributions of this paper are that: (1) first proposing a view-dependent cost; (2) establishing a cost-sensitive multi-view learning framework; (3) developing a wrapper technique that is universal to most multiple kernel based classifier.

Keywords:

References

N. Abe, B. Zadrozny and J. Langford, An iterative method for multi-class cost-sensitive learning, Proc. Tenth ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (ACM, NY, USA, 2004) pp. 3–11. Google Scholar
S. Abney, Bootstrapping, Proc. 40th Annual Meeting on Association for Computational Linguistics (Association for Computational Linguistics, Philadelphia, USA, 2002) pp. 360–367. Google Scholar
A. Agarwal, Selective sampling algorithms for cost-sensitive multiclass prediction, Proc. 30th Int. Conf. Machine Learning (Atlanta, USA, 2013) pp. 1220–1228. Google Scholar
U. Alonet al., Proc. Natl. Acad. Sci. U.S.A. 96(12), 6745 (1999). Crossref, Web of Science, Google Scholar
F. R. Bach , G. R. G. Lanckriet and M. I. Jordan , Multiple kernel learning, conic duality, and the smo algorithm , Proc. Twenty-First Int. Conf. Machine Learning ( ACM , 2004 ) . Google Scholar
K. Bache and M. Lichman. UCI machine learning repository (2013) . Google Scholar
N. Baili and H. Frigui, Fuzzy clustering with multiple kernels in feature space, IEEE Int. Conf. Fuzzy Systems (IEEE, Brishane, QLD, 2012) pp. 1–8. Google Scholar
A. Blum and T. Mitchell, Combining labeled and unlabeled data with co-training, Proc. Eleventh Annual Conf. Computational Learning Theory (ACM, Madison, Wisconsin, 1998) pp. 92–100. Google Scholar
J.-P. Brunetet al., Proc. Natl. Acad. Sci. U.S.A. 101(12), 4164 (2004). Crossref, Web of Science, Google Scholar
P. K. Chan and S. J. Stolfom, Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection, KDD1998 (1998) pp. 164–168. Google Scholar
N. V. Chawla, N. Japkowicz and A. Kotcz, ACM Sigkdd Explorations Newsletter 6(1), 1 (2004). Crossref, Google Scholar
Z.-Y. Chen and Z.-P. Fan , Knowl.-Based Syst. 43 , 123 ( 2013 ) . Crossref, Web of Science, Google Scholar
M. Collins and Y. Singer, Unsupervised models for named entity classification, Proc. Joint SIGDAT Conf. Empirical Methods in Natural Language Processing and Very Large Corpora (ACL, New Brunswick, 1999) pp. 100–110. Google Scholar
S. Dasgupta, M. L. Littman and D. McAllester, Pac generalization bounds for co-training, Annual Conf. Neural Information Processing Systems (Cambridge, 2001) pp. 375–382. Google Scholar
V. R. De Sa, Learning classification with unlabeled data, Advances in Neural Information Processing Systems (NIPS)6 (1993) pp. 112–119. Google Scholar
C. Demir and E. Alpaydin, Pattern Recogn. Lett. 26(14), 2206 (2005). Crossref, Web of Science, Google Scholar
T. Diethe and M. Girolami, Neural Comput. 25(3), 567 (2013). Crossref, Web of Science, Google Scholar
P. Domingos, Metacost: A general method for making classifiers cost-sensitive, Proc. Fifth ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (ACM, San Diego, CA, 1999) pp. 155–164. Google Scholar
C. Drummondet al., C4. 5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling, Workshop on Learning from Imbalanced Datasets II (Citeseer, Washington DC, 2003) pp. 1–8. Google Scholar
C. Elkan, The foundations of cost-sensitive learning, Int. Joint Conf. Artificial Intelligence (Citeseer, Seattle, Washington, 2001) pp. 973–978. Google Scholar
G. Funget al., A fast iterative algorithm for fisher discriminant using heterogeneous kernels, Proc. Twenty-First Int. Conf. Machine Learning (ACM, New York, USA, 2004) pp. 40–47. Google Scholar
M. Gönen and E. Alpaydın, Pattern Recogn. Lett. 31(9), 959 (2010). Crossref, Web of Science, Google Scholar
H.-C. Huang, Y.-Y. Chuang and C.-S. Chen, IEEE Trans. Fuzzy Syst. 20(1), 120 (2012). Crossref, Web of Science, Google Scholar
L. Jianet al., Neural Netw. 24(5), 476 (2011). Crossref, Web of Science, Google Scholar
U. Kreßel, Advances in Kernel Methods (MIT Press, 1999) pp. 255–268. Google Scholar
S. Kumar, M. Mohri and A. Talwalkar, Ensemble nystrom method, Advances in Neural Information Processing Systems (NIPS) (2009) pp. 1060–1068. Google Scholar
G. R. G. Lanckriet et al. , J. Mach. Learn. Res. 5 , 27 ( 2004 ) . Web of Science, Google Scholar
Y. Lee, Y. Lin and G. Wahba, J. Am. Stat. Assoc. 99(465), 67 (2004). Crossref, Web of Science, Google Scholar
J. Leski , Int. J. Appl. Math. Comput. Sci. 14 , 53 ( 2004 ) . Google Scholar
Y.-F. Li, J. T. Kwok and Z.-H. Zhou, Cost-sensitive semi-supervised support vector machine, AAAI Conf. Artificial Intelligence (Atlanta, Georgia, 2010) pp. 500–505. Google Scholar
Y. Lin, Y. Lee and G. Wahba, Mach. Learn. 46(3), 191 (2002). Crossref, Web of Science, Google Scholar
J. Liu and S. Chen, Pattern Recogn. Lett. 27(9), 1002 (2006). Crossref, Web of Science, Google Scholar
A. C. Lozano and N. Abe, Multi-class cost-sensitive boosting with p-norm loss functions, Proc. 14th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (ACM, New York, USA, 2008) pp. 506–514. Google Scholar
Y. Luet al., Int. J. Pattern Recogn. Artif. Intell. 26(5), 1263001 (2012). Link, Web of Science, Google Scholar
M. A. Maloof, Learning when data sets are imbalanced and when costs are unequal and unknown, Int. Conf. Machine Learning 2003 Workshop on Learning from Imbalanced Data Sets II (Washington, DC, 2003) pp. 1–8. Google Scholar
B. A. Pireset al., Cost-sensitive multiclass classification risk bounds, Int. Conf. Machine Learning (Atlanta, Georgia, 2013) pp. 1391–1399. Google Scholar
M. H. Quang, L. Bazzani and V. Murino, A unifying framework for vector-valued manifold regularization and multi-view learning, Proc. 30th Int. Conf. Machine Learning (Atlanta, Georgia, USA, 2013) pp. 100–108. Google Scholar
S. Sonnenburg, G. Rätsch and C. Schäfer, A general and efficient multiple kernel learning algorithm, Annual Conf. Neural Information Processing Systems (Vancouver, British Columbia, Canada, 2005) pp. 1275–1282. Google Scholar
S. Sonnenburg et al. , J. Mach. Learn. Res. 7 , 1531 ( 2006 ) . Web of Science, Google Scholar
Y. Sunet al., Pattern Recogn. 40(12), 3358 (2007). Crossref, Web of Science, Google Scholar
K. M. Ting, IEEE Trans. Knowl. Data Eng. 14(3), 659 (2002). Crossref, Web of Science, Google Scholar
I. W. Tsang, A. Kocsor and J. T. Kwok, Efficient kernel feature extraction for massive data sets, Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (ACM, New York, USA, 2006) pp. 724–729. Google Scholar
Z. Wang and S. Chen, Neurocomputing 72(10), 2444 (2009). Crossref, Web of Science, Google Scholar
Z. Wang, S. Chen and D. Gao, Exp. Syst. Appl. 38(9), 11193 (2011). Crossref, Web of Science, Google Scholar
Z. Wanget al., IEEE Trans. Neural Netw. 19(5), 758 (2008). Crossref, Web of Science, Google Scholar
Z. Wang, S. Chen and T. Sun, IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 348 (2008). Crossref, Web of Science, Google Scholar
H. Wang, F. Nie and H. Huang, Multi-view clustering and feature learning via structured sparsity, Proc. 30th Int. Conf. Machine Learning (Atlanta, Georgia, USA, 2013) pp. 352–360. Google Scholar
F. Wei-Li, Y. Ying-Kuei and P. Jung-Kuei, Int. J. Pattern Recogn. Artif. Intell. 26(5), 1256015 (2012). Link, Web of Science, Google Scholar
M. Westet al., Proc. Natl. Acad. Sci. U.S.A. 98(20), 11462 (2001). Crossref, Web of Science, Google Scholar
G. Wu and E. Y. Chang, Class-boundary alignment for imbalanced dataset learning, Int. Conf. Machine Learning 2003 Workshop on Learning from Imbalanced Data Sets II (Washington, DC, 2003) pp. 49–56. Google Scholar
Q.-S. Xu and Y.-Z. Liang, Chemometr. Intell. Lab. Syst. 56(1), 1 (2001). Crossref, Web of Science, Google Scholar
C. Xu, D. Tao and C. Xu, A survey on multi-view learning (2013) , arXiv:1304.5634 . Google Scholar
D. Yarowsky, Unsupervised word sense disambiguation rivaling supervised methods, Proc. 33rd Annual Meeting on Association for Computational Linguistics (Association for Computational Linguistics, Stroudsburg, PA, 1995) pp. 189–196. Google Scholar
B. Zadrozny, J. Langford and N. Abe, Cost-sensitive learning by cost-proportionate example weighting, IEEE Int. Conf. Data Mining (IEEE, Los Alamitos, CA, 2003) pp. 435–442. Google Scholar
Y. Zhang and Z.-H. Zhou, IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1758 (2010). Crossref, Web of Science, Google Scholar
Z.-H. Zhou and X.-Y. Liu, Comput. Intell. 26(3), 232 (2010). Crossref, Web of Science, Google Scholar