Hierarchical classification learning: A novel two-layer framework for multiclass classification

This paper reports on how to transform a multiclass classification problem into a set of simpler classification problems and then combine the solutions to the simpler problems into a solution to the original multiclass classification. A novel two-layer framework is presented, called Hierarchical Classification Learning. Different machine learning algorithms can be employed as the base classifier in this classification learning framework. First of all, the multiclass data set is reformed for every pair of classes, resulting in multiple 3-class sensor data sets — two classes for the pair of classes and the third class for any data instance that not belong to any of the two classes. A classification model, called sensor model, is constructed for each of the sensor data sets using a machine learning algorithm, or the base classifier. Then every data instance of the original data set is put into all the sensor models, generating a set of sensor outputs or secondary features which compose a sensor vector. Every sensor vector is viewed as a reformed version of the original data that has the same class label as the original, so put all sensor vectors together and get a new reformed data set. A classification model, decision model, is constructed from the new reformed data set. At last, extensive experiments have been conducted to evaluate the framework, and neural network, decision tree, random tree, and support vector machine are used as the base classifiers. Experiment results on UCI datasets and some popular face classification datasets show that the classification learning framework has achieved superior performance than their corresponding base classifiers.