Narrow Results

The prevalence of student dropout in academic settings is a serious issue that affects individuals and society as a whole. Timely intervention and support can be provided to such students if we get an accurate prediction of student performance. However, class imbalance and data complexity in education data are major challenges for traditional predictive analytics. Our research focusses on utilising machine learning techniques to predict student performance while handling imbalanced datasets. To address the imbalanced class problem, we employed both oversampling and undersampling techniques in our decision tree ensemble methods for the risk classification of prospective students. The effectiveness of classifiers was evaluated by varying the sizes of the ensembles and the oversampling and undersampling ratios. Additionally, we conducted experiments to integrate the feature selection processes with the best ensemble classifiers to further enhance the prediction. Based on the extensive experimentation, we concluded that ensemble methods such as Random Forest, Bagging, and Random Undersampling Boosting perform well in terms of performance measures such as Recall, Precision, $F$1-score, Area Under the Receiver Operating Characteristic Curve, and Geometric Mean. The $F$1-score of 0.849 produced by the Random Undersampling Boost classifier in conjunction with the Least Absolute Shrinkage and Selection Operator feature selection method indicates that this ensemble produces the best results.

Imbalanced classification is related to those problems that have an uneven distribution among classes. In addition to the former, when instances are located into the overlapped areas, the correct modeling of the problem becomes harder. Current solutions for both issues are often focused on the binary case study, as multi-class datasets require an additional effort to be addressed. In this research, we overcome these problems by carrying out a combination between feature and instance selections. Feature selection will allow simplifying the overlapping areas easing the generation of rules to distinguish among the classes. Selection of instances from all classes will address the imbalance itself by finding the most appropriate class distribution for the learning task, as well as possibly removing noise and difficult borderline examples. For the sake of obtaining an optimal joint set of features and instances, we embedded the searching for both parameters in a Multi-Objective Evolutionary Algorithm, using the C4.5 decision tree as baseline classifier in this wrapper approach. The multi-objective scheme allows taking a double advantage: the search space becomes broader, and we may provide a set of different solutions in order to build an ensemble of classifiers. This proposal has been contrasted versus several state-of-the-art solutions on imbalanced classification showing excellent results in both binary and multi-class problems.

The aim of this paper is to provide better understanding of a few approaches that have been proposed for treating nonequilibrium (time-dependent) processes in statistical mechanics with the emphasis on the interrelation between theories. The ensemble method, as it was formulated by Gibbs, has great generality and broad applicability to equilibrium statistical mechanics. Different macroscopic environmental constraints lead to different types of ensembles, with particular statistical characteristics. In the present work, the statistical theory of nonequilibrium processes which is based on nonequilibrium ensemble formalism is discussed. We also outline the reasoning leading to some other useful approaches to the description of the irreversible processes. The kinetic approach to dynamic many-body problems, which is important from the point of view of the fundamental theory of irreversibility, is alluded to. Appropriate references are made to papers dealing with similar problems arising in other fields. The emphasis is on the method of the nonequilibrium statistical operator (NSO) developed by Zubarev. The NSO method permits one to generalize the Gibbs ensemble method to the nonequilibrium case and to construct a nonequilibrium statistical operator which enables one to obtain the transport equations and calculate the transport coefficients in terms of correlation functions, and which, in the case of equilibrium, goes over to the Gibbs distribution. Although some space is devoted to the formal structure of the NSO method, the emphasis is on its utility. Applications to specific problems such as the generalized transport and kinetic equations, and a few examples of the relaxation and dissipative processes, which manifest the operational ability of the method, are considered.

Univariate regression trees have representation problems for non-orthogonal regression functions. Ensembles of univariate regression trees have better representational power. In some cases, weighted ensembles have shown better performance than unweighted ensembles. In this paper, we study the properties of ensembles of regression trees by using regression classification models. We propose a theoretical framework to study the representational power of infinite-sized weighted ensembles, consisting of randomised finite-sized regression trees. We show for some datasets that the weighted ensembles may have better representational power than unweighted ensembles, but the performance is highly dependent on the weighting scheme and the properties of datasets. Our model cannot be used for all the datasets. However, for some datasets, we can accurately predict the experimental results of ensembles of regression trees.

With the objective of helping diagnose some pathologies in recently born babies, we present the experiments and results obtained in the classification of infant cry using a variety of single classifiers, and ensembles from the combination of them. Three kinds of cry were classified: normal, hypoacoustic (deaf), and asphyxia. The feature vectors were formed by the extraction of Mel Frequency Cepstral Coefficients (MFCC). The vectors were then processed and reduced through the application of five statistics operations, namely: minimum, maximum, average, standard deviation and variance. LDA, a data reduction technique is implemented with the purpose of comparing the results of our proposed method. Four supervised machine learning methods including Support Vector Machines, Neural Networks, J48, Random Forest and Naive Bayes are used. The ensembles tested were combinations of these under different approaches like Majority Vote, Staking, Bagging and Boosting.

In this paper, we present an approach for sample selection using an ensemble of neural networks for credit scoring. The ensemble determines samples that can be considered outliers by checking the classification accuracy of the neural networks on the original training data samples. Those samples that are consistently misclassified by the neural networks in the ensemble are removed from the training dataset. The remaining data samples are then used to train and prune another neural network for rule extraction. Our experimental results on publicly available benchmark credit scoring datasets show that by eliminating the outliers, we obtain neural networks with higher predictive accuracy and simpler in structure compared to the networks that are trained with the original dataset. A rule extraction algorithm is applied to generate comprehensible rules from the neural networks. The extracted rules are more concise than the rules generated from networks that have been trained using the original datasets.

Data mining is much more than simply building statistical models from large collections of data. In particular, this paper records a core task of mining as exploring through the space of models that are built in a data mining project. The idea was first introduced through the concept of multiple inductive learning (MIL) (Williams, 1988, 1991) and further developed in practice as mining the data mine (Williams and Huang, 1997). Many data mining advances that have since emerged have further developed the idea: multiple modelling, ensemble learning, bagging and boosting all help the data miner explore different ideas and look for different insights in modelling. In this paper we review these ideas and a number of data mining projects that highlight the significant role played by mining the data mine.