Narrow Results

In metal-cutting operations, the surface roughness of the end product plays a significant role. It not only affects the aesthetic appearance of the end product but also signifies the product’s performance in the long run. Products with a high surface finish have higher endurance limits with negligible local stresses. On the other hand, products with rough surfaces are subjected to high stresses when they are engaged in various mechanical operations with varying loads. Surface roughness depends on various machining factors such as feed rate, depth of cut, cutting speed, or spindle speed. Therefore, it is required to predict surface roughness for the given machining parameters to reduce the cost and increase the life of the end product. In this work, an attempt has been made to evaluate the surface roughness of AZ91 alloy during the end milling operation. In this regard, various state-of-the-art ensemble learning models have been adopted and compared with the proposed hybrid ensemble model. The proposed hybrid ensemble model is the integration of random forest, gradient boosting, and a deep multi-layered neural network. In order to evaluate the performance of the proposed model, comparative analyses have been made in terms of mean square error, mean average error, and $R^2$ score. The result shows that the proposed hybrid model gives minimum error for surface roughness.

This paper presents a novel model for performing classification and visualization of high-dimensional data by means of combining two enhancing techniques. The first is a semi-supervised learning, an extension of the supervised learning used to incorporate unlabeled information to the learning process. The second is an ensemble learning to replicate the analysis performed, followed by a fusion mechanism that yields as a combined result of previously performed analysis in order to improve the result of a single model. The proposed learning schema, termed S²-Ensemble, is applied to several unsupervised learning algorithms within the family of topology maps, such as the Self-Organizing Maps and the Neural Gas. This study also includes a thorough research of the characteristics of these novel schemes, by means quality measures, which allow a complete analysis of the resultant classifiers from the viewpoint of various perspectives over the different ways that these classifiers are used. The study conducts empirical evaluations and comparisons on various real-world datasets from the UCI repository, which exhibit different characteristics, so to enable an extensive selection of situations where the presented new algorithms can be applied.

Parkinson’s Disease (PD) is the second most prevalent neurodegenerative disorder among adults. Although its triggers are still not clear, they may be due to a combination of different types of biomarkers measured through medical imaging, metabolomics, proteomics or genetics, among others. In this context, we have proposed a Computer-Aided Diagnosis (CAD) system that combines structural and functional imaging data from subjects in Parkinson’s Progression Markers Initiative dataset by means of an Ensemble Learning methodology trained to identify and penalize input sources with low classification rates and/ or high-variability. This proposal improves results published in recent years and provides an accurate solution not only from the point of view of image preprocessing (including a comparison between different intensity preservation techniques), but also in terms of dimensionality reduction methods (Isomap). In addition, we have also introduced a bagging classification schema for scenarios with unbalanced data.

As shown by our results, the CAD proposal is able to detect PD with $96.48\%$ of balanced accuracy, and opens up the possibility of combining any number of input data sources relevant for PD.

Artificial intelligence (AI)-based approaches are crucial in computer-aided diagnosis (CAD) for various medical applications. Their ability to quickly and accurately learn from complex data is remarkable. Deep learning (DL) models have shown promising results in accurately classifying Alzheimer’s disease (AD) and its related cognitive states, Early Mild Cognitive Impairment (EMCI) and Late Mild Cognitive Impairment (LMCI), along with the healthy conditions known as Cognitively Normal (CN). This offers valuable insights into disease progression and diagnosis. However, certain traditional machine learning (ML) classifiers perform equally well or even better than DL models, requiring less training data. This is particularly valuable in CAD in situations with limited labeled datasets. In this paper, we propose an ensemble classifier based on ML models for magnetic resonance imaging (MRI) data, which achieved an impressive accuracy of 96.52%. This represents a 3–5% improvement over the best individual classifier. We evaluated popular ML classifiers for AD classification under both data-scarce and data-rich conditions using the Alzheimer’s Disease Neuroimaging Initiative and Open Access Series of Imaging Studies datasets. By comparing the results to state-of-the-art CNN-centric DL algorithms, we gain insights into the strengths and weaknesses of each approach. This work will help users to select the most suitable algorithm for AD classification based on data availability.

Overlapping community detection is a hot topic in the research of data mining and graph theory. In this paper, we propose a link community detection method based on ensemble learning (LCDEL). First, we transform graph into line graph and construct node adjacency matrix of line graph. Second, we calculate node distance of line graph through a new distance metric and get node distance matrix of line graph. Third, we use PCA method to reduce dimensions of node distance matrix of line graph. Then, we cluster on the reduced node distance matrix by k-means clustering algorithm. Finally, we convert line graph back into original graph and get overlapping communities of original graph with ensemble learning. Experimental results on several real-world networks demonstrate effectiveness of LCDEL method in terms of Normalized Mutual Information (NMI), Extended Modularity (EQ) and F-score evaluation metrics.

Protein structure classification is an important issue in understanding the associations between sequence and structure as well as possible functional and evolutionary relationships. Recently structural genomes initiatives and other high-throughput experiments have populated the biological databases at a rapid pace. In this paper, three types of classifiers, k nearest neighbors, class center and nearest neighbor and probabilistic neural networks and their homogenous ensemble for multiclass protein fold recognition problem are evaluated firstly, and then a heterogenous ensemble Voting System is designed for the same problem. The different features and/or their combinations extracted from the protein fold dataset are used in these classification models. The heterogenous classification results are then put into a voting system to get the final result. The experimental results show that the proposed method can improve prediction accuracy by 4%–10% on a benchmark dataset containing 27 SCOP folds.

In human society, people learn from each other and knowledge is accumulated from generation to generation. This provides some hints to distributed learning. For distributed applications, each site has its own data. If we can build a local model for each site and improve the model based on models learned by its neighbor sites with low communication cost, then it would be very helpful to the distributed applications. In this paper, we propose a new distributed learning method called distributed network boosting (DNB) algorithm for distributed applications. The learned hypotheses are exchanged between neighboring sites during learning process. Theoretical analysis shows that the DNB algorithm minimizes the cost function through collaborative functional gradient descent in hypotheses space. We also give upper bounds of training error and generalization error of the DNB algorithm. Comparison results of the DNB algorithm with other algorithms on real data sets with different sizes show the effectiveness of the proposed algorithm for distributed applications. In order to show the influence of network topology on the performance of the DNB algorithm, we tested it on random graphs and scale-free networks. Bias-variance decomposition shows that the network topology plays an important role in controlling the diversity of the learned classifier ensemble.

In this paper, we propose different ensemble learning algorithms and their application to the face recognition problem. Three types of attributes are used for image representation: statistical, spectral, and segmentation features and regional descriptors. Classification is performed by nearest neighbor using different p-norms defined in the corresponding spaces of attributes. In this approach, each attribute together with its corresponding type of the analysis (local or global) and the distance criterion (norm or cosine), define a different classifier. The classification is unsupervised since no class information is used to improve the design of the different classifiers. Three different versions of ensemble classifiers are proposed in this paper: CAV1, CAV2, and CBAG, being the main differences among them the way the image candidates that perform the consensus are selected. The main results shown in this paper are the following: 1. The statistical attributes (local histogram and percentiles) are the individual classifiers that provided the higher accuracies, followed by the spectral methods (DWT), and the regional features (texture analysis). 2. No single attribute is able to provide systematically 100% accuracy over the ORL database. 3. The accuracy and stability of the classification is increased by consensus classification (ensemble learning techniques). 4. Optimum results are obtained by reducing the number of classifiers taking into account their diversity, and by optimizing the parameters of these classifiers using a member of the Particle Swarm Optimization (PSO) family. These results are in accord with the conclusions that are presented in the literature using ensemble learning methodologies, that is, it is possible to build strong classifiers by assembling different weak (or simple) classifiers based on different and diverse image attributes. Due to these encouraging results, future research will be devoted to the use of supervised ensemble techniques in face recognition and in other important biometric problems.

Machine learning is among the most popular methods in designing unknown and variant malware detection algorithms. However, most of the existing methods take a single type of features to build binary classifiers. In practice, these methods have limited ability in depicting malware characteristics and the binary classification suffers from inadequate sampling of benign samples and extremely imbalanced training samples when detecting malware. In this paper, we present a malware detection Framework based on ENsemble One-Class Learning, namely FENOC. It uses hybrid features at different semantic layers to ensure a comprehensive insight of the program to be analyzed. We construct the malware detector by a novel learning algorithm called Cost-sensitive Twin One-class Classifier (CosTOC), which uses a pair of one-class classifiers to describe malware and benign programs respectively. CosTOC is more flexible and robust in comparison to conventional binary classifiers when training samples are extremely imbalanced or the benign programs are inadequately sampled. Finally, random subspace method and clustering-based ensemble method are developed to enhance the generalization ability of CosTOC. Experimental results show that FENOC gives a comparative detection rate and a lower false positive rate than many other binary classification algorithms, especially when the detector are trained with imbalanced data, or evaluated in terms of false positive rate.

Machine-learning-based software defect prediction (SDP) methods are receiving great attention from the researchers of intelligent software engineering. Most existing SDP methods are performed under a within-project setting. However, there usually is little to no within-project training data to learn an available supervised prediction model for a new SDP task. Therefore, cross-project defect prediction (CPDP), which uses labeled data of source projects to learn a defect predictor for a target project, was proposed as a practical SDP solution. In real CPDP tasks, the class imbalance problem is ubiquitous and has a great impact on performance of the CPDP models. Unlike previous studies that focus on subsampling and individual methods, this study investigated 15 imbalanced learning methods for CPDP tasks, especially for assessing the effectiveness of imbalanced ensemble learning (IEL) methods. We evaluated the 15 methods by extensive experiments on 31 open-source projects derived from five datasets. Through analyzing a total of 37504 results, we found that in most cases, the IEL method that combined under-sampling and bagging approaches will be more effective than the other investigated methods.

Speech recognition in multilingual scenario is not trivial in the case when multiple languages are used in one conversation. Language must be identified before we process speech recognition as such tools are language-dependent. We present a language identification system (or AI tool) to distinguish top-seven world languages namely Chinese, Spanish, English, Hindi, Arabic, Bangla and Portuguese [G. F. Simons and C. D. Fennig (eds.), Ethnologue: Laguage of the Americas and the Pacific, Twentieth Edn. (SIL Internatinal, 2017)]. The system uses linear predictive coefficients-based feature, i.e. the line spectral pair–grade ratio (LSP–GR) feature, and ensemble learning for classification. Experiments were performed on more than 200h of real-world YouTube data and the highest possible accuracy of 96.95% was received. The results can be compared with other machine learning classifiers.

Binary neural networks (BNNs) have drawn much attention because of the most promising techniques to meet the desired memory footprint and inference speed requirements. However, they still suffer from the severe intrinsic instability of the error convergence, resulting in increase in prediction error and its standard deviation, which is mostly caused by the inherently poor representation with only two possible values of $-$1 and $+$1. In this work, we have proposed a cost-aware layer-wise ensemble method to address the above issue without incurring any excessive costs, which is characterized by (1) layer-wise bagging and (2) cost-aware layer selection for the bagging. One of the experimental results has shown that the proposed method reduces the error and its standard deviation by 15% and 54% on CIFAR-10, respectively, compared to the BNN serving as a baseline. This paper demonstrated and discussed such error reduction and stability performance with high versatility based on the comparison results under the various cases of combinations of the network base model with the proposed and the state-of-the-art prior techniques while changing the network sizes and datasets of CIFAR-10, SVHN, and MNIST for the evaluation.

In the recent decades, various techniques based on deep convolutional neural networks (DCNNs) have been applied to scene classification. Most of the techniques are established upon single-spectral images such that environmental conditions may greatly affect the quality of images in the visible (RGB) spectrum. One remedy for this downside is to merge the infrared (IR) with the visible spectrum for gaining the complementary information in comparison with the unimodal analysis. This paper incorporates the RGB, IR and near-infrared (NIR) images into a multispectral analysis for scene classification. For this purpose, two strategies are adopted. In the first strategy, each RGB, IR and NIR image is separately applied to DCNNs and then classified according to the output score of each network. In addition, an optimal decision threshold is obtained based on the same output score of each network. In the second strategy, three image components are extracted from each type of image using wavelet transform decomposition. Independent DCNNs are then trained on the image components of all the scene classes. Eventually, the final classification of the scene is accomplished through an appropriate ensemble architecture. The use of this architecture alongside a transfer learning approach and simple classifiers leads to lesser computational costs in small datasets. These experiments reveal the superiority of the proposed method over the state-of-the-art architectures in terms of the accuracy of scene classification.

Multi-label classification problem, where each instance can be associated with multiple labels, has received considerable attention from machine learning community. To address the inherent challenges of multi-label classification including data imbalance, label dependence, and high dimensionality, ensemble approaches have been developed, gaining popularity across various real-world applications. This paper proposes a novel ensemble method called ConfBoost that addresses these challenges and enhances the generalization ability of learning systems. ConfBoost which is a meta-model based on a weighted stacking paradigm using local confidence, combines heterogeneous and complementary ensembles of multi-label classifiers. The proposed approach achieves two main objectives: Firstly, by focusing on label weights based on their confidence scores, the model can generate more relevant predictions and enhance the accuracy at the base-level by mitigating the impact of irrelevant labels during the stacking process. Moreover, assigning higher weights to certain labels exhibits better discrimination and adaptability to capture complex label relationships. Second, applying adjusted thresholds enables the model to generate predictions adapted to the specific characteristics of each label, effectively addressing imbalanced label distributions. Extensive experiments on publicly available datasets demonstrate that ConfBoost outperforms conventional combination methods and consistently surpasses related state-of-the-art methods. These findings highlight the effectiveness and potential of ConfBoost as an advanced ensemble method for multi-label classification tasks.

Emotions indicate the feelings of the individual which are linked with personal experiences, moods, and affective states. Detection of emotion can be helpful in many fields like maintaining a patient’s psychological well-being, surveillance, driver monitoring, etc. In this paper, an effective machine learning approach has been put forth for emotion detection where an ensemble of three out of five best-performing classifiers has been formed to enhance the classification accuracy. Two deep learning models (AlexNet and ResNet) have been optimally combined with $k$-nearest neighbor (KNN). The optimal weights for ensemble weighted averaging of classifiers have been computed with aid of particle swarm optimization (PSO) and genetic algorithm (GA) optimization. The developed framework has been tested on two publicly available datasets. An overall accuracy of above 95% has been achieved on the testing set for both datasets. The best performance was obtained by training the classifiers with segmented images and combining them by using the weights obtained through PSO. The results depicted the efficiency of the optimized ensemble machine learning approach for all performance measures used in this study in comparison to the performance of individual classifiers and majority voting fusion.

Since Deep Neural Networks (DNNs) have been more and more widely used in safety-critical Intelligent System (IS) applications, the robustness of DNNs becomes a great concern in IS design. Due to the vulnerability of DNN models, adversarial examples generated by malicious attacks may result in disasters. Although there are plenty of defense methods for these adversarial attacks, existing methods can only resist special adversarial attacks. Meanwhile, the accuracy of existing methods degrades dramatically when they process nature examples. To address this problem, we propose an effective Cooperative Defensive Architecture (CDA) that can enhance the robustness of IS devices by integrating heterogeneous base classifiers. Because of the parallel mechanism in ensemble learning, the compressed heterogeneous base classifiers do not increase the prediction time on device. Comprehensive experimental results show that the modified DNNs by our approach cannot only resist adversarial examples more effectively than original model, but also achieve a high accuracy when they process nature examples.

The application of convolutional neural networks (CNNs) to plant disease recognition is widely considered to enhance the effectiveness of such networks significantly. However, these models are nonlinear and have a high bias. To address the high bias of the single CNN model, the authors proposed an ensemble method of three lightweight CNNs models (MobileNetv2, NasNetMobile and a simple CNN model from scratch) based on a stacking generalization approach. This method has two-stage training, first, we fine-tuned and trained the base models (level-0) to make predictions, then we passed these predictions to XGBoost (level-1 or meta-learner) for training and making the final prediction. Furthermore, a search grid algorithm was used for the hyperparameter tuning of the XGBoost. The proposed method is compared to the majority voting approach and all base learner models (MobileNetv2, NasNetMobile and simple CNN model from scratch). The proposed ensemble method significantly improved the performance of plant disease classification. Experiments show that the ensemble approach achieves higher prediction accuracy (98% for majority voting and 99% for staking method) than a single CNN learner. Furthermore, the proposed ensemble method has a lightweight size (e.g., 10$\times$ smaller than VGG16), allowing farmers to deploy it on devices with limited resources such as cell phones, internet of things (IoT) devices, unmanned aerial vehicles (UAVs) and so on.

It is crucial to correctly assess whether a potential borrower can repay the loan in the credit scoring model. The credit loan data has a serious data imbalance because the number of defaulters is far less than the nondefaulters. However, most current methods for dealing with data imbalance are designed to improve the classification performance of minority data, which will reduce the performance of majority data. For a financial institution, the economic loss caused by the decrease in the classification performance of nondefaulters (majority data) cannot be ignored. This paper proposes an ensemble learning method based on one-class and binary classification (EMOBC) for credit scoring. The purpose is to improve the classification accuracy of the minority class while mitigating the loss of classification accuracy of the majority class as much as possible. EMOBC uses undersampling for the majority class (nondefault samples in credit scoring) and perform binary-class learning on the balanced data to improve the classification accuracy of the minority. To alleviate the decline in classification performance of the majority class, EMOBC uses one-class and binary collaborative classification to train classifiers. The classification result is determined by the average of one-class and binary-class classifiers. The experimental results show that EMOBC has good comprehensive performance compared with the existing methods.

Accurately predicting the maximum temperature is essential for studying human comfort, ecological environment development and social progress. However, traditional prediction methods are inefficient and inaccurate when dealing with large volumes of meteorological data. To tackle these challenges, this paper introduces an integrated approach, the ARIMA–LSTM–XGBoost model, which combines the strengths of autoregressive integrated moving average (ARIMA), long short-term memory network (LSTM) and eXtreme Gradient Boosting (XGBoost) to predict the maximum temperature. The proposed model enhances the prediction accuracy and convergence rate through techniques like MAPE reciprocal weight (MAPE-RW) and Schedule Sampling. Additionally, the model selects the best performing model using the early stopping method. This paper compares and analyzes the prediction results of the ARIMA, LSTM, XGBoost and ARIMA–LSTM–XGBoost models. The experimental results indicate that the ARIMA–LSTM–XGBoost model proposed in this paper achieves superior prediction accuracy, performance, and confidence. The ARIMA–LSTM–XGBoost model shows a Root-Mean-Squared Error (RMSE) of 1.381, significantly outperforming the ARIMA model (3.828), LSTM model (3.360) and XGBoost model (1.422). The coefficient of determination ($R^2)$ is 0.977, surpassing the values of 0.905 for the ARIMA model, 0.922 for the LSTM model and 0.910 for the XGBoost model. The ARIMA–LSTM–XGBoost model also exhibits a higher confidence level compared to the individual models.

The credit risk assessment acts as an important part in daily affairs for financial institutions. But in the era of big data, the growing business volume makes it an urgent demand to develop digital ways of credit risk assessment. Currently, the machine learning is universally employed to establish various data-driven models for this purpose. However, machine learning models generally suffer from limited ability of feature representation and robustness, and cannot deal with more complex financial security scenarios. To deal with this issue, this work introduces ensemble learning to construct a stronger credit risk prediction model via integration of several basic machine learning models. Thus, an ensemble learning-enhanced smart prediction model for financial credit risk is proposed in this paper. Three classification-based machine learning models (support vector machine, artificial neural network and radial basis function) are selected as the basic classifiers, and “voting” strategy is utilized to integrate them into a novel strong classifier. A real-world financial credit dataset released by a Chinese commercial bank was selected as the experimental scenario. The obtained results show that the proposal has better prediction accuracy compared with basic machine learning models without ensemble learning.