Narrow Results

Engineers use software defect prediction (SDP) to locate vulnerable areas of software. Recently, statement-level SDP has attracted the attention of researchers due to its ability to localize faulty code areas. This paper proposes DP-Tramo, a new model dedicated to improving the state-of-the-art statement-level SDP. We use Clang to extract abstract syntax trees from source code and extract 32 statement-level metrics as static features for each sentence. Then we feed static features and token sequences as inputs to our improved R-Transformer to learn the syntactic and semantic features of the code. Furthermore, we use label smoothing and weighted loss to improve the performance of DP-Tramo. To evaluate DP-Tramo, we perform a 10-fold cross-validation on 119,989 C/C++ programs selected from Code4Bench. Experimental results show that DP-Tramo can classify the dataset with an average performance of 0.949, 0.602, 0.734 and 0.737 regarding the recall, precision, accuracy and F1-measure, respectively. DP-Tramo outperforms the baseline method on F1-measure by 1.2% while maintaining a high recall rate.

The primary goal of software quality engineering is to produce a high quality software product through the use of some specific techniques and processes. One strategy is applying data mining techniques to software metric and defect data collected during the software development process to identify potential low-quality program modules. In this paper, we investigate the use of feature selection in the context of software quality estimation (also referred to as software defect prediction), where a classification model is used to predict whether program modules (instances) are fault-prone or not-fault-prone. Seven filter-based feature ranking techniques are examined. Among them, six are commonly used, and the other one, named signal to noise ratio (SNR), is rarely employed. The objective of the paper is to compare these seven techniques for various software data sets and assess their effectiveness for software quality modeling. A case study is performed on 16 software data sets, and classification models are built with five different learners and evaluated with two performance metrics. Our experimental results are summarized based on statistical tests for significance. The main conclusion is that the SNR technique performs as well as the best performer of the six commonly used techniques.

Software defect prediction models that use software metrics such as code-level measurements and defect data to build classification models are useful tools for identifying potentially-problematic program modules. Effectiveness of detecting such modules is affected by the software measurements used, making data preprocessing an important step during software quality prediction. Generally, there are two problems affecting software measurement data: high dimensionality (where a training dataset has an extremely large number of independent attributes, or features) and class imbalance (where a training dataset has one class with relatively many more members than the other class). In this paper, we present a novel form of ensemble learning based on boosting that incorporates data sampling to alleviate class imbalance and feature (software metric) selection to address high dimensionality. As we adopt two different sampling methods (Random Undersampling (RUS) and Synthetic Minority Oversampling (SMOTE)) in the technique, we have two forms of our new ensemble-based approach: selectRUSBoost and selectSMOTEBoost. To evaluate the effectiveness of these new techniques, we apply them to two groups of datasets from two real-world software systems. In the experiments, four learners and nine feature selection techniques are employed to build our models. We also consider versions of the technique which do not incorporate feature selection, and compare all four techniques (the two different ensemble-based approaches which utilize feature selection and the two versions which use sampling only). The experimental results demonstrate that selectRUSBoost is generally more effective in improving defect prediction performance than selectSMOTEBoost, and that the techniques with feature selection do help for getting better prediction than the techniques without feature selection.

Defect prediction is very challenging in software development practice. Classification models are useful tools that can help for such prediction. Classification models can classify program modules into quality-based classes, e.g. fault-prone (fp) or not-fault-prone (nfp). This facilitates the allocation of limited project resources. For example, more resources are assigned to program modules that are of poor quality or likely to have a high number of faults based on the classification. However, two main problems, high dimensionality and class imbalance, affect the quality of training datasets and therefore classification models. Feature selection and data sampling are often used to overcome these problems. Feature selection is a process of choosing the most important attributes from the original dataset. Data sampling alters the dataset to change its balance level. Another technique, called boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models), is found to also be effective for resolving the class imbalance problem.

In this study, we investigate an approach for combining feature selection with this ensemble learning (boosting) process. We focus on two different scenarios: feature selection performed prior to the boosting process and feature selection performed inside the boosting process. Ten individual base feature ranking techniques, as well as an ensemble ranker based on the ten, are examined and compared over the two scenarios. We also employ the boosting algorithm to construct classification models without performing feature selection and use the results as the baseline for further comparison. The experimental results demonstrate that feature selection is important and needed prior to the learning process. In addition, the ensemble feature ranking method generally has better or similar performance than the average of the base ranking techniques, and more importantly, the ensemble method exhibits better robustness than most base ranking techniques. As for the two scenarios, the results show that applying feature selection inside boosting performs better than using feature selection prior to boosting.

Defect prediction is an important process activity frequently used for improving the quality and reliability of software products. Defect prediction results provide a list of fault-prone modules which are necessary in helping project managers better utilize valuable project resources. In the software quality modeling process, high dimensionality and class imbalance are the two potential problems that may exist in data repositories. In this study, we investigate three data preprocessing approaches, in which feature selection is combined with data sampling, to overcome these problems in the context of software quality estimation. These three approaches are: Approach 1 — sampling performed prior to feature selection, but retaining the unsampled data instances; Approach 2 — sampling performed prior to feature selection, retaining the sampled data instances; and Approach 3 — sampling performed after feature selection. A comparative investigation is presented for evaluating the three approaches. In the experiments, we employed three sampling methods (random undersampling, random oversampling, and synthetic minority oversampling), each combined with a filter-based feature subset selection technique called correlation-based feature selection. We built the defect prediction models using five common classification algorithms. The case study was based on software metrics and defect data collected from multiple releases of a real-world software system. The results demonstrated that the type of sampling methods used in data preprocessing significantly affected the performance of the combination approaches. It was found that when the random undersampling technique was used, Approach 1 performed better than the other two approaches. However, when the feature selection technique was used in conjunction with an oversampling method (random oversampling or synthetic minority oversampling), we strongly recommended Approach 3.

Software defect prediction suffers from the class-imbalance. Solving the class-imbalance is more important for improving the prediction performance. SMOTE is a useful over-sampling method which solves the class-imbalance. In this paper, we study about some problems that faced in software defect prediction using SMOTE algorithm. We perform experiments for investigating how they, the percentage of appended minority class and the number of nearest neighbors, influence the prediction performance, and compare the performance of classifiers. We use paired t-test to test the statistical significance of results. Also, we introduce the effectiveness and ineffectiveness of over-sampling, and evaluation criteria for evaluating if an over-sampling is effective or not. We use those concepts to evaluate the results in accordance with the evaluation criteria for the effectiveness of over-sampling.

The results show that they, the percentage of appended minority class and the number of nearest neighbors, influence the prediction performance, and show that the over-sampling by SMOTE is effective in several classifiers.

Software defect prediction technology has been widely used in improving the quality of software system. Most real software defect datasets tend to have fewer defective modules than defective-free modules. Highly class-imbalanced data typically make accurate predictions difficult. The imbalanced nature of software defect datasets makes the prediction model classifying a defective module as a defective-free one easily. As there exists the similarity during the different software modules, one module can be represented by the sparse representation coefficients over the pre-defined dictionary which consists of historical software defect datasets. In this study, we make use of dictionary learning method to predict software defect. We optimize the classifier parameters and the dictionary atoms iteratively, to ensure that the extracted features (sparse representation) are optimal for the trained classifier. We prove the optimal condition of the elastic net which is used to solve the sparse coding coefficients and the regularity of the elastic net solution. Due to the reason that the misclassification of defective modules generally incurs much higher cost risk than the misclassification of defective-free ones, we take the different misclassification costs into account, increasing the punishment on misclassification defective modules in the procedure of dictionary learning, making the classification inclining to classify a module as a defective one. Thus, we propose a cost-sensitive software defect prediction method using dictionary learning (CSDL). Experimental results on the 10 class-imbalance datasets of NASA show that our method is more effective than several typical state-of-the-art defect prediction methods.

Development without any defect is unsubstantial. Timely detection of software defects favors the proper resource utilization saving time, effort and money. With the increasing size and complexity of software, demand for accurate and efficient prediction models is increasing. Recently, search-based techniques (SBTs) have fascinated many researchers for Software Defect Prediction (SDP). The goal of this study is to conduct an empirical evaluation to assess the applicability of SBTs for predicting software defects in object-oriented (OO) softwares. In this study, 16 SBTs are exploited to build defect prediction models for 13 OO software projects. Stable performance measures — GMean, Balance and Receiver Operating Characteristic-Area Under Curve (ROC-AUC) are employed to probe into the predictive capability of developed models, taking into consideration the imbalanced nature of software datasets. Proper measures are taken to handle the stochastic behavior of SBTs. The significance of results is statistically validated using the Friedman test complied with Wilcoxon post hoc analysis. The results confirm that software defects can be detected in the early phases of software development with help of SBTs. This paper identifies the effective subset of SBTs that will aid software practitioners to timely detect the probable software defects, therefore, saving resources and bringing up good quality softwares. Eight SBTs — sUpervised Classification System (UCS), Bioinformatics-oriented hierarchical evolutionary learning (BIOHEL), CHC, Genetic Algorithm-based Classifier System with Adaptive Discretization Intervals (GA_ADI), Genetic Algorithm-based Classifier System with Intervalar Rule (GA_INT), Memetic Pittsburgh Learning Classifier System (MPLCS), Population-Based Incremental Learning (PBIL) and Steady-State Genetic Algorithm for Instance Selection (SGA) are found to be statistically good defect predictors.

Software defect prediction is an effective approach to save testing resources and improve software quality, which is widely studied in the field of software engineering. The effort-aware just-in-time software defect prediction (JIT-SDP) aims to identify defective software changes in limited software testing resources. Although many methods have been proposed to solve the JIT-SDP, the effort-aware prediction performance of the existing models still needs to be further improved. To this end, we propose a differential evolution (DE) based supervised method DEJIT to build JIT-SDP models. Specifically, first we propose a metric called density-percentile-average (DPA), which is used as optimization objective on the training set. Then, we use logistic regression (LR) to build a prediction model. To make the LR obtain the maximum DPA on the training set, we use the DE algorithm to determine the coefficients of the LR. The experiment uses defect data sets from six open source projects. We compare the proposed method with state-of-the-art four supervised models and four unsupervised models in cross-validation, cross-project-validation and timewise-cross-validation scenarios. The empirical results demonstrate that the DEJIT method can significantly improve the effort-aware prediction performance in the three evaluation scenarios. Therefore, the DEJIT method is promising for the effort-aware JIT-SDP.

Just-in-time software defect prediction (JIT-SDP) is a fine-grained software defect prediction technology, which aims to identify the defective code changes in software systems. Effort-aware software defect prediction is a software defect prediction technology that takes into consideration the cost of code inspection, which can find more defective code changes in limited test resources. The traditional effort-aware defect prediction model mainly measures the effort based on the number of lines of code (LOC) and rarely considers additional factors. This paper proposes a novel effort measure method called Multi-Metric Joint Calculation (MMJC). When measuring the effort, MMJC takes into account not only LOC, but also the distribution of modified code across different files (Entropy), the number of developers that changed the files (NDEV) and the developer experience (EXP). In the simulation experiment, MMJC is combined with Linear Regression, Decision Tree, Random Forest, LightGBM, Support Vector Machine and Neural Network, respectively, to build the software defect prediction model. Several comparative experiments are conducted between the models based on MMJC and baseline models. The results show that indicators ACC and $P_{\mathrm{\text{opt}}}$ of the models based on MMJC are improved by 35.3% and 15.9% on average in the three verification scenarios, respectively, compared with the baseline models.

Software defect prediction (SDP), which predicts defective code regions, can help developers reasonably allocate limited resources for locating bugs and prioritizing their testing efforts. Previous work on defect prediction has used machine learning and artificial software metrics. However, traditional defect prediction features extracted from artificial software metrics often fail to capture the syntactic and semantic information of defective modules. This work on defect prediction mostly focuses on abstract syntax tree (AST). Moreover, because current research on AST technology is relatively mature, it is difficult to further improve the accuracy of defect prediction when only using AST to characterize codes. In this paper, in order to capture more semantic features, we extract semantic information both from the sequences of AST tokens and code change tokens. In addition, to leverage the traditional features extracted from statistical metrics, we also combine the semantic features with traditional defect prediction features to perform SDP, and use the gated fusion mechanism to determine the combination ratio of the two kinds of features. In our empirical studies, 10 open-source Java projects from the PROMISE repository are chosen as our empirical subjects. Experimental results show that our proposed approach can perform better than several state-of-the-art baseline SDP methods.

For a new project, it is impossible to get a reliable prediction model because of the lack of sufficient training data. To solve the problem, researchers proposed cross-project defect prediction (CPDP). For CPDP, most researchers focus on how to reduce the distribution difference between training data and test data, and ignore the impact of class imbalance on prediction performance. This paper proposes a hybrid multiple models transfer approach (HMMTA) for cross-project software defect prediction. First, several instances that are most similar to each target project instance are selected from all source projects to form the training data. Second, the same number of instances as that of the defected class are randomly selected from all the non-defect class in each iteration. Next, instances selected from the non-defect classes and all defected class instances are combined to form the training data. Third, the transfer learning method called ETrAdaBoost is used to iteratively construct multiple prediction models. Finally, the prediction models obtained from multiple iterations are integrated by the ensemble learning method to obtain the final prediction model. We evaluate our approach on 53 projects from AEEEM, PROMISE, SOFTLAB and ReLink four defect repositories, and compare it with 10 baseline CPDP approaches. The experimental results show that the prediction performance of our approach significantly outperforms the state-of-the-art CPDP methods. Besides, we also find that our approach has the comparable prediction performance as within-project defect prediction (WPDP) approaches. These experimental results demonstrate the effectiveness of HMMTA approach for CPDP.

Software defect prediction can detect modules that may have defects in advance and optimize resource allocation to improve test efficiency and reduce development costs. Traditional features cannot capture deep semantic and grammatical information, which limits the further development of software defect prediction. Therefore, it has gradually become a trend to use deep learning technology to automatically learn valuable deep features from source code or relevant data. However, most software defect prediction methods based on deep learning extraction features from a single information source or only use a single deep learning model, which leads to the fact that the extracted features are not comprehensive enough to affect the final prediction performance. In view of this, this paper proposes a Hierarchical Feature Ensemble Deep Learning (HFEDL) Approach for software defect prediction. Firstly, the HFEDL approach needs to obtain three types of information sources: abstract syntax tree (AST), class dependency network (CDN) and traditional features. Then, the Convolutional Neural Network (CNN) and the Bidirectional Long Short-Term Memory based on Attention mechanism (BiLSTM+Attention) are used to extract different valuable features from the three information sources and multiple prediction sub-models are constructed. Next, all the extracted features are fused by a filter mechanism to obtain more comprehensive features and construct a fusion prediction sub-model. Finally, all the sub-models are integrated by an ensemble learning method to obtain the final prediction model. We use 11 projects in the PROMISE defect repository and evaluate our approach in both non-effort-aware and effort-aware scenarios. The experimental results show that the prediction performance of our approach is superior to state-of-the-art methods in both scenarios.

The class overlap problem refers to instances from different categories heavily overlapping in the feature space. This issue is one of the challenges in improving the performance of software defect prediction (SDP). Currently, the studies on the impact of class overlap on SDP mainly focused on within-project defect prediction and cross-project defect prediction. Moreover, the existing class overlap instances cleaning methods are not suitable for cross-version defect prediction. In this paper, we propose a class overlap instances cleaning method based on the Ratio of K-nearest neighbors with the Same Label (RKSL). This method removes instances with the abnormal neighbor ratio in the training set. Based on the RKSL method, we investigate the impact of class overlap on the performance and interpretability of the cross-version defect prediction model. The experiment results show that class overlap can affect the performance of cross-version defect prediction models significantly. The RKSL method can handle the class overlap problem in defect datasets, but it may impact the interpretability of models. Through the analysis of feature changes, we consider that class overlap instances cleaning can assist models in identifying more important features.

Automated reliability assessment is essential for systems that entail dynamic adaptation based on runtime mission-specific requirements. One approach along this direction is to monitor and assess the system using machine learning-based software defect prediction techniques. Due to the dynamic nature of software data collected, Instance-based learning algorithms are proposed for the above purposes. To evaluate the accuracy of these methods, the paper presents an empirical analysis of four different real-time software defect data sets using different predictor models.

The results show that a combination of 1R and Instance-based learning along with Consistency-based subset evaluation technique provides a relatively better consistency in achieving accurate predictions as compared with other models. No direct relationship is observed between the skewness present in the data sets and the prediction accuracy of these models. Principal Component Analysis (PCA) does not show a consistent advantage in improving the accuracy of the predictions. While random reduction of attributes gave poor accuracy results, simple Feature Subset Selection methods performed better than PCA for most prediction models. Based on these results, the paper presents a high-level design of an Intelligent Software Defect Analysis tool (ISDAT) for dynamic monitoring and defect assessment of software modules.

Machine learning methods in software engineering are becoming increasingly important as they can improve quality and testing efficiency by constructing models to predict defects in software modules. The existing datasets for software defect prediction suffer from an imbalance of class distribution which makes the learning problem in such a task harder. In this paper, we propose a novel approach by integrating Over-Bagging, static and dynamic ensemble selection strategies. The proposed method utilizes most of ensemble learning approaches called Omni-Ensemble Learning (OEL). This approach exploits a new Over-Bagging method for class imbalance learning in which the effect of three different methods of assigning weight to training samples is investigated. The proposed method first specifies the best classifiers along with their combiner for all test samples through Genetic Algorithm as the static ensemble selection approach. Then, a subset of the selected classifiers is chosen for each test sample as the dynamic ensemble selection. Our experiments confirm that the proposed OEL can provide better overall performance (in terms of G-mean, balance, and AUC measures) comparing with other six related works and six multiple classifier systems over seven NASA datasets. We generally recommend OEL to improve the performance of software defect prediction and the similar problem based on these experimental results.

Data mining techniques have been effectively used for software defect prediction in the last decade. The general process is that a classifier is first trained on historical software data (software metrics and fault data) collected during the software development process and then the classifier is used to predict new program modules (waiting for testing) as either fault-prone or not-fault-prone. The performance of the classifier is influenced by two factors: the software metrics in the training dataset and the proportions of the fault-prone and not-fault-prone modules in that dataset. When a dataset contains too many software metrics and/or very skewed proportions of the two types of modules, several problems may arise including extensive computation and a decline in predictive performance. In this paper, we use feature ranking and data sampling to deal with these problems. We investigate two types of feature ranking techniques (wrapper-based and filter-based), and compare their performances through two case studies on two groups of software measurement datasets. The empirical results demonstrate that filter-based ranking techniques not only show better classification performance but also have a lower computational cost.

Software quality prediction models are useful tools for creating high quality software products. The general process is that practitioners use software metrics and defect data along with various data mining techniques to build classification models for identifying potentially faulty program modules, thereby enabling effective project resource allocation. The predictive accuracy of these classification models is often affected by the quality of input data. Two main problems which can affect the quality of input data are high dimensionality (too many independent attributes in a dataset) and class imbalance (many more members of one class than the other class in a binary classification problem). To resolve both of these problems, we present an iterative feature selection approach which repeatedly applies data sampling (to overcome class imbalance) followed by feature selection (to overcome high dimensionality), and finally combines the ranked feature lists from the separate iterations of sampling. After feature selection, models are built either using a plain learner or by using a boosting algorithm which incorporates sampling. In order to assess the impact of various balancing, filter, and learning techniques in the feature selection and model-building process on software quality prediction, we employ two sampling techniques, random undersampling (RUS) and synthetic minority oversampling technique (SMOTE), and two ensemble boosting approaches, RUSBoost and SMOTEBoost (in which RUS and SMOTE, respectively, are integrated into a boosting technique), as well as six feature ranking techniques. We apply the proposed techniques to several groups of datasets from two real-world software systems and use two learners to build classification models. The experimental results demonstrate that RUS results in better prediction than SMOTE, and also that boosting is more effective in improving classification performance than not using boosting. In addition, some feature ranking techniques, like chi-squared and information gain, exhibit better and more stable classification behavior than other rankers.

The software is applied in various areas so the quality of the software is very important. The software defect prediction (SDP) is used to solve the software issues and enhance the quality. The robustness and reliability are the major concerns in the existing SDP approaches. Hence, in this paper, the hybrid optimization-based neural network (Optimized NN) is developed for the effective detection of the defects in the software. The two main steps involved in the Optimized NN-based SDP are feature selection and SDP utilizing Optimized NN. The data is fed forwarded to the feature selection module, where relief algorithm selects the significant features relating to the defect and no-defects. The features are fed to the SDP module, and the optimal tuning of NN classifier is obtained by the hybrid optimization developed by the integration of the social spider algorithm (SSA) and gray wolf optimizer (GWO). The comparative analysis of the developed prediction model reveals the effectiveness of the proposed method that attained the maximum accuracy of 93.64%, maximum sensitivity of 95.14%, maximum specificity of 99%, maximum $F_1$-score of 93.53%, and maximum precision of 99% by considering the K-fold.

Software development involves plenty of risks, and errors exist in software modules represent a major kind of risk. Software defect prediction techniques and tools that identify software errors play a crucial role in software risk management. Among software defect prediction techniques, classification is a commonly used approach. Various types of classifiers have been applied to software defect prediction in recent years. How to select an adequate classifier (or set of classifiers) to identify error prone software modules is an important task for software development organizations. There are many different measures for classifiers and each measure is intended for assessing different aspect of a classifier. This paper developed a performance metric that combines various measures to evaluate the quality of classifiers for software defect prediction. The performance metric is analyzed experimentally using 13 classifiers on 11 public domain software defect datasets. The results of the experiment indicate that support vector machines (SVM), C4.5 algorithm, and K-nearest-neighbor algorithm ranked the top three classifiers.