International Journal of Software Engineering and Knowledge Engineering

Feature selection for supervised learning concerns the problem of selecting a number of important features (w.r.t. the class labels) for the purposes of training accurate prediction models. Traditional feature selection methods, however, fail to take the sample distributions into consideration which may lead to poor prediction for minority class examples. Due to the sophistication and the cost involved in the data collection process, many applications, such as biomedical research, commonly face biased data collections with one class of examples (e.g., diseased samples) significantly less than other classes (e.g., normal samples). For these applications, the minority class examples, such as disease samples, credit card frauds, and network intrusions, are only a small portion of the data but deserve full attention for accurate prediction. In this paper, we propose three filtering techniques, Higher Weight (HW), Differential Minority Repeat (DMR) and Balanced Minority Repeat (BMR), to identify important features from datasets with biased sample distribution. Experimental comparisons with the ReliefF method on five datasets demonstrate the effectiveness of the proposed methods in selecting informative features for accurate prediction of minority class examples.

In this paper, a new framework to build an adaptive classifier is introduced. At first, a clustering algorithm, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is applied to a set of sample data to form initial set of clusters. The clusters are represented as classes. Using support vector machine (SVM), a classifier model is generated. In real world application, data comes in continuously. Therefore, if the model does not learn from the new data, the model may not perform as well with the new data especially when the model's training data is different from the test data. The new framework proposed in this paper rebuilds the classifier model using selected data from test data set to improve the accuracy of the model. A case study on intrusion detection data set has been performed to evaluate our methodology. The result shows that this approach lead to have more accurate classification models over time.

Natural disasters, such as hurricanes, have had an enormous social and economical impact on society in the United State and around the world for many years. With the goal of preventing, diverting, or weakening the destructive forces of tropical cyclones, the preparedness of the public plays a major role in the magnitude of inflicted damage due to these storms. Acknowledging the captivating power of social networking and Web 2.0 over society, we present a prototype system which integrates meteorological data along with user generated content with the aim of improving public response by increasing their situational awareness due to such natural threats. The proposed system aggregates storm track and wind analysis data from the existing H*Wind system along with videos taken from YouTube and presents it to the user in Google Earth. A content-based concept detection mechanism is used to evaluate the relevance of the extracted YouTube videos to the storm of interest. The proposed system demonstrates the potential public benefit resulting from the integration of the areas of multimedia content analysis, Web 2.0, and meteorology.

Class imbalance is a fundamental problem in data mining and knowledge discovery which is encountered in a wide array of application domains. Random undersampling has been widely used to alleviate the harmful effects of imbalance, however, this technique often leads to a substantial amount of information loss. Repetitive undersampling techniques, which generate an ensemble of models, each trained on a different, undersampled subset of the training data, have been proposed to allieviate this difficulty. This work reviews three repetitive undersampling methods currently used to handle imbalance and presents a detailed and comprehensive empirical study using four different learners, four performance metrics and 15 datasets from various application domains. To our knowledge, this work is the most thorough study of repetitive undersampling techniques.

The goal of Gene Normalization (GN) is to identify the unique database IDs of genes and proteins mentioned in biomedical literature. A major difficulty in GN comes from the ambiguity of gene names. That is, the same gene name can refer to different database IDs depending on the species in question. In this paper, we introduce a method to exploit contextual information in an abstract, like tissue type, chromosome location, etc., to tackle this problem. Using this technique, we have been able to improve system performance (F-score) by 14.3% on the BioCreAtIvE-II GN task test set. We also examined our method on a full-text dataset with cross-species genes. The experimental results show a promising performance (AUC) of 42.94%. Our experimental results also show that with full text, versus abstract only, the system performance was 12.24% higher.

In the last few years, several works in the literature have addressed the problem of data extraction from web pages. The importance of this problem derives from the fact that, once extracted, data can be handled in a way similar to instances of a traditional database, which in turn can facilitate application of web data integration and various other domain specific problems. In this paper, we propose a novel table extraction technique that works on web pages generated dynamically from a back-end database. The proposed system can automatically discover table structure by relevant pattern mining from web pages in an efficient way, and can generate regular expression for the extraction process. Moreover, the proposed system can assign intuitive column names to the columns of the extracted table by leveraging Wikipedia knowledge base for the purpose of table annotation. To improve accuracy of the assignment, we exploit the structural homogeneity of the column values and their co-location information to weed out less likely candidates. This approach requires no human intervention and experimental results have shown its accuracy to be promising. Moreover, the wrapper generation algorithm works in linear time.

To develop Web applications of high quality, it is important to apply efficient frameworks to standardize the process of development in projects, or apply useful design patterns to produce the program code that can easily be enhanced. In addition to these efforts, we have to check the programs to see whether they keep various kinds of rules such as verification items for security which are common for all kinds of Web applications, and verification items for coding styles or code conventions which pertain to each project. This paper proposes a verification method for an extensible syntax-oriented verifier, and describes the implementation of its prototype system, SyntaxVerifier. SyntaxVerifier makes it possible to detect syntactical objects based on their syntactic contexts. It realizes a context-dependent recursive verification which makes it easy to dynamically trace a syntax tree in a verification process itself.

Proof scores are documents of comprehensible plans to prove theorems. The proof score approach to systems analysis is a method in which proof scores are used to verify that systems enjoy properties (or analyze systems). In this paper, we describe a way to analyze electronic commerce protocols with the proof score approach, which has been developed and refined through several case studies conducted.