Narrow Results

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.

When working with real-world applications we often find imbalanced datasets, those for which there exists a majority class with normal data and a minority class with abnormal or important data. In this work, we make an overview of the class imbalance problem; we review consequences, possible causes and existing strategies to cope with the inconveniences associated to this problem. As an effort to contribute to the solution of this problem, we propose a new rule induction algorithm named Rule Extraction for MEdical Diagnosis (REMED), as a symbolic one-class learning approach. For the evaluation of the proposed method, we use different medical diagnosis datasets taking into account quantitative metrics, comprehensibility, and reliability. We performed a comparison of REMED versus C4.5 and RIPPER combined with over-sampling and cost-sensitive strategies. This empirical analysis of the REMED algorithm showed it to be quantitatively competitive with C4.5 and RIPPER in terms of the area under the Receiver Operating Characteristic curve (AUC) and the geometric mean, but overcame them in terms of comprehensibility and reliability. Results of our experiments show that REMED generated rules systems with a larger degree of abstraction and patterns closer to well-known abnormal values associated to each considered medical dataset.

Data Mining is a powerful technology to help organization to concentrate on most important data by extracting useful information from large database. One of the most commonly used techniques in data mining is Artificial Neural Network due to its high performance in many application domains. Despite many advantages of Artificial Neural Network, one of its main drawbacks is its inherent black box nature which is the main problem of using Artificial Neural Network in data mining. Therefore, this paper proposes a rule extraction algorithm from neural network using classified and misclassified data to convert the black box nature of Artificial Neural Network into a white box. The proposed algorithm is a modification of the existing algorithm, Rule Extraction by Reverse Engineering (RxREN). The proposed algorithm extracts rules from trained neural network for datasets with mixed mode attributes using pedagogical approach. The proposed algorithm uses both classified as well as misclassified data to find out the data ranges of significant attributes in respective classes, which is the innovation of the proposed algorithm. The experimental results clearly show that the performance of the proposed algorithm is superior to existing algorithms.

Neural networks are good at representing functions or data transformations. However just as in the case of the biological brain the mathematical description of the data transformation is hidden. In the case of the human brain the transformation, in terms of rules, may be extracted by interviewing the person, In the case of the artificial neural network other approaches have to be utilized.

In the case described here a second neural network that represents the transformation in terms of fuzzy rules is trained using gradient descent. The parameters that are learned are the parameters of the fuzzy sets and also the connection weights in [0,1] between the outputs of the membership function units and the final output units. There is an output unit for each rule and consequent membership function. The fuzzy output set with the highest membership value is taken to be the output fuzzy set. The extracted rules are of the form if x₀ is Small or x₀ is Medium and x₁ is Large or x₁ is Medium then y is Large. x₀ and x₁ are inputs and y is the output.

The cost measure consists of several terms indicating how close the actual output is to a target output, how close the weights are to 0 and 1, and how close the output of membership values is to a 1 of n vector. The cost measure is a linear combination of these individual terms. By changing the constant multipliers the relative importance of the cost measures can be changed and studied.

The method has been tried on randomly generated feedforward neural networks and also on data produced by functions with specific properties. The fizzy network is trained using data produced by the feedforward neural network or the known function. This method can also be used in extracting rules such as control rules implicitly used by a human if input and output data is gathered from the human.

Fuzzy logic programming has been lately used as a general framework for representing and handling imprecise knowledge. In this paper, we define the syntax and the semantics of definite weighted fuzzy logic programs, which extend definite fuzzy logic programs by allowing the inclusion of different significance weights in the individual atoms that make up the antecedent of a fuzzy logic rule. The weights add expressiveness to a fuzzy logic program and allow the determination of the level up to which an atom in the antecedent of a rule may affect the truth value of its consequent. In describing the semantics of definite weighted fuzzy logic programs we introduce the notion of the generalized weighted fuzzy conjunction operator, which can be regarded as a weighted t-norm based aggregation. We determine the properties of generalized weighted fuzzy conjunction operators and provide several examples. A methodology for constructing generalized weighted fuzzy conjunction operators using generator functions of existing t-norms is also introduced. Finally, a method for setting up a parametric weighted fuzzy logic program and automatically adapting the weights of its rules using a numerical dataset is developed.

Symbolically representing the knowledge acquired by a neural network is a profound endeavor aimed at illuminating the latent information embedded within the network. The literature offers a multitude of algorithms dedicated to extracting symbolic classification rules from neural networks. While some excel in producing highly accurate rules, others specialize in generating rules that are easily comprehensible. Nevertheless, only a scant few algorithms manage to strike a harmonious balance between comprehensibility and accuracy. One such exemplary technique is the Rule Extraction from Neural Network Using Classified and Misclassified Data (RxNCM) algorithm, which adeptly generates straightforward and precise rules outlining input data ranges with commendable accuracy. This article endeavors to enhance the classification performance of the RxNCM algorithm by leveraging ensemble technique. Ensembles, a burgeoning field, focus on augmenting classifier performance by harnessing the strengths of individual classifiers. Extraction of rules through neural network ensembles is relatively underexplored, this paper bridges the gap by introducing the Rule extraction using Neural Network Ensembles (RENNE) algorithm. RENNE is designed to refine the classification rules derived from the RxNCM algorithm through ensemble strategy. Specifically, RENNE leverages patterns correctly predicted by an ensemble of neural networks during the rule generation process. The efficacy of the algorithm is validated using seven datasets sourced from the UCI repository. The outcomes indicate that the proposed RENNE algorithm outperforms the RxNCM algorithm in terms of performance.

Extracting rules from RBFs is not a trivial task because of nonlinear functions or high input dimensionality. In such cases, some of the hidden units of the RBF network have a tendency to be "shared" across several output classes or even may not contribute to any output class. To address this we have developed an algorithm called LREX (for Local Rule EXtraction) which tackles these issues by extracting rules at two levels: hREX extracts rules by examining the hidden unit to class assignments while mREX extracts rules based on the input space to output space mappings. The rules extracted by our algorithm are compared and contrasted against a competing local rule extraction system. The central claim of this paper is that local function networks such as radial basis function (RBF) networks have a suitable architecture based on Gaussian functions that is amenable to rule extraction.

A method for extracting Zadeh–Mamdani fuzzy rules from a minimalist constructive neural network model is described. The network contains no embedded fuzzy logic elements. The rule extraction algorithm needs no modification of the neural network architecture. No modification of the network learning algorithm is required, nor is it necessary to retain any training examples. The algorithm is illustrated on two well known benchmark data sets and compared with a relevant existing rule extraction algorithm.

This paper presents theoretical and historical backgrounds related to neural network rule extraction. It also investigates approaches for neural network rule extraction by ensemble concepts. Bologna pointed out that although many authors had generated comprehensive models from individual networks, much less work had been done to explain ensembles of neural networks. This paper carefully surveyed the previous work on rule extraction from neural network ensembles since 1988. We are aware of three major research groups i.e., Bologna' group, Zhou' group and Hayashi' group. The reason of these situations is obvious. Since the structures of previous neural network ensembles were quite complicated, the research on the efficient rule extraction algorithm from neural network ensembles was few although their learning capability was extremely high. Thus, these issues make rule extraction algorithm for neural network ensemble difficult task. However, there is a practical need for new ideas for neural network ensembles in order to realize the extremely high-performance needs of various rule extraction problems in real life. This paper successively explain nature of artificial neural networks, origin of neural network rule extraction, incorporating fuzziness in neural network rule extraction, theoretical foundation of neural network rule extraction, computational complexity of neural network rule extraction, neuro-fuzzy hybridization, previous rule extraction from neural network ensembles and difficulties of previous neural network ensembles. Next, this paper address three principles of proposed neural network rule extraction: to increase recognition rates, to extract rules from neural network ensembles, and to minimize the use of computing resources. We also propose an ensemble-recursive-rule extraction (E-Re-RX) by two or three standard backpropagation to train multi-layer perceptrons (MLPs), which enabled extremely high recognition accuracy and the extraction of comprehensible rules. Furthermore, this enabled rule extraction that resulted in fewer rules than those in previously proposed methods. This paper summarizes experimental results of rule extraction using E-Re-RX by multiple standard backpropagation MLPs and provides deep discussions. The results make it possible for the output from a neural network ensemble to be in the form of rules, thus open the "black box" of trained neural networks ensembles. Finally, we provide valuable conclusions and as future work, three open questions on the E-Re-RX algorithm.

In this paper, we present an approach for sample selection using an ensemble of neural networks for credit scoring. The ensemble determines samples that can be considered outliers by checking the classification accuracy of the neural networks on the original training data samples. Those samples that are consistently misclassified by the neural networks in the ensemble are removed from the training dataset. The remaining data samples are then used to train and prune another neural network for rule extraction. Our experimental results on publicly available benchmark credit scoring datasets show that by eliminating the outliers, we obtain neural networks with higher predictive accuracy and simpler in structure compared to the networks that are trained with the original dataset. A rule extraction algorithm is applied to generate comprehensible rules from the neural networks. The extracted rules are more concise than the rules generated from networks that have been trained using the original datasets.

Mortality rate due to fatal heart disease (HD) or cardiovascular disease (CVD) has increased drastically over the world in recent decades. HD is a very hazardous problem prevailing among people which is treatable if detected early. But in most of the cases, the disease is not diagnosed until it becomes severe. Hence, it is requisite to develop an effective system which can accurately diagnosis HD and provide a concise description for the underlying causes [risk factors (RFs)] of the disease, so that in future HD can be controlled only by managing the primary RFs. Recently, researchers are using various machine learning algorithms for HD diagnosis, and neural network (NN) is one among them which has attracted tons of people because of its high performance. But the main obstacle with a NN is its black-box nature, i.e., its incapability in explaining the decisions. So, as a solution to this pitfall, the rule extraction algorithms can be very effective as they can extract explainable decision rules from NNs with high prediction accuracies. Many neural-based rule extraction algorithms have been applied successfully in various medical diagnosis problems. This study assesses the performance of rule extraction algorithms for HD diagnosis, particularly those that construct rules recursively from NNs. Because they subdivide a rule’s subspace until the accuracy improves, recursive algorithms are known for delivering interpretable decisions with high accuracy. The recursive rule extraction algorithms’ efficacy in HD diagnosis is demonstrated by the results. Along with the significant data ranges for the primary RFs, a maximum accuracy of 82.59% is attained.

In the current work, we consider the applicability of Ant Colony Systems (ACS) to the bankruptcy prediction problem. ACS are nature-based algorithms that mimic the functions of live organisms to find the best performing solution. In our work, ACS are used for the extraction of classification rules for bankruptcy prediction. An experimental study was conducted in order to evaluate the performance of the system and identify well performing parameters. Results were compared to the performance obtained by state-of-the-art methods for classification, namely the Artificial Neural Networks, the Support Vector Machines, the Partial Decision Trees and the Fuzzy Lattice Reasoning. Comparison indicates the high performance of the ACS which is further supported by their ability to extract classification rules, thus offering interpretation of the prediction results. The latter is of great importance in the field of corporate distress where no unified theory on distress prediction exists. Most studies with distress prediction have focused on increasing the accuracy of the model and have not always paid attention to the model interpretation.

In real life, due to the complexity of objective things and the ambiguity of human thinking, people are often accustomed to expressing them with fuzzy linguistic values. To solve the decision-making problem with uncertain information of fuzzy linguistic values, this paper proposes a rule extraction method based on linguistic 3-tuple concept lattice. Introducing linguistic 3-tuple into formal context, the linguistic 3-tuple formal context and linguistic 3-tuple formal concept are proposed. Based on the linguistic 3-tuple formal context, we put forward the linguistic 3-tuple decision formal context and the algorithm of rule extraction based on linguistic 3-tuple concept lattice. Finally, the effectiveness and practicability of this model are illustrated by an example of student competition prediction system.

As an effective tool for analyzing human behavior and decision cognition, rule extraction is one of the important steps of knowledge discovery. In order to make decision with high confidence level and improve the rate of information acquisition in an uncertainty environment, this paper establishes a rule extraction algorithm of fuzzy linguistic concept knowledge under the fuzzy linguistic concept decision formal context. Introduce the weak consistence relationship in the decision context first. And then define the finer relationship between the conditional concept lattice and the decision concept lattice to obtain the consistent relationship between the fuzzy linguistic concept knowledge. Further, mine the implicit rules and their confidence degrees in the decision-making environment. Finally, taking the financial decision-making as an example to illustrate the effectiveness and practicability.

Neural network ensembles have made an impressive contribution in a number of different medical domains. Like simple neural network models, the neural network ensembles are known as 'black boxes' since how the outputs are produced is not obvious. Due to this limitation these techniques are not widely used by medical professionals. This paper first provides a short review of the different neural network rule extraction techniques. Then it describes a novel approach, namely "RDC-ANNE" that is designed to extract useful explanations from several combined neural network classifiers. The methodology employed utilises a dataset made available to us from a kidney transplant database. The dataset embodies a number of important properties, which make it a good starting point for the purpose of this research. Results reveal that this approach can be used to identify and extract the regions in the data space that have positive impact on the system performance, provide useful explanations from several combined neural networks and enhance the overall utility of current neural network models.

In recent years, extracting useful information from enterprise data and subsequently making sense of the extracted knowledge are IT (information technology) activities of utmost importance to many organizations. Frequently, the extracted knowledge is represented in the form of rules. This chapter describes a hybrid approach that integrates rough sets, tabu search, and genetic algorithms (GAs) for extracting rules from enterprise data for maintenance. The intensification and diversification strategies of tabu search are embedded in a GA search engine, in a bid to facilitate rule extraction. A case study on the maintenance of bridge cranes in an organization was used to illustrate the effectiveness of the proposed hybrid approach. The extracted rules appear to be reasonable. The details of the hybrid approach, the results of a comparative study between a traditional GA search engine and a tabu-enhanced GA search engine, and the details of the case study are presented.

Most methods of fuzzy rule based system identification either ignore feature analysis or do it in a separate phase. In this chapter we propose a novel neuro-fuzzy system that can simultaneously do feature analysis and system identification in an integrated manner. It is a five-layered feed-forward network for realizing a fuzzy rule based system. The second layer of the net is the most important one, which along with fuzzification of the input also learns a modulator function for each input feature. This enables online selection of important features by the network. The system is so designed that learning maintains the non-negative characteristic of certainty factors of rules. The proposed method is tested on both synthetic and real data sets and the performance is found to be quite satisfactory.

We first discuss the importance of making a controller interpretable and give an overview of the existing models and structures for that purpose. We then summarise our approach to designing fuzzy controllers based on the B-spline model by learning. Too large number of rules will not only result in the over-fitting problem, but also the lost of interpretability of the model. By using an optimal partition algorithm and using linguistic modificators like "between", "at most", "at least" etc., the rule base can be reduced to the minimum. We tested this approach in different benchmark problems and achieved a rule compression ratio till 71%. In this way, the readability of a rule base is significantly improved.

We propose a method for fuzzy rule generation directly from numerical data for designing classifiers. First a fuzzy partition is imposed on the domain of each feature, which results in a set of fuzzy values for each feature. Then a descriptor-pattern table is constructed using the training data and the fuzzy feature values. Rules are now discovered from the descriptor-pattern table. The rule generation process finds the distinct descriptors to discover simple rules and if required generates further rules using conjunction of common descriptors or conjunction of common descriptors and negation of distinct descriptors. A rule minimization process is then initiated to retain a small set of rules adequate to learn the training data. We suggest three possible schemes for generation of the initial fuzzy partitioning of the feature space and a genetic algorithm based tuning scheme is used to refine the rule base. Finally, the proposed scheme is tested on some real data. Unlike, most of the classifiers, the proposed method can detect ambiguous data and declare them to be unclassified - this is a distinct advantage.

Prostate cancer is a common malignancy among men, necessitating accurate and timely diagnosis at an early stage. With the advent of Artificial Intelligence (AI) technologies in the health field, support vector machines (SVMs) as one of the most well-known machine learning methods have been widely applied for prostate cancer detection. They have good generalization performances but no interpretability on the learned patterns, which bring difficulties for health professionals to understand the inner working of the predictive model. In this paper, we aim to build a computer aided diagnostic tool for prostate cancer using the SVMs where rule extraction is enabled. Experimental results on a real-world prostate cancer dataset collected in a Hong Kong hospital show that the proposed model not only had the ability for rule generation but also achieved better prediction results compared with decision tree, exhibiting a potential to assist physicians with clinical decision support in future.