Integration of Swarm Intelligence and Artificial Neural Network

The following sections are included:

• Preface

• Contents

This chapter discusses the recent development of swarm intelligence and neural networks. Swarm intelligence deals with the collective behavior of systems having many individuals that interact locally with each other and with their environment, and that use forms of decentralized control and self-organization to achieve their targets. Specifically, this chapter is restricted with particle swarm optimization (PSO), ant colony optimization (ACO) and bee colony optimization (BCO) techniques of swarm intelligence.

Artificial neural networks (ANNs) are known as the universal approximators and computational models with particular characteristics such as the ability to learn or adapt. In particular, it is very difficult to obtain an optimal neural network architecture for solving a complex problem by a human expert. For instance, automatic determination of the optimal number of hidden layers and hidden neurons in each (hidden) layer is the most critical task. Several earlier attempts can provide a few off-the-self solutions but the application of swarm intelligence introduce a new dimension with better solutions in terms of robustness and time. Some of the potential higher order neural networks and their possible integration with swarm intelligence is another attracting items of this chapter.

This chapter describes the classification task of data mining using nature inspired techniques such as neural network and swarm intelligence algorithm. Neural network and swarm intelligence techniques have an advantage over conventional statistical techniques like maximum likelihood classifier, nearest neighbor classifier etc, because they are distribution free, i.e., no knowledge is required about the distribution of data. The extraction of knowledge from the dataset in the form of weights and rules using various neural network and swarm intelligence methods is increasing over the last few years. In this chapter, we present the knowledge discovery process using three types of testbeds such as the iris data, the pima diabetes data and the shuttle data. The performance analysis of the classification task of data mining is done based on per class accuracy as well as overall accuracy. Finally, we evaluate the computational complexity of neural network and swarm intelligence techniques.

Ant Colony Optimization (ACO) is one of the most representative meta-heuristics derived from the broad concept known as Swarm Intelligence (SI) where the behavior of social insects is the main source of inspiration. Being a particular SI approach, the ACO metaheuristic is mainly characterized by its distributiveness, flexibility, capacity of interaction among simple agents, and its robustness. The ACO metaheuristic has been successfully applied to an important number of discrete and continuous single-objective optimization problems. However, this metaheuristic has shown a great potential to also cope with multi-objective optimization problems as evidenced by the several proposals currently available in that regard. This chapter is aimed at describing the most relevant and recent developments on the use of the ACO metaheuristic for solving multi-objective optimization problems. Additionally, we also derive a refined taxonomy of the types of ACO variants that have been used for multi-objective optimization. Such a taxonomy intends to serve as a quick guide for those interested in using an ACO variant for solving multi-objective optimization problems. In the last part of the chapter, we provide some potential paths for further research in this area.

Over the past two decades, recurrent neural networks have been shown to be very powerful computational models for optimization. A number of recurrent neural network models have been designed for solving various constrained optimization problems. Based on the Karush-Kuhn-Tucker optimality conditions and non-smooth functions, recurrent neural networks with discontinuous activation functions are designed for constrained optimization. The present chapter presents several recurrent neural networks with discontinuous activation functions for solving linear and quadratic programming problems and a class of non-smooth optimization problems. Moreover, two k-winners-take-all networks are designed based on the presented neural networks.

This chapter presents the application of Wavelet Transform (WT), one of the most frequently used signal processing techniques for the extraction of important information in different frequency sub-bands from the non-stationary Power Quality (PQ) signals. The features derived by WT are used for the classification purpose to identify the nature of the PQ disturbances. We have used the Multi Layer Feed Forward Neural Network (FNN) for the classification. Traditional learning algorithm like back propagation learning is used for the training of the FNN. Simultaneously an attempt has been made for the design of an evolvable neural network classifier using one of the most promising Swarm Intelligence algorithms known as Particle Swarm Optimization (PSO). The basic PSO algorithm was modified to make it adaptive and the proposed algorithm is known as adaptive PSO (APSO). The performance of the NN for the classification is compared for both back propagation based learning and APSO based learning.

Condition monitoring, fault diagnosis as well as Quality Assurance have become of paramount importance in various industrial sectors such as automobile industries, aeronautics industries, power generating units as well as manufacturing units. Interestingly, all these sectors can use generic solution approach to achieve a greater autonomy with quality assurance using non-parametric classification and prediction methodologies. This chapter provides a generic solution using intelligent methodologies based on statistical techniques, neural networks as well as other machine learning approaches. The methods were justified by using a real life engine data taken from one of the Two Wheeler Industry. The following applications has been investigated at length: Classifying good and bad single cylinder engines based on audio data collected during the running of the engine at various speeds and predicting possible type of the defect in engines. The chapter has been concluded by futuristic approach and vide applicability of the proposed techniques.

Image enhancement techniques are used to improve image quality by maximizing the information content in the input image using intensity transformation functions. For color image enhancement, gray-level enhancement techniques may be applied directly on R, G, B color planes separately, but in this case colors of the enhanced image is distorted as R, G, B planes are enhanced in different scales. So, perception of the original color in the image will be changed, which means hue of the input image is altered. Sometimes enhancement is done in HSI/LHS color space by processing intensity and saturation components keeping the hue unaltered to preserve the original color perception. While performing reverse transformation from these color spaces to RGB space, it usually creates gamut problem. In this article a PSO based hue preserving gamut problem free color image enhancement technique is proposed. The process is as follows. Image enhancement is considered as an optimization problem and particle swarm optimization (PSO) is used to solve it. The three color components are used to construct an intensity image. The contrast of the intensity image is enhanced by a parameterized transformation function, in which parameters are optimized by PSO, based on an objective function. The transformation function uses local and global information of the input image and the objective function considers the entropy and edge information to measure the image quality. We tried to achieve the best enhanced image according to the objective criterion. The enhanced R, G and B components are obtained from enhanced intensity image by scaling. Scaling may produce gamut problem for few pixels in the enhanced color image. Pixels with the gamut problem are transformed to HSI color space and rescaling is done to the saturation component to remove the gamut problem. Afterwards the pixel is again transformed to RGB space. The algorithm is tested on several color images and the results are compared with two other popular color image enhancement techniques like hue-preserving color image enhancement without gamut problem (HPCIE) and a genetic algorithm based approach to color image enhancement (GACIE). Visual analysis, detail and background variance of the resulted images are reported. It has been found that the proposed method produce better results than the other two compared methods.

This chapter presents the procedure for designing classifier model using polynomial neural network (PNN). The PNN is a flexible neural architecture. The number of layers of the PNN is not fixed in advance but it is developed on the fly. This network is also considered as a self-organizing network. The essence of the design procedure of PNN dwells on the group methods of data handling (GMDH) technique. Each node of PNN exhibits a high level of flexibility and realizes a polynomial mapping between input and output variables. This chapter presents how artificial neural network (ANN) is incorporated with PNN to enhance the performance substantially. During the study of PNN, it is observed that a huge amount of time is consumed during training the classifier and the developed model is complex in nature. To alleviate these drawbacks the growth of PNN layers is restricted but the approximation capability is maintained by incorporating ANN. Bench mark databases of different domain have been taken to test the performance of this hybrid model. The simulation results of PNN and its hybrid models are compared, which shows that the performance gain obtained in the hybrid model is much better in comparison to the PNN model.

The chapter develops an efficient adaptive prediction model using recently developed differential evolution (DE) technique. The forecasting model employs an adaptive linear combiner architecture and DE based learning rule to predict seasonally adjusted (SA) and non seasonally adjusted (NSA) sales data for short and long ranges. The prediction performance of proposed model is assessed through simulation study and using real life data. For comparison purpose the corresponding results are also obtained using genetic algorithm (GA), bacterial foraging optimization (BFO) and particle swarm optimization (PSO) based forecasting models. It is, in general, observed that the new DE forecasting model offers fastest training, best sales prediction and least mean square error after training compared to other three evolutionary computing based models.

This chapter deals with particle swarm optimization (PSO)(i.e., one of the components of swarm intelligence) for single and multi-objective problems. In PSO, an initial swarm propagates in the design space towards the optimal solution in cooperation of members of the swarm by updating velocity and position of each member. A considerable number of algorithms have been and are being proposed for single and multi-objective problems based on either tuning or introducing various parameters of PSO. These algorithms have there own merits and demerits. Further, since these algorithms are scattered in various journals, proceedings and book chapters, therefore it is a herculean task for a reader to assemble in one place and learn. Hence, this study can reduce the effort and time of the reader to digest the various approaches of PSO. Additionally, we identify some application areas, where PSO has given a clear edge over other meta heuristic approaches for solving single and multi-objective optimization problems. This can motivate and lead the readers for extensive application of PSO in new domains of interest. The last part of the chapter concludes with a discussion and future research scope followed by an extensive set of relevant references.

Microarray technology is a powerful tool for geneticists to monitor interactions among tens of thousands of genes simultaneously. There has been extensive research on coherent subspace clustering of gene expressions measured under consistent experimental settings. However, these methods assume that all experiments are run using the same batch of microarray chips with similar characteristics of noise. Algorithms developed under this assumption may not be applicable for analyzing data collected from heterogeneous settings, where the set of genes being monitored may be different and expression levels may not be directly comparable even for the same gene. In this chapter, we propose a biclustering model that imitates the ecosystem taking into account the features of biological data for mining subspace coherent patterns from heterogeneous gene expression data. We have implemented the system using an Ant colony algorithm (ACO). The algorithm decides the number of bi-clusters automatically. This processes the input biological data, runs the ACO, assigns biclusters to the genes and conditions simultaneously and displays the output.

The following sections are included:

• Author Index

• Subject Index