Narrow Results

This paper presents a revisiting, with developments, of the so-called kinetic theory for active particles, with the main focus on the modeling of nonlinearly additive interactions. The approach is based on a suitable generalization of methods of kinetic theory, where interactions are depicted by stochastic games. The basic idea consists in looking for a general mathematical structure suitable to capture the main features of living, hence complex, systems. Hopefully, this structure is a candidate towards the challenging objective of designing a mathematical theory of living systems. These topics are treated in the first part of the paper, while the second one applies it to specific case studies, namely to the modeling of crowd dynamics and of the immune competition.

This paper proposes a systems approach to social sciences based on a mathematical framework derived from a generalization of the mathematical kinetic theory and of theoretical tools of game theory. Social systems are modeled as a living evolutionary ensemble composed of many individuals, who express specific strategies, cooperate, compete and might aggregate into groups which pursue a common interest. A critical analysis on the complexity features of social system is developed and a differential structure is derived to provide a general framework toward modeling. Then, a case study shows how the systems approach is applied. Moreover, it is shown how the theory leads to the interpretation and use of the so-called big data. Finally some research perspectives are brought to the attention of readers.

This paper addresses some preliminary steps toward the modeling and qualitative analysis of swarms viewed as living complex systems. The approach is based on the methods of kinetic theory and statistical mechanics, where interactions at the microscopic scale are nonlocal, nonlinearly additive and modeled by theoretical tools of stochastic game theory. Collective learning theory can play an important role in the modeling approach. We present a kinetic equation incorporating the Cucker–Smale flocking force and stochastic game theoretic interactions in collision operators. We also present a sufficient framework leading to the asymptotic velocity alignment and global existence of smooth solutions for the proposed kinetic model with a special kernel. Analytic results on the global existence and flocking dynamics are presented, while the last part of the paper looks ahead to research perspectives.

This paper focuses on Herbert A. Simon’s visionary theory of the Artificial World. The artificial world evolves over time as a result of various actions, including interactions with the external world as well as interactions among its internal components. This paper proposes a mathematical theory of the conceptual framework of the artificial world. This goal requires the development of new mathematical tools, inspired in some way by statistical physics and stochastic game theory. The mathematical theory is applied in particular to the study of the dynamics of organizational learning, where cooperation and competition evolve through decomposition and recombination of organizational structures; the effectiveness of the evolutionary changes depends on the dynamic prevalence of cooperative over selfish behaviors, showing features common to the evolution of all living systems.

Regional Knowledge is useful in identifying patterns of relationships between variables, and it is particularly important in solving constrained global optimization problems. However, regional knowledge is generally unavailable prior to the optimization search. The questions here are: 1) Is it possible for an evolutionary system to learn regional knowledge during the search instead of having to acquire it beforehand? and 2) How can this regional knowledge be used to expedite evolutionary search? This paper defines regional schemata to provide an explicit mechanism to support the acquisition, storage and manipulation of regional knowledge. In a Cultural Algorithm framework, the belief space "contains" a set of these regional schemata, arranged in a hierarchical architecture, to enable the knowledge-based evolutionary system to learn regional knowledge during the search and apply the learned knowledge to guide the search. This mechanism can be used to guide the optimization search in a direct way, by "pruning" the infeasible regions and "promoting" the promising regions. Engineering problems with nonlinear constraints are tested and the results are discussed. It shows that the proposed mechanism is potential to solve complicated non-linear constrained optimization problems, and some other hard problems, e.g. the optimization problems with "ridges" in landscapes.

This paper describes a learning/adaptive approach to automatically building knowledge bases for information extraction from text based web pages. A frame based representation is introduced to represent domain knowledge as knowledge unit frames. A frame learning algorithm is developed to automatically learn knowledge unit frames from training examples. Some training examples can be obtained by automatically parsing a number of tabular web pages in the same domain, which greatly reduced the amount of time consuming manual work. This approach was investigated on ten web sites of real estate advertisements and car advertisements and nearly all the information was successfully extracted with very few false alarms. These results suggest that both the knowledge unit frame representation and the frame learning algorithm work well, domain specific knowledge bases can be learned from training examples, and the domain specific knowledge base can be used for information extraction from flexible text-based semi-structured Web pages on multiple Web sites. The investigation of the knowledge representation on five other domains suggests that this approach can be easily applied to other domains by simply changing the training examples.

Latent class analysis is a popular statistical learning approach. A major challenge for learning generalized latent class is the complexity in searching the huge space of models and parameters. The computational cost is higher when the model topology is more flexible. In this paper, we propose the notion of dominance which can lead to strong pruning of the search space and significant reduction of learning complexity, and apply this notion to the Generalized Latent Class (GLC) models, a class of Bayesian networks for clustering categorical data.

GLC models can address the local dependence problem in latent class analysis by assuming a very general graph structure. However, The flexible topology of GLC leads to large increase of the learning complexity. We first propose the concept of dominance and related theoretical results which is general for all Bayesian networks. Based on dominance, we propose an efficient learning algorithm for GLC. A core technique to prune dominated models is regularization, which can eliminate dominated models, leading to significant pruning of the search space. Significant improvements on the model.

In this paper a new learning scheme for SAT is proposed. The originality of our approach arises from its ability to achieve clause learning even if no conflict occurs. This kind of learning from successes clearly contrasts with all the traditional learning approaches which generally refer to conflict analysis. To make such learning possible, relevant clauses, taken from the satisfied part of the formula are conjointly used with the classical implication graph to derive new and more powerful reasons for the implication of a given literal. Based on this extension a first learning scheme called Learning for Dynamic Assignments Reordering (LDAR) is proposed. It exploits the new derived reasons to dynamically reorder partial assignments. Experimental results show that the integration of LDAR within a state-of-the-art SAT solver achieves interesting improvements particularly on satisfiable instances.

"Smart Terrain" is an efficient algorithm used in many games, in which objects that meet needs transmit signals to non-player characters with those needs, influencing the character to move towards those objects. We describe how probabilistic reasoning can be added to this algorithm, enabling an object to broadcast that it "may" meet a need with a given probability. These probabilities are used to estimate an expected distance to an object that meets a need, allowing non-player characters to make plausible decisions about which direction to move towards in an uncertain environment. We also show how the algorithm can factor in learned knowledge of whether an object actually does meet a need, as well as how to revert to the original probabilities for objects not recently explored. We also describe a hierarchical system for realistic exploration of a complex world.

This article tackles a significant aspect of the problem of scheduling personal individual activities, that is, the generation of qualitative, significantly different alternative plans. Solving this problem is important for intelligent calendar applications, since average users cannot adequately express their preferences over the way their activities should be scheduled in time, thus it is common that they are not satisfied by the plans generated for them by a scheduler, although they are near-optimal according to their stated preferences. Hence generating alternative plans and asking the user to select one among them is a sensible approach, provided that the alternative plans are both near-optimal, according to the user-defined preferences, as well as significantly different to each other, in order to increase the chances that at least one of them satisfies the user. Furthermore, based on the assumption that a user might systematically misweight his preferences over the various aspects of a plan, an online non-intrusive method to learn his actual preferences is presented, based on monitoring his selections over the alternative plans. The proposed methods have been evaluated successfully on a variety of problems. Furthermore, they have been implemented in two deployed systems.

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

Visualizing the decision making procedure of a deep neural network is one of the main challenges towards transparent and trustworthy artificial intelligence. This paper presents an approach which extracts latent variables from a trained network and, through clustering, constructs a set of anchors that represent the network’s data driven knowledge. This set is then used to inform users about the features that create network’s decision.

A popular approach to training feed-forward nets is to treat the problem of adaptation as a function approximation and to use curve fitting techniques. We discuss here the problems which the use of pure curve fitting techniques entail for the generalization capability and robustness of the net. These problems are in general inherently associated with the use of pure supervised learning techniques. We argue that a better approach to the training of feed-forward nets is to use adaptive techniques that combine properties of both supervised and unsupervised learning. A new formulation of the training problem is presented here. According to this formulation the net is viewed as two coupled sub-nets the first of which is trained by an unsupervised learning technique and the second by a supervised one. The same formulation gives rise to analytic expressions of the goals of the adaptation and leads to a new method for the adaptation of feed-forward nets.

There are a large number of highly complex, apparently disparate, Artificial Intelligence (AI) algorithms for planning and learning. These entail the (often tedious) construction of specialized data and control structures. In this article, we present Call-Graph Caching (CGC) as an organizing principle for many of these methods. CGC is the preservation of the trace of a computational process for subsequent reuse; CGC allows the operation of highly efficient, but unintuitive, AI algorithms to be recast as far simpler recursive processes. Thus, we shall describe simple recursive constructions that, together with CGC, provide new motivations and derivations of certain classical AI planning and learning algorithms.

The Version Space Controlled Genetic Algorithms (VGA) uses the structure of the version space to cache generalizations about the performance history of chromosomes in the genetic algorithm. This cached experience is used to constrain the generation of new members of the genetic algorithms population. The VGA is shown to be a specific instantiation of a more general framework, Autonomous Learning Elements (ALE). The capabilities of the VGA system are demonstrated using the Boole problem suggested by Wilson [Wilson 1987]. The performance of the VGA is compared to that of decision trees and genetic algorithms. The results suggest that the VGA is able to exploit a certain set of symbiotic relationships between its components, so that the resulting system performs better than either component individually.

A study of the function approximation capabilities of single hidden layer neural networks strongly motivates the investigation of constructive learning techniques as a means of realizing established error bounds. Learning characteristics employed by constructive algorithms provide ideas for development of new algorithms applicable to the function approximation problem. In addition, constructive techniques offer efficient methods for network construction and weight determination. The development of a novel neural network algorithm, the Constructive Locally Fit Sigmoids (CLFS) function approximation algorithm, is presented in detail. Basis functions of global extent (piecewise linear sigmoidal functions) are locally fit to the target function, resulting in a pool of candidate hidden layer nodes from which a function approximation is obtained. This algorithm provides a methodology of selecting nodes in a meaningful way from the infinite set of possibilities and synthesizes an n node single hidden layer network with empirical and analytical results that strongly indicate an O(1/n) mean squared training error bound under certain assumptions. The algorithm operates in polynomial time in the number of network nodes and the input dimension. Empirical results demonstrate its effectiveness on several multidimensional function approximate problems relative to contemporary constructive and nonconstructive algorithms.

This article is a survey of recent advances on multilayer neural networks. The first section is a short summary on multilayer neural networks, their history, their architecture and their learning rule, the well-known back-propagation. In the following section, several theorems are cited, which present one-hidden-layer neural networks as universal approximators. The next section points out that two hidden layers are often required for exactly realizing d-dimensional dichotomies. Defining the frontier between one-hidden-layer and two-hidden-layer networks is still an open problem.

Several bounds on the size of a multilayer network which learns from examples are presented and we enhance the fact that, even if all can be done with only one hidden layer, more often, things can be done better with two or more hidden layers. Finally, this assertion 'is supported by the behaviour of multilayer neural networks in two applications: prediction of pollution and odor recognition modelling.

Given a fuzzy logic system, how can we determine the membership functions that will result in the best performance? If we constrain the membership functions to a certain shape (e.g., triangles or trapezoids) then each membership function can be parameterized by a small number of variables and the membership optimization problem can be reduced to a parameter optimization problem. This is the approach that is typically taken, but it results in membership functions that are not (in general) sum normal. That is, the resulting membership function values do not add up to one at each point in the domain. This optimization approach is modified in this paper so that the resulting membership functions are sum normal. Sum normality is desirable not only for its intuitive appeal but also for computational reasons in the real time implementation of fuzzy logic systems. The sum normal constraint is applied in this paper to both gradient descent optimization and Kalman filter optimization of fuzzy membership functions. The methods are illustrated on a fuzzy automotive cruise controller.

The Bayesian Network Retrieval Model is able to represent the main (in)dependence relationships between the terms from a document collection by means of a specific type of Bayesian network, namely a polytree. However, although the learning and propagation algorithms designed for this topology are very efficient, in collections with a very large number of terms, these two tasks might be very time-consuming. This paper shows how by reducing the size of the polytree, which will only comprise one subset of terms which are selected according to their retrieval quality, the performance of the model is maintained, whereas the efforts needed to learn and later propagate in the model are considerably reduced. A method for selecting the best terms, based on their inverse document frequency and term discrimination value, is also presented.

We introduce and study a new concept of fuzzy computing units. This construct is is aimed at coping with "negative" (inhibitory) information and accommodating it in the language of fuzzy sets. The essential concept developed in this study deals with computing units exploiting the concept of balanced fuzzy sets. We recall how the membership notion of fuzzy sets can be extended to the [-1,1] range giving rise to balanced fuzzy sets and then summarize properties of augmented (extended) logic operations for these constructs. We show that this idea is particularly appealing in neurocomputing as the "negative" information captured through balanced fuzzy sets exhibits a straightforward correspondence with inhibitory processing mechanisms encountered in neural networks. This gives rise to interesting properties of balanced computing units when compared with fuzzy and logic neurons developed on the basis of classical logic and classical fuzzy sets. Illustrative examples concerning topologies and properties and learning of balanced fuzzy computing units are included. A number of illustrative examples concerning topologies, properties and learning of balanced fuzzy fuzzy computing units are included.