Narrow Results

This paper defines the truncated normalized max product operation for the transformation ofstates of a network and provides a method for solving a set of equations based on this operation. The operation serves as the transformation for the set of fully connected units in a recurrent network that otherwise might consist of linear threshold units. Component values of the state vector and ouputs of the units take on the values in the set {0, 0.1, …, 0.9, 1}. The result is a much larger state space given a particular number of units and size of connection matrix than for a network based on threshold units. Since the operation defined here can form the basis of transformations in a recurrent network with a finite number of states, fixed points or cycles are possible and the network based on this operation for transformations can be used as an associative memory or pattern classifier with fixed points taking on the role of prototypes. Discrete fully recurrent networks have proven themselves to be very useful as associative memories and as classifiers. However they are often based on units that have binary states. The effect of this is that the data to be processed consisting of vectors in ℜⁿ have to be converted to vectors in {0, 1}^m with m much larger than n since binary encoding based on positional notation is not feasible. This implies a large increase in the number of components. The effect can be lessened by allowing more states for each unit in our network. The network proposed demonstrates those properties that are desirable in an associative memory very well as the simulations show.

The main contribution of this paper is the development of an Integer Recurrent Artificial Neural Network (IRANN) for classification of feature vectors. The network consists both of threshold units or perceptrons and of counters, which are non-threshold units with binary input and integer output. Input and output of the network consists of vectors of natural numbers that may be used to represent feature vectors. For classification purposes, representatives of sets are stored by calculating a connection matrix such that all the elements in a training set are attracted to members of the same training set. The class of its attractor then classifies an arbitrary element if the attractor is a member of one of the original training sets. The network is successfully applied to the classification of sugar diabetes data, credit application data, and the iris data set.

Fully recurrent networks have proven themselves to be very useful as associative memories and as classifiers. However, they are generally based on units that have binary states. The effect of this is that data to be processed consisting of vectors in have to be converted to vectors in {0, 1}^m with m much larger than n since binary encoding based on positional notation is not feasible. This implies a large increase in number of components. This effect can be lessened by allowing more states for each unit in our network.

This paper describes two effective learning algorithms for a network whose units take the dot product of the input with a weight vector, followed by a tanh transformation and a discretization transformation in the form of rounding or truncation. The units have states that are in {0, 0.1, 0.2, …, 0.9, 1} rather than in {0, 1} or {-1, 1}. The result is a much larger state space given a particular number of units and size of connection matrix. Two convergent learning algorithms for training such a network to store fixed points or attractors are proposed. The network exhibits those properties that are desirable in an associative memory such as limit cycles of 1, attraction to the closest attractor and few transitions required to reach attractors. Since memories that are stored can be used to represent prototypes of patterns the network is useful for pattern classification. A pattern to be classified would be entered and its class would be the same as the class of the prototype to which it is attracted to which it is.

This paper argues that mechanisms underlying consciousness and qualia are likely to arise from the information processing that takes place within the detailed micro-structure of the cerebral cortex. It looks at two key issues: how any information processing system can recognize its own activity; and secondly, how this behavior could lead to the subjective experience of qualia. In particular, it explores the pattern processing capabilities of attractor networks, and the way that they can attribute meaning to their input patterns and goes on to show how these capabilities can lead to self-recognition. The paper suggests that although feedforward processing of information can be effective without attractor behavior, when such behavior is initiated, it would lead to self-recognition in those networks involved. It also argues that attentional mechanisms are likely to play a key role in enabling attractor behavior to take place. The paper explores the ability of attractor networks to generate representations of the meaning they assign to input patterns. It goes on to show how the way that they interpret representations of their own activity could give rise to qualia. The paper includes an examination of some limited neurobiological evidence that supports the theory outlined.