Computational Models of Cognitive Processes

The following sections are included:

• Preface
• Contents

Multimodal integration is a central characteristic of human cognition. However our understanding of the interaction between modalities and its influence on behaviour is still in its infancy. This paper examines the value of the Hub & Spoke framework as a tool for exploring multimodal interaction in cognition. We present a Hub and Spoke model of language–vision information interaction and report the model's ability to replicate a range of phonological, visual and semantic similarity word-level effects reported in the Visual World Paradigm. The model provides an explicit connection between the percepts of language and the distribution of eye gaze and demonstrates the scope of the Hub-and-Spoke architectural framework by modelling new aspects of multimodal cognition.

In this study, we model human letter-recognition times using neural networks that extract visual features from real images of the letters. We focus on learning, and on how different learning methods and other factors affect the correlation between simulated reaction times and behavioural data. Specifically, we are interested in studying the effect of 3 factors on this correlation: (i) utilisation of an error signal during learning (supervised vs. unsupervised learning), (ii) whether or not the letter labels exert a top-down influence on the extracted features, and (iii) the effect of letter frequencies. To do so, we used Restricted Boltzmann Machines (RBMs), Back-propagation networks, and RBM/Perceptron hybrid architectures. We find the highest correlations (r = 0.67) with supervised models when using top-down information of letter labels on the feature layer during training, but only when the letters' frequencies are taken into account during learning. This study shows that to account for human letter identification times, letter frequency seems to be the most important factor. In addition, top down information of letter labels on the extracted visual features appears to be essential (making the difference between a significant and nonsignificant correlation). Whether or not the model is supervised makes little difference in the correlation to human reaction time data, but fully unsupervised models have more difficulty generating accurate categorisation for letters with very low frequencies.

To understand the brain mechanisms underlying language phenomena, and sentence construction in particular, a number of approaches have been followed that are based on artificial neural networks, where words are encoded as distributed patterns of activity. Still, issues like the distinct encoding of semantic vs syntactic features, word binding, and the learning processes through which words come to be encoded that way, have remained tough challenges. We explore a novel approach to address these challenges, which focuses first on encoding words of an artificial language of intermediate complexity (BLISS) into a Potts attractor net. Such a network has the capability to spontaneously latch between attractor states, offering a simplified cortical model of sentence production. The network stores the BLISS vocabulary, and hopefully its grammar, in its semantic and syntactic subnetworks. Function and content words are encoded differently on the two subnetworks, as suggested by neuropsychological findings. We propose that a next step might describe the self-organization of a comparable representation of words through a model of a learning process.

In Polynesian phonology and morphology, passives have traditionally been analyzed in two steps: (1) by assigning a single default pattern, which has been characterized as fully predictable and productive; and (2) by lexicalizing the remaining patterns, which have been assumed to be unpredictable. We overturn this assumption of unpredictability here for the Hawaiian passive by finding a computational model capable of predicting more than just the single default. This is achieved by learning statistical regularities over active verbs which predict their passive forms. Whether Hawaiian speakers are able to make use of these model predictions in a productive way can now be tested behaviorally.

Zipf's law states that the frequency of a word in a corpus is inversely proportional to its rank. The extension of Zipf's law to n-grams is an interesting and novel subject. We investigate the difference between spoken and written language based on an n-gram version of Zipf's law. N-grams are similar to n-th order hyperedges in a hypernetworks model. We find that clear and definite differences exist between the spoken and written language in 1- and 2-gram results. Also, the fitting parameter for the exponent of the rank varies between the spoken and the written corpora.

As a simple demonstration of action and language integration in cognitive systems, we have compared the speeds of reading (saying) words in two experimental conditions. One condition was reading normally, i.e. reading aloud. The other was reading with your silent inner voice without making any sounds, i.e. your brain just simulates reading words without using your mouth, your tongue, nor your throat. We found reading aloud is significantly quicker than reading silently. This means that cognitive performance is facilitated with the existence of action to read.

Recently, Dynamic Neural Field models have shed light on the flexible and dynamic processes underlying young children's emergent categorisation and word learning (DNF; e.g., Spencer & Schöner). DNF models are a distinct class of neural network in which perceptual features can be represented topologically and time continuously, complementing existing connectionist models of cognitive development by building category representations that are available for inspection at any given stage in learning. Recent research in infant categorization and word learning has demonstrated that young children's ability to learn and generalise labels for novel object categories is profoundly affected by the perceptual variability of the to-be-learned category. We have captured these data in a DNF model of children's category label learning. Given a known vocabulary, our model exploits mutual exclusivity via simple associative processes to map novel labels to novel categories, and is able to retain and generalize these newly-formed mappings. The model was used to generate the testable prediction that children's generalizations of novel category labels should be contingent on the number and closeness of objects' perceptual neighbours. We present a replication of this prediction, via an empirical study with 30-month-old children. In line with the model, children were only able to generalize novel words to completely novel objects when those objects were central to the just-encountered category, rather than peripheral. This empirical replication demonstrates the predictive validity of DNF models when applied to cognitive development. Further, the data suggest that children's ability to categorise and learn labels is not a conceptually-based, stepwise phenomenon, but rather a graded, emergent process. As such, these data add weight to associative, dynamic systems approaches to understanding language learning, categorisation, and cognition more generally.

It was shown that a simple connectionist autoencoder was able to simulate the asymmetric categorization effect reported by Quinn, Eimas and Rosenkrantz in 3- to 4- month-old infants. Moreover, other studies have reported that a careful control of the variance distribution and inclusion relationship of the inputs (meaning that a broader category subsumed a narrower visual category) is sufficient to reverse or even suppress the asymmetry. In the current paper, we show that variance distribution and the inclusion relationship are not sufficient to produce the asymmetry. We show that (i) correlation information reported at the level of the 10 features previously encoded for connectionist simulations was actually very weak and (ii) if we increase these correlations among the different features, we completely suppress the asymmetry categorization effect, even with the same distribution and inclusion relationship that were assumed to produce the asymmetry.

Neural network models of early motor speech vocal learning are reviewed, with a focus on those models that utilize reinforcement to modulate what would otherwise be self-organized learning. It is argued that such a mechanism likely plays a role in bringing about the changes observed in prespeech vocalizations produced by human infants. Such models complement well the already popular purely self-organized learning models that focus on effects of exposure to sounds from the ambient language.

The Headturn Preference Procedure (HPP) is a frequently used method (e.g., Jusczyk & Aslin; and subsequent studies) to investigate linguistic abilities in infants. In this paradigm infants are usually first familiarised with words and then tested for a listening preference for passages containing those words in comparison to unrelated passages. Listening preference is defined as the time an infant spends attending to those passages with his or her head turned towards a flashing light and the speech stimuli. The knowledge and abilities inferred from the results of HPP studies have been used to reason about and formally model early linguistic skills and language acquisition. However, the actual cause of infants' behaviour in HPP experiments has been subject to numerous assumptions as there are no means to directly tap into cognitive processes. To make these assumptions explicit, and more crucially, to understand how infants' behaviour emerges if only general learning mechanisms are assumed, we introduce a computational model of the HPP. Simulations with the computational HPP model show that the difference in infant behaviour between familiarised and unfamiliar words in passages can be explained by a general learning mechanism and that many assumptions underlying the HPP are not necessarily warranted. We discuss the implications for conventional interpretations of the outcomes of HPP experiments.

The aim of this study is to analyze the effects of right otitis media on language development in the first three years of a child's life. A study of middle-latency evoked potentials was carried out to observe the correlation between right otitis media history and language disorders. We tested two groups of children, both aged between 3 and 6.11 years old. The first group had a history of right otitis media in the earliest years of life (n=12) and the second group had a history of left otitis media, had not suffered otitis or had suffered from otitis but made a full recovery (n=17). Several battery language tests were applied to assess the children's language abilities. Auditory brainstem-evoked responses and middle-latency evoked potentials were employed to establish if plastic changes had occurred after prolonged periods of hearing loss in the auditory path and the cerebral cortex. The fact of being a case is related to lower scores in phonetic, phonological and morphosyntax tasks. There is a correlation between the phonetic and phonological system and the auditory brainstem evoked response with the energy of the middle latency response.

Rogers et al. present a model of semantic cognition – the “hub” model – that reproduces the behaviour of neurologically healthy and neurologically impaired individuals on a range of tests of semantic knowledge. The model and associated theory provide a comprehensive explanation for deficits, such as semantic dementia, by appealing to the breakdown of attractors within a recurrently connected system following damage. We report findings from an attempted replication of the Rogers et al. model. While normal behaviour was reproduced, lesioning the reimplementation did not fully replicate the behaviour of the original model, meaning that the reimplementations contain healthy semantic representations which are in line with the hub theory, but the effects of damage on the structure of the semantic representations are not theoretically accounted for. The hub theory predicts that after damage semantic representations must decay in certain ways in order to give rise to patient behaviour. Our results show that the reimplementations do not fully exhibit these symptoms. This suggests that while semantic impairments reminiscent of patients may arise following lesioning of the hub model, such patterns are not a necessary consequence of the model as initially described. We discuss the implication of this apparently negative result for the hub theory of semantic cognition, focusing on differences between our reimplementation and the implementation of Rogers et al., and on the theory-model relationship more generally.

We used an Echo State Network (ESN) to investigate cognitive control tasks that involve temporal and policy abstraction, namely an n-back task and a Wisconsin Card Sorting task. In the task conditions where those abstractions were most relevant, separating the abstraction levels increased ESN performance, while the performance was reduced by such a separation in the conditions where the abstractions were less important. This result gives a possible explanation for the topological mapping of abstraction levels over the prefrontal cortex, because of wiring economy in the brain.

Interactive activation networks have a long pedigree in the cognitive sciences, and have been widely used to illustrate phenomena of conceptual processing and memory. However, despite numerous proposals regarding their relevance, there are few implementations of interactive activation networks for simulating cognitive processing in problem solving tasks. This chapter discusses potential obstacles to applying such models to problem solving, accounting for this historical lack, and points to ways in which actual implementations can circumvent these obstacles and lead to insights into not only problem solving but also properties of semantic memory more generally.

Many of the tasks we perform during our everyday lives are achieved through sequential execution of a set of goal-directed actions. Quite often these actions are organized hierarchically, corresponding to a nested set of goals and subgoals. Several computational models address the hierarchical execution of goaldirected actions by humans. However, the neural learning mechanisms supporting the temporal clustering of goal-directed actions in a hierarchical structure remain to a large extent unexplained. In this paper we investigate in simulations, of a dynamic neural field (DNF) model, biologically-based learning and adaptation mechanisms that can provide insight into the development of hierarchically organized internal representations of naturalistic tasks. In line with recent experimental evidence from observational learning studies, the DNF model implements the idea that prediction errors play a crucial role for grouping fine-grained events into larger units. Our ultimate goal is to use the model to endow the humanoid robot ARoS with the capability to learn hierarchies in sequential tasks, and to use that knowledge to enable efficient collaborative joint actions with human partners. For testing the ability of the system to deal with the real-time constraints of a learning-by-demonstration paradigm we use the same assembly task from our previous work on human-robot collaboration. The model provides some insights on how hierarchically structured task representations can be learned and on how prediction errors made by the robot and signaled by the demonstrator can be used to control this process.

Autonomous learning is the ability to learn effectively without much external assistance. An important strength of autonomous learners is that they can shape their own learning and development, in large part by choosing which problems to learn. Such choices include selecting a problem to learn and deciding whether to continue learning on that selected task or abandon it in favor of something else. We extend a constructive neural-learning algorithm, sibling-descendant cascade-correlation, to monitor lack of progress in learning so that unproductive learning can be abandoned. Learning is abandoned when network error fails to change by more than a specified threshold for a specified number of consecutive learning cycles. Here we explore the space defined by these threshold and patience parameters on problems of different degrees of learnability. Our results simulate findings from recent experiments with infants who abandon learning on difficult tasks and focus their attention on tasks of moderate difficulty.

A computational model is presented which draws upon previous research on prefrontally mediated executive functioning to investigate resource allocation in response to increased task demand. Central to the model is the process of monitoring task demand through detection of cognitive conflict. The model is shown to account for performance in a stimulus matching task in which participants are matching stimuli that are presented in the same or opposite visual hemi-field. The model captures the within-hemispheric advantage in both young and older adults and the between-hemisphere advantage when the cognitive demand increases. We conclude that this model provides a first step in the direction of unpacking the mechanisms underlying cognitive and brain reserve capacity.

Humans are often able to adapt their behavior, based on experience, so as to optimize the likelihood of obtaining future rewards. These reinforcement learning (RL) phenomena have been well captured by computational models employing temporal difference (TD) learning. TD learning typically involves an “actor-critic architecture” (AC), in which an “adaptive critic” module learns to predict future reward. These models are proven to converge to optimal solutions in simple RL environments; however the proof assumes a discrete environment. The “value function” (VF) in the AC needs to be approximated in continuous and very large discrete environments. Traditional “value function approximators” (VFAs) such as artificial neural networks with back-propagation, while having proven fruitful, are not reliable in complex or continuous environments, often times never converging to any solution. Headway has been made in using complex neural network implementations for the VFA; for some problems, convergence to a bounded region of solutions has been proven when using an “echo state network” (ESN), a type of “reservoir computing”. More general spiking neural networks have been used to implement an AC architecture. Other work has been done suggesting that “spiketiming-dependent plasticity” (STDP) may be a possible underlying mechanism for computing temporal differences of reward. None of this work has addressed the problem that VFAs in AC architectures fail to learn in a general way in complex environments. We report on simulations involving spiking neural networks including liquid state machines applied to the problem with VFAs in such complex environments demonstrating the benefits and pitfalls using the temporal dynamics of spikes to encode the continuous state value information.

A plethora of words are used to describe the spectrum of human emotions, but how many distinct emotions exist, and how do they interact? Over the past few decades, several theories of emotion have been proposed, each founded upon a set of basic emotions, and each supported by an extensive variety of research including studies in facial expression, ethology, neurology and physiology. Here we propose a theory that people transmit their understanding of emotions through the language they use that surrounds mentioned emotion keywords. Using a labelled corpus of over 21,000 tweets, six of the basic emotion sets proposed in existing literature were analysed using Latent Semantic Clustering (LSC) to propose the distinctiveness of the semantic meaning attached to the emotional label. We hypothesise that the more distinct language is used to express a certain emotion, then the more distinct the perception (including proprioception) of that emotion is, and thus more basic. This allows us to select the dimensions best representing the entire spectrum of emotion. We find that Ekman's set, arguably the most frequently used for classifying emotions, is the most semantically distinct. Next, taking all analysed (that is, previously proposed) emotion terms into account, we determine the optimal semantically irreducible basic emotion set using an iterative LSC algorithm. Our newly-derived set (Accepting, Ashamed, Contempt, Interested, Joyful, Pleased, Sleepy, Stressed) generates a 6.1% increase in distinctiveness over Ekman's set (Angry, Disgusted, Joyful, Sad, Scared).

A new core hypothesis on laughter is presented. It has been built by putting together ideas from several disciplines: neurodynamics, evolutionary neurobiology, paleoanthropology, social networks, and communication studies. The hypothesis contributes to ascertain the evolutionary origins of human laughter in connection with its cognitive and emotional signalling functions within social groups. The new behavioural and neurodynamic tenets introduced about this unusual sound feature of our species justify the ubiquitous presence it has in social interactions and along the life cycle of the individual. Laughter, far from being a curious evolutionary relic or a rather trivial innate behaviour, should be considered as a highly efficient tool for inter-individual problem solving and for maintenance of social bonds. In spite of a number of recent studies, links between different classes of laughter and emotional conditions of individuals have not been clearly established yet. There is also a lack of research about the presence of emotional patterns in spontaneous laughing sounds, which could provide keys to a subsequent classification. There are no studies in psychiatry attempting the use of laughter as a diagnostic tool either. In order to advance laughter research in these aspects, the present study has extracted nine temporal and acoustic variables from laughter episodes, looking for establishing better pattern-classification methods. These variables are number of bouts, voiced percentage, bout duration, mean energy, mean entropy, standard deviation of fundamental frequency, jitter, shimmer and harmonic to noise ratio. A library of records of laughter episodes was created and the values of the previous variables were statistically analyzed. In spite of the variability of acoustical parameters, there were significant differences which might be associated to the different laugh types. In future works, these statistical outcomes will be contrasted with the evaluations by human subjects.