Chapter 1: Introduction

One of the most important challenges for modern computer science is how to emulate the remarkable cognitive capabilities of the mammalian brain; particularly in situations where there is a necessity for rapidly selecting among multiple possible interpretations for sensory data (cf. Fig. 1.1). At first sight the “machine learning” techniques that are currently used to emulate the information processing capabilities of the mammalian brain would appear to lack any coherent mathematical foundation. It has long been thought [1] that “deep” artificial neural networks (DNNs), which can be either deterministic or probabilistic, might provide a pathway to emulating the cognitive capabilities of the mammalian brain. However, it turns out that these techniques are better suited to interpolation than extrapolation. Moreover, the Herculean efforts typically required to train neural networks for complex reinforcement learning (RL) [2] or feedback control [3] problems have given pause to the assumption that deep neural networks (DNNs) are the best path forward. An alternative view is that probability theory [4–6], and Bayes’s theorem [5–9], provide the best framework for automating pattern recognition and decision-making…