Towards Human Brain Inspired Lifelong Learning

The following sections are included:

• About the Editors
• About the Contributors
• Contents
• List of Figures
• List of Tables

The following sections are included:

• Introduction
• Lifelong Learning
• Continual Learning
• Online Learning in Open- and Closed-World Environments

Continual learning is the ability of a learning system to solve new tasks by utilizing previously acquired knowledge from learning and performing prior tasks without having significant adverse effects on the acquired prior knowledge. Numerous approaches were developed to achieve this ability while avoiding the well-known problem of catastrophic forgetting in neural networks. This chapter reviews a number of architectural approaches to continual learning in neural networks which tackle the problem by modifying the architecture of the neural network, for example, by adding new adaptive parameters to the model for each new task. The architectural paradigm for continual learning in neural networks can potentially completely eliminate the problem of catastrophic forgetting and maintain competitive performance, often at the expense of increased computational complexity.

In this work, we endeavor to investigate and propose a novel unsupervised online learning algorithm, namely the online restricted Boltzmann machine (O-RBM). The O-RBM is able to construct and adapt the architecture of a restricted Boltzmann machine (RBM) artificial neural network according to the statistics of the streaming input data. Specifically, for training data that are not fully available at the onset of training, the proposed O-RBM begins with a single neuron in the hidden layer of the RBM and progressively adds and suitably adapts the network to account for the variations in streaming data distributions. Such unsupervised learning helps to effectively model the probability distribution of the entire data stream and generate robust features. We demonstrate that such unsupervised representations can be used for discriminative classifications on a set of multi-category and binary classification problems for unstructured image and structured signal data sets with varying degrees of class imbalance. We first demonstrate the O-RBM algorithm and characterize the network evolution using the simple and conventional multi-class MNIST image data set, aimed at recognizing handwritten digits. We then benchmark the O-RBM’s performance against that of other machine learning, neural network and ClassRBM techniques using a number of public non-stationary data sets. Finally, we study the performance of the O-RBM on a real-world problem involving the predictive maintenance of an aircraft component and ocean environment prediction using time series data sets. In all these studies, it is observed that the O-RBM converges to a stable, concise network architecture, wherein individual neurons are inherently discriminative to the class labels despite unsupervised training. It can be observed from the performance results that, on average, the O-RBM improves accuracy by 2.5–3% over conventional offline batch learning techniques while requiring at least 24–70% fewer neurons.

This chapter describes a general Bayesian framework for the lifelong learning of artificial neural networks that can handle catastrophic forgetting in a principled way. The framework can be applied to both discriminative and generative models as well as task-aware and task-agnostic settings. We introduce the variational continual learning algorithm, a realization of this framework that uses online variational inference with a small amount of memory or coreset for effective lifelong learning. We examine various practical considerations when using this algorithm and show that it performs competitively against other lifelong learning approaches on different benchmarks. We also discuss several improvements to the algorithm and outline some future research directions for Bayesian lifelong learning.

Zero-shot learning (ZSL) is a new paradigm to classify objects from classes that are not available at training time. ZSL methods have attracted considerable attention in recent years because of their ability to classify unseen/novel class examples. Most of the existing approaches on ZSL work when all the samples from seen classes are available to train the model, which does not suit real life. In this chapter, we tackle this hindrance by developing a generative replay-based continual ZSL (GRCZSL). The proposed method endows traditional ZSL with the ability to learn from streaming data and acquire new knowledge without forgetting the previous tasks’ gained experience. We handle catastrophic forgetting in GRCZSL by replaying the synthetic samples of seen classes which have appeared in the earlier tasks. These synthetic samples are synthesized using the trained conditional variational autoencoder (VAE) over the immediate past task. Moreover, we only require the current and immediately previous VAE at any time for training and testing. The proposed GRZSL method is developed for a single-head setting of continual learning, simulating a real-world problem setting. In this setting, task identity is given during training but unavailable during testing. The GRCZSL performance is evaluated on five benchmark datasets for the generalized setup of ZSL with fixed and dynamic (incremental class) settings of continual learning. The existing class setting presented recently in the literature is not suitable for a class-incremental setting. Therefore, this study proposes a new setting to address this issue. Experimental results show that the proposed method significantly outperforms the baseline and the state-of-the-art method and is more suitable for real-world applications.

In recent years, we have witnessed a renewed interest in machine learning methodologies, especially for deep representation learning, that could overcome basic i.i.d. assumptions and tackle non-stationary environments subject to various distributional shifts or sample selection biases. Within this context, several computational approaches based on architectural priors, regularizers and replay policies have been proposed with different degrees of success depending on the specific scenario in which they were developed and assessed. However, designing comprehensive hybrid solutions that can be flexibly and generally applied with tunable efficiency–effectiveness trade-offs still seems like a distant goal. In this chapter, we propose “Architect, Regularize and Replay” (ARR), a hybrid generalization of the renowned AR1 algorithm and its variants, which can not only achieve state-of-the-art results in classic scenarios (e.g., class-incremental learning) but also generalize to arbitrary data streams generated from real-world datasets, such as CIFAR-100, CORe50 and ImageNet-1000.

Continual learning (CL) aims to learn new tasks through the forward transfer of information learned from previous tasks without forgetting them. In task-incremental CL, task information is vital during both strategy development and inference. Providing such partial knowledge about the test sample demands additional complexity and may become intractable, especially when the sample source is ambiguous. In this work, we design a task-agnostic inference approach that uses a base–child hybrid setup to incrementally learn tasks while mitigating forgetting. Multiple base classifiers guided by reference points learn new tasks, and this information is distilled via a feature space-induced sampling strategy. A central child classifier consolidates information across tasks and infers the task identifier automatically. Experimental results on standard datasets show that the proposed approach outperforms the various state-of-the-art regularization and replay CL algorithms in terms of accuracy by 50% and 7% with homogeneous and heterogeneous tasks, respectively, in task-agnostic scenarios.

Deep neural networks model data for a task or a sequence of tasks, where the knowledge extracted from the data is encoded in the parameters and representations of the network. Extraction and utilization of these representations are vital when data are no longer available in the future, especially in a continual learning scenario. Flashcards are visual representations that capture the encoded knowledge of a network as a recursive function of some predefined random image patterns. In a continual learning scenario, flashcards help prevent catastrophic forgetting by consolidating the knowledge of all the previous tasks. Flashcards are required to be constructed only before learning the subsequent task and are hence independent of the number of tasks trained previously, making them task-agnostic. The efficacy of flashcards in capturing learned knowledge representation (as an alternative to the original data) is empirically validated on a variety of continual learning tasks: reconstruction, denoising and task-incremental classification, using several heterogeneous (varying background and complexity) benchmark datasets. Experimental evidence indicates that flashcards as a replay strategy are task-agnostic, perform better than generative replay and are on par with episodic replay without additional memory overhead.

This chapter discusses the challenges of machine learning models that need to learn from multiple sources due to regulations and confidentiality in practical domains such as security, finance and medical image analysis. This chapter presents a practical use case of continual learning using X-ray image classification in medical image analysis and demonstrates that continual learning is a helpful tool for data security through model sharing across multiple institutions/organizations. The chapter proposes an elastic weight consolidation (EWC)-based continual learning framework for X-ray image classification and an OOD detection framework using latent space features of the trained model. The proposed approach is experimentally validated using two X-ray image datasets, CheXpert and MIMIC-CXR, and provides a reliable and generalizable clinical decision support tool for X-ray image classification. The key elements of the study consist of continual learning for handling forward data transfer and minimizing catastrophic forgetting, the data drift analysis method and OOD detection across datasets and an experimental validation of the proposed framework.

The following sections are included:

• Bibliography
• Index