Chapter 6: A Survey of Neural Models for Abstractive Summarization

In this chapter, we survey recent developments in abstractive summarization which use neural networks. Those methods achieve state-of-the-art ROUGE results for summarization tasks, especially when using short text as a source, such as single sentences or short paragraphs. We cover essential neural network concepts for abstractive summarization models. Because such models require massive training data, we also overview datasets used to train such models. We first describe the basic methodological concepts (word embeddings, sequence to sequence recurrent networks and attention mechanism). We provide didactic source code in Python to explain these basic concepts. We then survey four recent systems which, when combined, have resulted in dramatic improvements in single-document generic abstractive summarization in the past couple of years. These systems introduce re-usable techniques which address each aspect of the summarization challenge: dealing with large vocabulary while exploiting the high similarity between source and target documents; dealing with rare named-entities by detecting and copying them from the source to the target; avoiding repetition and redundancy by introducing a distractor mechanism; introducing sentence level assessment with the use of reinforcement learning.