Analysis of word dependency relations and subword models in abstractive text summarization

Abstract

Abstractive text summarization is an important task in natural language pro cessing. As there are too many textual materials becoming available in the digital world at an unprecedented speed, people begin to need automated text summarization systems to summarize such bulk data in a condensed form that only holds the neces sary information. With recent advances in deep learning techniques, abstractive text summarization has gained even more attention. Attention-based sequence-to-sequence models are adapted for this task and achieved state-of-the-art results. On top of it, several additional mechanisms like pointer/generator and coverage were proposed and have become the standard mechanisms to be used for abstractive summarization mod els. Using these approaches, we integrated word dependency relations and analyzed their effects on the models. We showed that integrating dependency relations increases performance. Recent models for many natural language processing tasks use subwords and achieve state-of-the-art results. We utilized three different subword models in our models and analyze their effectiveness in the abstractive summarization task. We found that subword usage is another viable option to be included for abstractive summariza tion models as well.