Auto-Regressive Extractive Summarization with Replacement

Tianyu Zhu, Wen Hua, Jianfeng Qu, Saeid Hosseini, Xiaofang Zhou

Research output: Journal article publicationJournal articleAcademic researchpeer-review

2 Citations (Scopus)


Auto-regressive extractive summarization approaches determine sentence extraction probability conditioning on previous decisions by maintaining a partial summary representation. Despite its popularity, the framework has two main drawbacks: 1) the partial summary representation is irresolutely denoted by a weighted summation of all the processed sentences without any filtering, resulting in a noisy representation and degrading the effectiveness of extracting subsequent sentences; 2) earlier sentences are biased towards a higher extraction probability due to the sequential nature of sequence tagging. To address these two problems, we propose the Auto-regressive Extractive Summarization with Replacement (AES-Rep), a novel auto-regressive extractive summarization model. In particular, the AES-Rep model consists of two main modules: the extraction decision module that determines whether a sentence should be extracted, and the replacement locater module that enables extracted deficient sentences to be replaced with latter sentences by comparing their expressiveness with respect to the main idea of the document. These modules update the partial summary with explicit actions using elaborated multidimensional guidance. We conduct extensive experiments on the benchmark CNN and DailyMail datasets. Experimental results show that AES-Rep can achieve better performance compared with various strong baselines in terms of multiple ROUGE metrics.

Original languageEnglish
Pages (from-to)2003-2026
JournalWorld Wide Web
Publication statusPublished - Dec 2022
Externally publishedYes


  • Auto-regressive model
  • Extractive summarization
  • Lead bias
  • Partial extraction discrepancy

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems


Dive into the research topics of 'Auto-Regressive Extractive Summarization with Replacement'. Together they form a unique fingerprint.

Cite this