Single-Document Abstractive Text Summarization: A Systematic Literature Review
Article No.: 60, Pages 1 - 37
Abstract
Abstractive text summarization is a task in natural language processing that automatically generates the summary from the source document in a human-written form with minimal loss of information. Research in text summarization has shifted towards abstractive text summarization due to its challenging aspects. This study provides a broad systematic literature review of abstractive text summarization on single-document summarization to gain insights into the challenges, widely used datasets, evaluation metrics, approaches, and methods. This study reviews research articles published between 2011 and 2023 from popular electronic databases. In total, 226 journal and conference publications were included in this review. The in-depth analysis of these papers helps researchers understand the challenges, widely used datasets, evaluation metrics, approaches, and methods. This article identifies and discusses potential opportunities and directions along with a generic conceptual framework and guidelines on abstractive summarization models and techniques for research in abstractive text summarization.
1 Introduction
The information explosion and the advancement of technology have raised the importance of making critical decisions by reading numerous documents. However, reading many documents and producing summaries has become tedious and endless. Here is where the text summarization comes into the picture. Text summarization is an important domain in Natural Language Processing (NLP) and Information Retrieval (IR). Text summarization was proposed in the 1950s by Luhn [155], who introduced an approach to extract prominent sentences from text using attributes such as word and phrase frequency. Edmundson et al. [62] introduced a method in which key phrases are extracted based on word frequencies. The authors introduced cue-, location-, and title-based methods to extract these words. Later, several studies contributed to a significant advancement in the field of abstractive text summarization [68, 72, 203, 217]. Since then, many methods have been introduced to address text summarization problems.
The text-summarization task summarizes the document in a condensed format by preserving vital information with minimal data loss. Text summarization is an effective way to reduce irrelevant information when reading documents. Text summarization is very difficult, because when humans summarize a text, the entire text is read to develop insight and then reproduce a summary highlighting its main points. Because machines lack a greater depth of human knowledge and linguistic capability, abstractive text summarization poses a challenging and nontrivial task for machines. Inspired by this problem, several researchers have developed models that automatically generate summaries from documents.
Text summarization is categorized based on the summary generation, application, and input document type. Based on summary generation, text summarization can be categorized as extractive, abstractive, and hybrid summarization. Extractive summarization focuses on extracting information directly from a source document without paraphrasing it. In this type of summarization, the extracted text document depends on the source document structure. Abstractive summarization generates short summaries by rewriting an entire document in a human-written manner. Summaries generated by humans are typically abstractive. The goal of abstractive text summarization is more complex than extractive text summarization, because it requires a model to comprehend the input text and produce a summary that is not constrained by the input’s existing sentences. Hybrid text summarization is a type of summarization that combines several methods or approaches. They can be extractive, abstractive, or both. This summarization leverages the advantages of the different types of summarization to generate accurate summaries. Recent research has focused more on abstractive text summarization, because it is more challenging than extractive text summarization [80].
According to the user’s application, different summarization methods are used to generate summaries. This includes query-focused, generic, and controlled summarization. Query-focused summarization generates summaries according to the user’s query and is used in applications such as question answering systems and search engines. Studies such as References [23, 178, 273] have focused on query-focused summarization and its application. In contrast, generic summarization generates a summary based on general content without focusing on specific content. However, this summarization is useful when a user requires a general document overview. Controlled summarization [65, 88] is another type of summarization method that generates a summary based on predetermined rules that control the generated summary length, tone, or style. In addition, there are a few variants of controlled summarization [36, 60] in which the constraint is facilitated to provide better control over the summaries generated.
Reinforcement learning-based summarization effectively generates informative, fluent, and cohesive summaries. This method yields accurate and contextually relevant summaries and is particularly useful for summarizing news articles. Aspect-based summarization [53] focuses on specific aspects of the context. For example, when a product in context needs to be known, aspect-based summarization focuses on the precise details of the product, such as features, designs, and performance. It helps the user quickly understand the product’s weaknesses and positive aspects. Opinion summarization [8, 26] effectively summarizes general sentiments and attitudes from vast text, which contains text data from customer reviews, social media posts, and online comments. It is helpful for various purposes, including market research, customer feedback analysis, and social media monitoring, as it enables businesses to swiftly spot patterns and trends in the attitudes and opinions of their target markets. Knowledge-based summarization is a method that summarizes a given text using external knowledge sources, such as knowledge graphs, ontologies, or domain-specific knowledge bases. Knowledge-based summarization is useful when generating domain-specific summaries, such as technical summaries from specific topics, such as medical documents.
Text summarization also supports multi-lingual summarization [4, 33]; the input text is taken, and the summaries are generated in different languages, enabling the user to access information in the language of interest. While the base models for text summarization remain consistent across different languages, the primary variation lies in the methods used for training and fine-tuning. The cross-lingual text summarization [253] requires creating a summary of a source article written in one language using resources written in another language[17, 190, 296]. It is helpful when data need to be communicated in different languages to reach a wider audience, such as news stories, social media posts, research publications, and business reports.
A hierarchical summarization technique includes creating a multi-level summary of a given document. This method generates a summary that encapsulates essential data at several levels of abstraction, from a high-level overview to more specific details. Extensive collections of papers and articles can be organized and summarized in content management systems using hierarchical summarization, making it more straightforward for users to access critical information. Abstractive dialogue summarization [141, 260] aims to generate a clear and readable summary of conversational exchanges between two or more speakers. This technique can be used in many contexts such as customer support chats, corporate meetings, and interviews, where condensing lengthy discussions can save time. When labeled training data are unavailable or when dealing with input from a different language, unsupervised summarization [117, 261] is used. Unsupervised summarization is used in various fields such as social media analysis, scientific article summarization, and news summarization.
Based on the input document type, text summarization is classified into single-document summarization and multi-document summarization. Single-document summarization is a summary generation technique in which generated summaries take a gist from a single document. Multi-document summarization [186] is a technique in which summaries are generated from different source documents that discuss the same topic.
Long and short document summaries are two distinct ways to summarize documents of varying lengths. A lengthy document summarization [111] compresses a long document or a vast corpus of text data into a more concise version containing key points, usually by picking the essential content and identifying key terms. Several studies [92, 160] have used efficient methods for long-document summarization. At the same time, short document summarization condenses a long text into a short and readable summary, which is useful for reading relevant information from sources such as news and blogs.
Humans tend to summarize the source article by reading and thoroughly understanding its context of the source article and rephrasing it in their own words in a comprehensive way. Summaries written by the authors can sometimes lead to bias and may not capture the necessary information from the source document. Manually summarizing a large text corpus is time-consuming and impractical because of the sheer amount of available data. This necessitated the demand for computational machines that can process vast amounts of data quickly and efficiently, making them important for NLP tasks such as text summarization. Abstractive text summarization algorithms aim to generate comprehensive and precise summaries with less bias and to preserve the essence of the original text.
Although state-of-the-art (SOTA) abstractive summarization models generate concise summaries, they face significant challenges. Identifying and preserving the key information of a source article while generating a summary in a shorter format is the foremost challenge in this field. Data availability is another major challenge in abstractive text summarization, which can lead to overfitting or poor generalization of summarization models during training. Furthermore, abstractive text summarization tasks require computational resources to produce high-quality summaries by using neural networks with extensive parameters. Despite advancements in the text summarization domain, a few issues persist, such as hallucinations and bias in the generated summaries.
These challenges emphasize the complexity of abstractive text summarization and highlight the need for advanced research on SLR in this area. The results of the SLR are transparent and accountable, and they can minimize the bias compared to traditional literature reviews [188] due to the involvement of scientific methods and systematic craft and a clear scientific outlook in the field.
The literature on text summarization indicates significant advancements in this field, as highlighted by key reviews and surveys [1, 6, 259, 278]. These studies have primarily focused on extractive and abstractive methods and their respective evaluation metrics. Despite the comprehensive coverage, only a few studies [7, 80, 168, 237] have exclusively targeted the domain of abstractive text summarization. This gap underscores new researchers’ challenges, as existing reviews require a systematic approach, complicating the understanding of the domain. While there are existing SLRs in the domain of abstractive summarization, such as References [105, 180, 199], these SLRs often cover only a limited scope of the literature and may not delve deeply into the core aspects of abstractive summarization. Many of these SLRs are constrained by their specific scope, selection criteria, time frame, and rapidly evolving nature of the field, which continuously introduces new methodologies and challenges that may not have been thoroughly analyzed at the time of their publication. Given these limitations, there is a significant need for a more comprehensive SLR that thoroughly analyzes the core aspects of abstractive summarization such as datasets, evaluation metrics, approaches, and methods.
As the field of abstractive summarization continues to evolve, researchers are attempting to refine existing technologies to overcome the limitations of earlier models and improve the reliability and efficiency of summarization methods. This systematic approach ensures that advancements in the field are built on a comprehensive understanding of past and current methodologies, thereby enabling more effective innovations and addressing persistent challenges more strategically. In addition, this SLR provides a generic conceptual framework and guidelines for abstractive summarization and serves as a practical guide for researchers to select appropriate summarization models, which is vital for optimizing performance in real-world applications.
2 Research Framework and Questions
This SLR conducted an in-depth review of 226 papers, positioning it as one of the most comprehensive reviews in single-document abstractive text summarization. The detailed review methodology, including the selection criteria, selection process, quality assessment, and data synthesis, is provided in Electronic Supplement.
In abstractive text summarization, a significant research gap exists because of the need to report and evaluate datasets consistently. This variability undermines the reproducibility and reliability of research findings, impeding scientific progress. Systematic documentation and evaluation of existing datasets are critical for understanding their limitations, enhancing their accessibility, and developing new, comprehensive datasets that are better aligned with the current demands of summarization tasks. In addition to the datasets, the evaluation metrics in text summarization often suffer from the semantic, cohesive, consistent, relevant, and fluency perspectives of automatic evaluation. Developing a standardized set of evaluation methods will guide more refined evaluation practices. This enhancement leads to higher performance and creative summarization models.
Enumerating and assessing recent methodological advancements is essential to ensure that research communities are well informed about the most effective and innovative practices in this field. Despite the rapid technological advancements in abstractive summarization, developing summarization models capable of handling the complexities of natural language requires a detailed understanding of effective approaches and methods, leveraging their strengths, and addressing their weaknesses.
Observing the research gaps in the field of abstractive text summarization has laid the foundation for setting clear objectives for this SLR. The objectives of this study are as follows:
—
To systematically document and evaluate the datasets used in abstractive text summarization;
—
To comprehensively list and review the evaluation metrics for abstractive text summarization;
—
To identify and analyze the most effective approaches employed in abstractive text summarization;
—
To enumerate and assess recent methodological advancements in abstractive text summarization.
The research questions for SLR are listed in Table 1. Research questions RQ1 to RQ3 are the main research questions, focusing on datasets, evaluation metrics, approaches, and methods. Table 2 presents the data extracted from the research papers to answer the research questions.
Table 1.
| ID | Research Question |
|---|---|
| RQ1 | What are the datasets used in abstractive text summarization? |
| RQ2 | What are the evaluation metrics used in abstractive text summarization? |
| RQ3 | What are the approaches and methods used in abstractive text summarization? |
Table 2.
| Extracted Data | Research Question |
|---|---|
| Dataset | RQ1 |
| Evaluation metrics | RQ2 |
| Approaches | RQ3 |
| Methods | RQ3 |
Based on the research gaps, objectives, and research questions, the contributions of SLR are as follows:
—
A detailed examination and analysis of 226 original studies published between 2011 and 2023 provides a comprehensive overview of single-document abstractive text summarization.
—
A comparative and evaluative study of datasets, evaluation metrics, approaches, and methodologies was presented.
—
The development of a generic conceptual framework for abstractive text summarization has been proposed.
—
A comprehensive set of guidelines to aid in the selection of the most suitable summarization techniques was presented.
3 Results
This section describes the results of the quantitative and qualitative analyses of the papers from the SLR. There are five subsections in this section, which follows a systematic approach to answer each research question in Table 1.
3.1 Datasets in Abstractive Text Summarization - RQ1
The number of datasets in the abstractive text summarization domain has increased in response to the needs of the past decades. A dataset is necessary to train, validate, and test the proposed method on SOTA abstractive text summarization models. This SLR has only focused on studies utilizing English datasets, because although many techniques for other language datasets exist, their underlying architectures and techniques are similar to those using English datasets.
Various datasets were used for abstractive text summarization. Each dataset, irrespective of its curated domain, contains a collection of source documents and its human-written reference summary, also known as a gold summary. Dataset curation in abstractive text summarization can be classified as general and domain-specific, regardless of whether it is a public or private dataset [259]. Public datasets such as the CNN/DailyMail dataset [173], Gigaword dataset [74, 216], Extreme Summarization (XSum) dataset [176], Document Understanding Conferences (DUC) dataset [87], New York Times Annotated Corpus [220], and Newsroom [75] comprise general topics. Domain-specific datasets such as Amazon Fine Food Reviews [164] and BigPatent [226] include data on specific topics. This study reviews the datasets used for abstractive text summarization between 2011 and 2023.
The following are the widely used general and domain-specific datasets in abstractive text summarization:
—
CNN/DailyMail: Of the 226 publications, 162 reported their findings on the CNN/DailyMail dataset. CNN/DailyMail is a widely used dataset in abstractive text summarization. Human-generated abstractive summaries were created from news articles on CNN and Daily Mail websites as questions, and the stories were used as relevant passages. The CNN/DailyMail dataset consists of human-generated news stories and contains 286,817 training pairs, 13,368 validation pairs, and 11,487 testing pairs.
—
Gigaword dataset: Forty-eight out of 226 publications reported their findings on the Gigaword dataset. The Gigaword dataset consists of approximately four million news articles. The Gigaword dataset has headline-article pairs consisting of 3,803,957 training pairs, 189,651 validation pairs, and 1,951 testing pairs of data.
—
XSum: The Extreme Summarization (XSum) dataset was used by 57 publications. The XSum dataset contained 226,711 news articles with one-sentence summaries. This dataset is primarily used for abstractive text summarization. The XSum dataset contains a one-sentence news summary that describes the details of an article. In the training, validation, and test sets, the official random split contained 204,045 (90%), 11,332 (5%), and 11,334 (5%) documents, respectively.
—
DUC dataset: The Document Understanding Conferences (DUC) dataset is the sentence summarization dataset used by 42 out of 226 publications. The most widely used DUC datasets were DUC 2002, DUC 2003, DUC 2004, and DUC 2006. Each document collection in the DUC has four to five extra human-written “reference” summaries for a single-source document. Usually, the DUC dataset contains fewer data samples than other datasets. Therefore, many summarization models use this dataset as the test set.
—
New York Times Annotated Corpus: Eleven of 226 publications used the New York Times Annotated Corpus dataset. This dataset contains over 1.8 million articles written and published by the New York Times. The New York Times Annotated Corpus contains 650,000 article-summary pairs written by library scientists.
—
Newsroom: The Cornell Newsroom summarization dataset was used by 12 studies. It has 1.3 million articles and summaries authored by journalists and editors working in the newsrooms of 38 significant periodicals. Summaries of the newsroom dataset were obtained using various summarizing algorithms.
—
Amazon Fine Food Reviews: Two publications used this domain-specific summarization dataset. This dataset contains approximately half a million food reviews from Amazon. The reviews include product and customer information, ratings, and textual reviews.
—
BigPatent: Four publications used this domain-specific summarization dataset. This dataset contains 1.3 million US patent filing records and abstractive summaries written by humans.
Out of the 226 publications we reviewed, 61 reported datasets. However, out of these 61 datasets, only 28 were used more than once by the publications. Nevertheless, few publications have reported their findings on multiple datasets to make it easy for researchers in this field to compare the efficiency of their models. Curating a dataset for abstractive text summarization is tedious and expensive. Unlike extractive text summarization, abstractive text summarization requires abstractive human-written summaries.
Even though there are many abstractive text summarization datasets, the trend in this field is to use publicly available news article-based datasets such as CNN/DM, XSum, or any domain-specific datasets. Publications reporting their findings on the widely used dataset help researchers in this field compare their abstractive text summarization models with benchmarks. However, one potential concern regarding datasets in abstractive text summarization is that the number of training samples is less than the size of the trainable parameters in the recently introduced SOTA abstractive text summarization models. This issue may also result in problems such as hallucinations in the generated summaries [28]. There is always a need to train the models with increasingly large training samples to perform better on downstream tasks such as abstractive text summarization, even if the models are good at capturing English sentence structures due to pre-training.
This review examined several datasets used in abstractive text summarization. The datasets reviewed included CNN/DailyMail, Gigaword, XSum, DUC, New York Times Annotated Corpus, Newsroom, Amazon Fine Food Reviews, and BigPatent. A comparative analysis and evaluation of the datasets based on the annotation process, diversity, and coverage, the model types they are suited for (extractive, abstractive, or both), and potential biases are presented in Table 5 (see electronic supplement). The findings suggest significant variations in the suitability of these datasets for training abstractive text summarization models. Among the datasets evaluated, CNN/DailyMail and XSum stood out as particularly robust for several reasons.
—
Wide Usage: Both datasets are extensively used in the field of abstractive text summarization, underscoring their reliability.
—
Quality: CNN/DM offers high-quality summaries that challenge models in capturing nuanced content. XSum, known for its concise one-sentence summaries, requires models to perform significant abstraction, making it a rigorous test for summarization capabilities.
—
Diversity and Coverage: CNN/DM covers many news topics and provides diverse articles for model training. Although XSum primarily includes BBC news articles, it presents a broad topic spectrum, supporting an effective model generalization.
—
Benchmarking and Performance Evaluation: Both datasets serve as standard benchmarks in the field, effectively comparing SOTA summarization models against established performance metrics.
Based on the criteria of wide usage, quality of summaries, diversity, and their role in benchmarking, CNN/DailyMail and XSum are currently the best datasets available for abstractive text summarization. Their widespread adoption and the challenges they pose to summarization models make them ideal for developing new models and improving existing ones.
3.2 Evaluation Metrics in Abstractive Text Summarization - RQ2
The evaluation metric evaluates the summaries and determines the performance of the abstractive summarization model. The evaluation technique compares the system-generated summary with the gold summary from the summarization dataset. The evaluations are based on the number of words that match the word’s lemma format and grammatical structures. In abstractive text summarization, evaluation is based on comparing the critical keyword or crucial information from the gold and the generated summaries. The evaluations are mainly conducted to understand the generated summary’s precision, recall, and F1 measure. The studies between 2011 and 2023 have proposed many automatic evaluation metrics.
The following are the widely used evaluation metrics in abstractive text summarization:
—
ROUGE: Recall-Oriented Understudy for Gisting Evaluation (ROUGE) [138] is one of the standard automatic evaluation metrics for abstractive text summarization. This evaluation metric is widely used in abstractive text summarization. Of the 226 publications, 212 relied solely on the ROUGE metric to evaluate summaries. The ROUGE metric scores the system-generated summaries based on the number of overlapping \(n\)-grams between the system-generated and gold summaries and generates scores based on precision, recall, and F1-metric. Based on the overlapping \(n\)-grams, the selection of ROUGE-1, ROUGE-2, and ROUGE-L depended on the specific needs of the summarization task, as each variant targeted different aspects of the summaries. ROUGE-1 assesses word overlap, ROUGE-2 examines consecutive word pairs, and ROUGE-L evaluates the longest common sub-sequence. Choosing the most suitable metric relies on the context of the task, and it is common practice to report multiple ROUGE scores to evaluate the summarization system comprehensively.
Variants of ROUGE evaluation metrics, such as ROUGE-WE [179], use word-embedding-based similarity metrics with a lexical matching method. ROUGE 2.0, which uses a synonym dictionary based on WordNet [166], and ROUGE-G [224] use semantics and lexical matching from WordNet to evaluate abstractive text summaries.
—
Human Evaluation: Human evaluation [70, 95] is considered an ideal evaluation metric for abstractive text summarization. Source documents, gold, and system-generated summaries were provided to human annotators. They score the system-generated summaries based on the summary’s quality and criteria for relevance, fluency, and consistency, respectively. Sixty-one publications used the human evaluation metrics. However, many studies have used ROUGE and human evaluation metrics to obtain the desired scores.
—
METEOR: The Metric for Evaluation of Translation with Explicit Ordering (METEOR) [21] was introduced for machine translation. However, abstractive text summarization follows an approach similar to machine translation. Therefore, METEOR was used for abstractive text summarization. METEOR uses the harmonic mean of precision and recall, with recall being weighted more than precision. Thirteen of 226 publications used the METEOR evaluation metric.
—
BLEU: Bilingual Evaluation Understudy (BLEU) [192] is an evaluation metric proposed for machine translation and later used for abstractive text summarization. Modified \(n\)-gram precision and best match length were used in BLEU to estimate precision and recall. Nine of 226 publications used the BLEU evaluation metric.
—
BERTScore: BERTScore [286] is an automatic evaluation metric for text generation in which token similarities are used to compare contextual embeddings. Scores were generated based on these token similarities. Twelve out of 226 publications used the BERTScore evaluation metric.
—
MoverScore: MoverScore [290] is an evaluation metric that encodes a gold summary and generated summary, showing a high correlation with human judgment in text quality. MoverScore uses contextualized embeddings and the earth mover distance to evaluate summaries. Two studies have used this evaluation metric.
—
QAGS: Question Answering and Generation for Summarization (QAGS) [249] is an automatic evaluation metric that measures the factual consistency of system-generated summaries. Three studies used this evaluation metric. The summaries generated by SOTA abstractive text summarization models tend to be factually inconsistent. The QAGS metric was proposed to evaluate the factualness of the summary. QAGS uses the Question Answering (QA) and Question Generation (QG) models. APES [63] is another metric that uses a similar approach to evaluate summaries using QA systems and is a reference-free evaluation metric.
—
FactCC: Factual Consistency Checking (FactCC) [114] is a model-based and weakly supervised evaluation metric for verifying factual consistency and identifying conflicts between source documents and generated summaries. Twelve studies used this evaluation metric. FactCC uses the BERT model to classify factually consistent summaries based on training.
In abstractive text summarization, summaries might contain novel words, phrases, and sentences that are semantically similar to gold summaries. However, fewer words, phrases, and sentences would overlap between these abstractive summaries and their gold summaries. Table 6 (see electronic supplement) shows the quantitative analysis of the time complexities of the evaluation metrics in the abstractive text summarization.
Evaluation of the quality of summaries generated by abstractive text summarization models can be challenging because of the subjective nature of summarization. The abstractive summary evaluation trend shifted from quantitative to qualitative summary evaluations. Traditional evaluation metrics such as ROUGE, METEOR, and BLEU have quantitatively evaluated system-generated summaries by comparing the number of overlapping \(n\)-grams between gold and system-generated summaries. In contrast, recently introduced evaluation metrics such as QAGS and FactCC have been qualitatively evaluating system-generated summaries by checking for factual inconsistency. However, automatic evaluation metrics have a few potential concerns, such as scoring summaries according to their semantics and abstractiveness. An efficient evaluation metric must capture the semantics, consistency, relevance, and fluency of the summaries and their abstractiveness, which is entirely subjective.
Because an efficient evaluation metric should capture the semantics and abstractiveness of summaries, human evaluation is considered the gold standard evaluation metric for abstractive text summarization [95]. However, researchers must scrutinize its reliability and investigate potential factors that may influence its accuracy in accepting human evaluation as the gold standard. The proficiency of human evaluators can influence their evaluation outcomes. Evaluators with a higher level of expertise in the topic of the text being summarized are more likely to offer accurate and insightful scores to summaries than are those without such expertise. Furthermore, the nature of summaries can affect the results of human evaluation. Specific summaries are straightforward for human evaluators to comprehend and judge, resulting in higher scores. In contrast, other summaries are more complex to understand and evaluate, resulting in lower scores. Despite the importance of expertise and summary generation methods, several challenges still arise during human evaluation, such as
—
Human evaluation of summaries generated by abstractive text summarization models can be time-consuming, expensive, and subject to bias in scoring summaries.
—
Human evaluators often fail to include crucial details, such as participant demographics, task design, and experimental protocols.
—
Human evaluators are often asked to evaluate text based on subjectivity, such as overall quality, cohesiveness, fluency, and relevancy, which can lead to inconsistencies in assessing the quality of the generated summaries.
—
Human evaluators cannot distinguish between human-generated and system-generated text [52].
Even though there are several challenges associated with human evaluation, different solutions can improve their accuracy and reliability. These solutions are as follows:
—
Using a large and diverse group of expert annotators from crowdsourcing platforms to obtain evaluations can increase the reliability and reduce the cost of the evaluations.
—
Providing established evaluation protocols and clear instructions to annotators can help improve the consistency and accuracy of human evaluations.
—
Automated metrics are provided to supplement human evaluation for a more efficient and comprehensive assessment of generated summaries.
—
Using multiple evaluation metrics and ensembling their results can provide a more comprehensive and accurate evaluation of generated summaries.
—
Incorporating user feedback for human evaluators to refine and improve the quality of evaluation of generated summaries over time.
These strategies and solutions can help to address human evaluation challenges and enhance their validity and usefulness. Researchers must undertake rigorous human evaluation studies and thoroughly investigate elements that could affect their accuracy. Nevertheless, this SLR suggests the need for a new evaluation metric for abstractive text summarization.
3.3 Approaches in Abstractive Text Summarization - RQ3
This subsection outlines the different approaches employed in abstractive text summarization, beginning with rule and graph-based techniques. Before exploring transformers and hybrid approaches, neural-network-based approaches, including CNNs, RNNs, and attention mechanisms, which are key components of many SOTA models for abstractive summarization, are explored. These approaches are broadly categorized as general strategies that provide frameworks for optimal summarization. Although many studies have utilized similar approaches, they distinguish themselves using specific methods. This subsection discusses the different methods used in this field, evaluates SOTA abstractive summarization models, and investigates how pre-training affects the quality of their summaries.
Our SLR records from 2011 to 2023 show four main model-design approaches. The four approaches are as follows:
(i)
Rule-based approach
(ii)
Graph-based approach
(iii)
Neural-network-based approaches
(a)
CNN-based architecture
(b)
RNN-based architecture
(c)
Transformer-based architecture
(iv)
Hybrid approaches.
Research on abstractive text summarization commenced with the summary generation process using rule-based and graph-based approaches. Later, these approaches were revolutionized into neural-network-based approaches. The CNN-, RNN-, and transformer architecture-based approaches are neural-network-based approaches. Then, the combinations of each approach are curated as hybrid approaches. Table 3 depicts the approaches and counts of papers in the abstractive text summarization. The following subsections will discuss each of these approaches in detail.
Table 3.
| Approaches | Counts |
|---|---|
| Rule-based | 6 |
| Graph-based | 6 |
| Neural-network-based (CNN Architecture) | 5 |
| Neural-network-based (RNN Architecture) | 104 |
| Neural-network-based (Transformer Architecture) | 86 |
| Hybrid | 19 |
3.3.1 Rule-based Approach.
In a rule-based approach, the source article initially undergoes preprocessing steps such as tokenization and sentence segmentation. Then, statistical or predefined linguistic rules to rank the sentences in the source article based on algorithms such as TF-IDF were applied. The top-ranked sentences were then chosen as key points for the summary generation. Despite its structured approach, this method may encounter challenges, such as grammatical inaccuracies and the need for more semantic comprehension. Table 7 (see electronic supplement) presents the taxonomic view of publications that used the rule-based approach.
3.3.2 Graph-based Approach.
In the graph-based approach, the input source article is represented as a graph, with nodes denoting the sentences of the source article and edges indicating the relationships between them. Employing graph algorithms such as PageRank identifies the most important nodes in the graph representing key points in the source article. Based on the identified key points, a summary was generated, providing a structured portrayals of the main ideas of the article. Although graph-based methodologies offer a systematic summarization framework, they may require assistance in managing large-scale graphs and capturing semantic nuances. It is important to explain how these graphs identify and prioritize key points to enhance the effectiveness of the summarization process. Table 8 (see electronic supplement) depicts the taxonomic view of the publications using a graph-based approach.
3.3.3 Neural-network-based Approaches.
Neural-network-based approaches, such as CNN, RNN, and transformer architectures, are preferred in the abstractive text summarization domain. The shift towards neural-network-based approaches is motivated by their inherent capacity to learn complex patterns and relationships within textual data, enabling them to capture semantic nuances and generate summaries with enhanced coherence and contextuality. Unlike rule-based methods, which rely on predefined linguistic rules and statistical metrics, neural-network-based models can automatically learn representations of textual data through training on large-scale datasets. Learning from data allows neural-network-based models to adapt to diverse text types and domains, thereby improving summarization performance across various contexts. Additionally, neural-network-based models, particularly transformer architectures, are capable of capturing long-range dependencies and global contextual information, surpassing the limitations of graph-based approaches in handling large-scale graphs and semantic subtleties.
Nevertheless, recognizing the computational demands inherent in neural-network-based approaches, notably transformer architectures, is essential, because they often require significant computational resources and prolonged training durations. Additionally, the interpretability of neural-network-based models poses a persistent challenge, as their internal mechanisms frequently need more transparency, hampering the explanation of decision-making processes. Tables 9–11 (see electronic supplement) represent the taxonomic view of the publications that used the CNN, RNN, and transformer architecture approaches.
—
CNN Architecture: CNNs process input articles by initially segmenting them into tokens and representing each token numerically using word embeddings. Next, CNNs employ convolutional filters on the input embeddings to extract features and recognize relevant phrases or sentences. CNNs can identify spatial hierarchies within an input by focusing on important sentences at various levels of granularity. The gathered features were combined to summarize the most salient information from the source article. This ability of CNN is particularly useful in abstractive summarization, where the aim is to reinterpret and condense the original material creatively.
—
RNN Architecture: In abstractive text summarization, RNNs transform input articles into a sequence of word embeddings by preserving the temporal order of words and sentences. The encoded input source article is processed sequentially by RNN by updating the hidden states at each timestep, enabling them to retain memory of past inputs and grasp contextual information. RNNs also incorporate attention mechanisms, enabling the model to concentrate on relevant input segments during summarization and to assign higher importance to significant words or phrases. Based on the acquired contextual representations and attention weights, RNNs generate summaries by decoding the encoded sequence into natural language text, ensuring an accurate reflection of the key ideas of the original article. RNNs capture the memory of preceding inputs, which are critical for abstractive text summarization models where the text length can vary.
—
Attention Mechanisms in Abstractive Text Summarization: Although SOTA pre-trained abstractive summarization models have successfully generated high-quality summaries, attention mechanisms have emerged as a powerful tool for improving the performance of various NLP tasks, including abstractive summarization. Many studies on CNN, RNN, and transformer architectures have utilized attention mechanisms or variants thereof to enhance the summarization process. The breakthrough research by Bahdanau et al. and Vaswani et al. [16, 244] introduced attention mechanisms for abstractive text summarization. By allowing abstractive summarization models to focus on relevant parts of the input text while generating a summary, attention mechanisms can help handle long input sequences and improve model accuracy. These mechanisms promote a better understanding of the relationships between words and phrases, leading to a more accurate and effective summarization. There are various types of attention mechanisms, and the choice of each attention mechanism depends on the tasks and characteristics of the input sequence. Different attention mechanisms used in abstractive text summarization are discussed in detail in Electronic Supplement Section A.3.
—
Transformer Architecture: Transformer models have revolutionized the NLP domain by capturing global dependencies and contextual information. Recent studies in the abstractive text summarization domain have widely used the transformer architecture introduced by Vaswani et al. [244] because of its flexibility and accuracy. The transformer architecture was introduced specifically to address the limitations of RNNs in capturing the long-term dependencies in sequences. This architecture comprises a stack of encoders and decoders with attention mechanisms. Summarization with transformers initiates with self and multi-head attention mechanisms, wherein each word in the input sequence attends to all others, allowing the model to assess the importance of each word relative to the overall context. The transformers followed pre-training and fine-tuning strategies. The transformer models were pre-trained on a large corpus of texts, such as the entire BookCorpus (800 million words) [297] and English Wikipedia articles (2,500 million words). These models were fine-tuned on the abstractive summarization datasets. Transformers allocate greater importance to pertinent words and phrases, simplify the extraction of salient information from the input article, and generate a summary by decoding the encoded sequence into natural language text, producing succinct and coherent summaries that accurately encapsulate the primary ideas and key points of the original text. This strategy helps language models better understand the English language’s sentence structures, which helps build SOTA abstractive text summarization models. Because the transformer architecture approach uses pre-training and fine-tuning strategies, it performs better than the rule and graph-based models. Currently, the models using transformer architectures are the SOTA in the abstractive text summarization domain.
3.3.4 Hybrid Approaches.
Hybrid summarization methodologies combine multiple techniques, including rule-based, graph-based, and neural-network-based approaches, to harness their strengths and address their limitations. By integrating complementary techniques, hybrid methodologies endeavor to enhance the overall efficiency and effectiveness of the summarization process. These techniques may employ ensemble methods to amalgamate key points identified by various components or assign weights based on their significance. Although hybrid methodologies offer versatility and adaptability, they may require heightened computational complexity and meticulous parameter tuning. Only a few publications have used hybrid approaches to improve the efficiency of abstractive text summarization models. The hybrid approach combines rule-based, graph-based, fuzzy-logic, and transformer architecture-based approaches with CNN- and RNN-based approaches. Table 12 (see electronic supplement) shows a taxonomic view of the research papers using this approach.
A comprehensive examination of abstractive text summarization approaches conducted through an SLR from 2011 to 2023 has provided invaluable insights into the field’s evolution. Among the diverse range of identified model design approaches, including rule-based, graph-based, neural-network-based (such as CNNs, RNNs, and transformer architectures), and hybrid methodologies, neural-network-based approaches have emerged as the most promising and impactful avenues for advancing the field. A comparative evaluation of these approaches highlights their distinctive strengths and limitations.
Although rule-based and graph-based methods offer structured frameworks for abstractive summarization, semantic comprehension may hinder their efficacy. Neural-network-based approaches, particularly those leveraging transformer architectures, exhibit superior performance in capturing complex patterns, long-range dependencies, and semantic nuances in textual data. These approaches generate more coherent and accurate summaries because of their ability to learn and adapt to diverse linguistic contexts, thereby enhancing the effectiveness of the summarization task. Moreover, hybrid approaches that integrate techniques from rule-based, graph-based, and neural-network-based approaches present a compelling way to combine the strengths of each approach. By leveraging combined techniques, hybrid models aim to address the limitations of individual methods while improving overall summarization performance. However, hybrid approaches may increase the computational complexity and require careful parameter tuning.
In conclusion, the findings of this review highlight the key role of neural-network-based approaches, particularly those employing transformer architectures, in shaping the future of abstractive text summarization. As this field continues to evolve, these approaches will facilitate the generation of more accurate, coherent, and contextually relevant summaries across various domains and applications.
3.3.5 Implementation of Abstractive Text Summarization Approaches.
This subsection describes the implementation of various abstractive text summarization approaches used in the publications included in the SLR.
The abstractive summarization models employing a rule-based approach use methods such as pattern extraction [197] and sentence ranking [218]. Graphs-based models use rich semantic graphs [167] and sentence enhancement methods[46]. The sequence-to-sequence architecture-based approaches, such as the pointer generator networks with coverage mechanisms and their variants, are also widely used in abstractive text summarization tasks [43, 108, 223, 291]. The pointer points to the source texts, the generator generates words from the vocabulary, and the coverage mechanism ensures no repetition of words in the generated summary. Reinforcement learning is another method used by some publications [79, 115, 193, 195] in CNN- and RNN-based approaches. It is a reward-based mechanism wherein the model obtains rewards for predicting the correct summary generation and a penalty for incorrect summary generation. This mechanism helps the model to maximize the possibility of positive outcomes through rewards.
While the foundational and subsequent specialized attention mechanisms uniquely contribute to advancing neural network architectures for abstractive text summarization, the current SOTA abstractive text summarization models employ transformer-based attention mechanisms. Building upon the dynamic attention capabilities introduced by earlier models, transformer architectures have catalyzed a paradigm shift, dramatically enhancing the scalability and processing efficiency. Incorporating self-attention and multi-head attention within these architectures allows for unprecedented processing of multiple representation subspaces simultaneously, significantly outperforming earlier models across a range of complex tasks. These attributes emphasize transformer-based models as the best current implementation of attention mechanisms, offering an unmatched performance in handling long-range dependencies and diverse linguistic features.
The transformer architecture-based approach uses pre-training and fine-tuning-based methods. The majority of the methods in the transformer architecture approach lie in proposing a new pre-training objective, such as the masked language model (MLM) [56], next sentence generation (NSG) [271], and sentence reordering [301]. These objectives help language models understand the language’s syntactic and structural representations. Even a few publications have used the few-shot transfer learning method [64], wherein abstractive summarization models can be trained using less training data.
Many publications on transformer architecture-based approaches are entirely based on pre-training and fine-tuning methods and their variants. Table 13 (see electronic supplement) provides a quantitative evaluation of the SOTA pre-trained models using ROUGE scores. In abstractive text summarization, transformer-based SOTA models involve the necessary steps that enable them to produce concise and coherent summaries. Different steps must be followed to obtain quality summaries, such as preprocessing and tokenization, embedding and representation, contextual understanding and content selection, advanced pre-training strategies, model training and optimization, summary generation and decoding strategies. These steps are crucial for analyzing and understanding how different models work. Table 14 (see electronic supplement) compares the SOTA text summarization models based on different abstraction steps. The following are the different steps of abstraction in abstractive text summarization:
(1)
Preprocessing and Tokenization: This initial step in abstractive text summarization involves preprocessing the raw input text articles. The preprocessing step includes a text-cleaning method, which removes punctuation and converts all text into lowercase. The next step is tokenization, which helps to segment the input text into manageable tokens. Several methods are available for tokenizing a given input text article. The WordPiece method breaks words into meaningful subwords; models such as BERT use this method. The SentencePiece method processes text into a raw input stream with unclear word boundaries. This method enhances the robustness across languages; therefore, it is used in models such as T5 and PEGASUS. Another method called Byte-Pair Encoding (BPE) iteratively merges the most frequent pairs of characters or bytes. This helps in effectively managing rare words and reducing vocabulary size. The BPE tokenization method is used in models such as the BART. These preprocessing and tokenization steps are essential to ensure that the text input to the model is uniform and optimized for subsequent processing steps.
(2)
Embedding and Representation: Once input text is preprocessed and tokenized, each piece of input text is converted into a numerical representation called embeddings. These embeddings are either pre-trained on large datasets to capture a wide range of semantics and syntactic properties through unsupervised learning or fine-tuned for specific tasks such as summarization to adapt to the general representation of the summary-specific language. These embeddings and representations set a foundation for the deep analysis of semantics, which is a crucial step for effective summarization.
(3)
Contextual Understanding and Content Selection: The Transformer-based abstractive text summarization models leverage the power of these robust embeddings to the attention mechanism to dynamically assign importance to every phrase and word based on the context. This step allows for a deep understanding of the importance of words or phrases that must be retained in the summary. The following techniques were used for contextual understanding and content selection:
—
Scaled-dot Product Attention: It is a fundamental mechanism in the self-attention process of a transformer. In this technique, the attention score is calculated by taking the dot product between the queries representing each token being processed, with all keys representing every token in the input.
—
Multi-head Attention: Multi-head transformers deploy multiple attention heads in parallel. Each attention head executes the scalar dot product independently, which concurrently explores the different relationships and patterns between input data. This parallel mechanism allows for a more profound understanding of the context of the model, leading to a quality summary.
(4)
Advanced Pre-training Strategies: Advanced pre-training strategies are the pre-training objectives considered during pre-training, allowing the model to understand linguistic structures. These strategies include masked language modeling (MLM), contrastive learning, and task-specific objectives. The MLM pre-training strategy involves masking certain words in the input text, and during training, the model is made to predict the masked word. This strategy helps understand the context of the input text, which is crucial during language generation. However, the contrastive learning strategy involves training the model to distinguish between correctly closely related and highly altered sequences of the input text. This strategy helps the model to understand and distinguish subtle changes in language. Another type of strategy is called task-specific objectives, in which the model is trained to directly generate specific tasks, such as next-sentence prediction (NSP) or sentence reordering. This strategy helps the model better understand the narrative flow and logical coherence of the text. These strategies expose the transformer-based model to perform well in complex-language-processing tasks.
(5)
Model Training and Optimization: Some targeted strategies that enhance the model’s performance are used during model training. These effective training and optimization methods improve the accuracy and ability of the model to generate a summary. The targeted strategies include the loss function, adversarial training, and regularization techniques. The loss function utilizes cross-entropy loss to ensure high prediction accuracy. The adversarial training strategy introduces minor disturbances during training to improve the robustness of the model. This strategy teaches the model to handle minute disturbances in the input data effectively. Regularization techniques use dropout and weight decay methods to prevent generalization and overfitting. Dropout prevents remodeling among neurons by rapidly deactivating specific neurons during training, whereas weight decay penalizes larger weights to maintain the stability and simplicity of the model. All these model training and optimization strategies help to efficiently train the abstractive summarization models.
(6)
Summary Generation and Decoding Strategies: This summarization phase involves refined decoding strategies and advanced generation techniques. The strategies involved in this phase include conditional generation, a softmax layer, beam search, top-k sampling, and nucleus sampling. Conditional generation uses a decoder to generate each word based on the previous word and overall context, ensuring that the output is coherent and contextually relevant. The softmax layer is a decoder component that computes the probability distribution of the next possible word, likely from the contextual and coherent appropriate continuation of the input text. Another strategy for content selection is beam search, in which multiple hypotheses are considered at each step of the generation process, keeping the top beam width as the most probable sequence. In comparison, top-k and nucleus sampling introduce the random selection process by restricting the model’s choice to the top-k and top-p percentage of the following words.
Approaches and methods in abstractive text summarization have evolved from statistical approaches, such as rule-based and graph-based approaches to neural-network-based approaches. Researchers have combined statistical and neural-network-based approaches to derive the benefits of both approaches. Building a summarization model using a neural-network-based approach, such as RNN and CNN, is easy. This requires minimum computational requirements, and the model parameters can be controlled while training the model. However, the accuracy of these models could be, at most, that of transformer-based approaches.
The SOTA summarization models use transformer-based approaches. Building and training a summarization model using this approach demands substantial computational resources due to the training of billions of parameters associated with these models. There is a requirement in this field to build a robust abstractive text summarization model that is efficient and can train with low computational resources and optimized parameters.
3.4 Abstractive Text Summarization using Large Language Models
Recent advancements in NLP have highlighted the growing prominence of LLMs, owing to their powerful generative capabilities and versatility in handling various tasks, including abstractive summarization. LLMs are built on top of the transformer architecture, which utilizes a self-attention mechanism, allowing models to weigh the importance of different words in a sentence, irrespective of their sequential order. This architecture facilitates a deeper understanding of the context and semantic relationships across long texts, making LLMs particularly effective for generating coherent and contextually relevant summaries. As a result, these models have become fundamental in advancing SOTA in NLP, driving innovations and applications in various NLP tasks.
Although originally not designed for summarization tasks, LLMs offered by OpenAI, Google, and Meta have shown exceptional capabilities in generating abstractive summaries. These models can effectively produce general summaries; however, achieving customized content or specific stylistic outputs generally requires methods such as fine-tuning and prompt engineering. Fine-tuning tailors the model to precise summarization tasks by training on targeted datasets, promising higher-quality results and the ability to handle more examples than standard prompting. This process, while delivering improved outcomes and reduced request latency, requires substantial computational power, potentially limiting accessibility to users with constrained hardware resources. By contrast, prompt engineering requires less computational effort and involves creating detailed prompts that direct the model’s attention to key text elements to generate succinct summaries.
SOTA LLMs include OpenAI’s GPT-3.5 and GPT-4 [189], which are notable and renowned for generating contextually rich summaries through advanced fine-tuning and prompt engineering techniques. Similarly, AI21 Labs developed GPT-Neo and Jurassic-1 Jumbo [137], which are robust to text generation. Megatron-Turing NLG [228], a collaborative effort between NVIDIA and Microsoft, stands out for its large-scale generative abilities. Google AI’s BARD and Gemini [12] models efficiently manage various NLP tasks, whereas Meta AI’s open-source models, LLaMA [241] and LLaMA-2 [242], offer significant customization with data privacy and fine-tuning flexibility. Mistral AI’s Mistral 7B [99] and Anthropic’s models, such as Claude 3 Opus, Sonnet, and Haiku [13], emphasize safety and usability, focusing on generating secure and user-friendly text outputs, including summaries. These models collectively push the boundaries of what is achievable in automated text summarization, showcasing a range of approaches for tackling the complexities of language understanding and generation.
The use of LLMs for abstractive summarization also involves critical tradeoffs between resource accessibility and operational convenience. Open-source models, such as LLaMA-2, provide extensive customization opportunities, allowing users to tailor the models to specific needs. However, they require significant computational power for running and fine-tuning, necessitating robust hardware setups, which can be a barrier for smaller research teams or individual researchers. Nevertheless, cloud-based API access to models such as GPT-4 or Gemini offers a user-friendly alternative, with the infrastructure managed by service providers. However, this comes at the cost of subscription fees, which may accumulate significantly, depending on the usage volume.
Moreover, LLMs can generate summaries that are accurate and stylistically aligned with the source material, reflecting nuances, such as tone and sentiment. This capability to adjust tone or style according to specific requirements makes LLMs particularly valuable for diverse summarization needs. However, their “black-box” nature can obscure how these summaries are generated and pose challenges in understanding and addressing potential biases within the models.
In conclusion, although LLMs offer significant advantages in terms of quality and versatility for abstractive summarization, they also present challenges related to computational demands, cost, and transparency. As the field continues to evolve, advancements are expected to mitigate these challenges, making them more accessible and understandable to a broader range of users.
4 Conclusion
This SLR follows a systematic review protocol with predefined research gaps, questions, objectives, inclusion and exclusion criteria, quality assessment, and systematic data extraction and analysis methods. This systematic methodology ensures that the findings are transparent, minimally biased and easily reproducible for further research. This SLR helps researchers in this domain by providing extremely useful insights into the comparative evaluation of the datasets, evaluation metrics, approaches, and methods, highlighting emergent trends within the field. This SLR reviews 226 papers and comprehensively explores abstractive text summarization methodologies, such as rule-based, graph-based, CNN-, RNN-, and transformer-based approaches. Among these approaches, the current SOTA systems employ transformer-based approaches, particularly owing to their ability to manage long-range dependencies and their scalability.
Starting from basic approaches, such as rule-based and graph-based approaches, and transitioning towards the most advanced neural network approaches, such as transformers and their attention mechanisms, help researchers in this field to understand the progress in this field and help them understand the current SOTA systems in this domain. The recent rise in LLMs such as GPT-3.5, GPT-4, LlaMa-2, Claude, and Gemini variants has revolutionized the capabilities of generating coherent and contextual summaries. Even though these models are the current SOTA and generate summaries according to the user’s needs with a desired tone and structure, they come with a demand for computational resources, which is a limitation regarding accessibility and environmental impact. This demands model optimization and the development of efficient models that can be run easily without heavy computational requirements. Another important concern about these LLMs is that their underlying architecture is known, but the pre-training strategies are not completely disclosed to the research community.
This SLR also splits the entire abstractive text summarization task into several layers of abstraction and explains how SOTA summarization models work in each abstraction layer. Additionally, a generic conceptual framework proposed for abstractive text summarization will ensure that researchers understand the task better and will help in future technological advances. Finally, guidelines for choosing the most appropriate summarization models tailored to abstractive text summarization are proposed. This helps researchers to optimize performance and efficiency in real-world applications and choose effective methods based on their needs. The findings of this SLR aim to encourage a more collaborative research environment responsive to the evolving needs of the global NLP community, thus encouraging continuous advancement in abstractive text summarization. This SLR fills a critical gap in the literature and catalyzes further innovation.
To advance the field of abstractive text summarization, it is crucial to develop more sophisticated and robust datasets that minimize bias and factual inconsistencies, thereby helping models reduce bias and hallucinations. Additionally, new evaluation metrics should be created to assess summaries beyond simple n-gram overlap, accurately reflecting human judgments by evaluating semantics, coherence, consistency, relevancy, and fluency. Future research should also prioritize optimizing model architecture and training processes to require low computational resources without compromising performance. Despite the current SOTA models being LLMs, their pre-training strategies are often undisclosed, posing challenges for reproducibility. Therefore, a focus on reproducibility will enable researchers to harness the power of LLMs while also optimizing them. Moreover, since LLMs are generally pre-trained and not specifically built for text summarization tasks, researchers should aim to develop LLMs tailored specifically for this area, utilizing few-shot fine-tuning or prompting techniques.
Acknowledgement
We would like to express our sincere gratitude to the anonymous reviewers for their invaluable feedback and constructive comments. Their thorough and insightful reviews have significantly enhanced the quality of this manuscript. We sincerely appreciate their time and effort in providing detailed critiques and suggestions, which have been instrumental in refining our work. Their expertise and dedication to the peer review process are greatly acknowledged.
Supplemental Material
PDF File
The detailed review methodology, including the selection criteria, selection process, quality assessment, and data synthesis, is provided in Electronic Supplement.
- Download
- 649.58 KB
References
[1]
Laith Abualigah, Mohammad Qassem Bashabsheh, Hamzeh Alabool, and Mohammad Shehab. 2020. Text Summarization: A brief review. In Recent Advances in NLP: The Case of Arabic Language, Mohamed Abd Elaziz, Mohammed A. A. Al-qaness, Ahmed A. Ewees and Abdelghani Dahou (Eds.). Springer International Publishing, Cham, 1–15. DOI:
[2]
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. 2021. Muppet: Massive multi-task representations with pre-finetuning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 5799–5811. DOI:DOI:
[3]
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2021. Better fine-tuning by reducing representational collapse. In Proceedings of the 9th International Conference on Learning Representations (ICLR’21). OpenReview.net. Retrieved from https://openreview.net/forum?id=OQ08SN70M1V
[4]
Roee Aharoni, Shashi Narayan, Joshua Maynez, Jonathan Herzig, Elizabeth Clark, and Mirella Lapata. 2022. mFACE: Multilingual summarization with factual consistency evaluation. CoRR abs/2212.10622 (2022).
[5]
Hassan Aliakbarpour, Mohammad Taghi Manzuri, and Amir Masoud Rahmani. 2022. Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism. J. Supercomput. 78, 2 (2022), 2528–2555. DOI:DOI:
[6]
Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saeid Safaei, Elizabeth D. Trippe, Juan B. Gutierrez, and Krys Kochut. 2017. Text summarization techniques: A brief survey. Int. J. Advan. Comput. Sci. Applic. 8, 10 (2017). DOI:DOI:
[7]
Ayham Alomari, Norisma Idris, Aznul Qalid Md Sabri, and Izzat Alsmadi. 2022. Deep reinforcement and transfer learning for abstractive text summarization: A review. Comput. Speech Lang. 71 (2022), 101276. DOI:DOI:
[8]
Reinald Kim Amplayo and Mirella Lapata. 2021. Informative and controllable opinion summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL’21), Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, 2662–2672. DOI:DOI:
[9]
Reinald Kim Amplayo, Seonjae Lim, and Seung-won Hwang. 2018. Entity commonsense representation for neural abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 697–707. DOI:DOI:
[10]
Chenxin An, Ming Zhong, Zhichao Geng, Jianqiang Yang, and Xipeng Qiu. 2021. RetrievalSum: A retrieval enhanced framework for abstractive summarization. CoRR abs/2109.07943 (2021).
[11]
Dang Trung Anh and Nguyen Thi Thu Trang. 2019. Abstractive text summarization using pointer-generator networks with pre-trained word embedding. In Proceedings of the 10th International Symposium on Information and Communication Technology. ACM, 473–478. DOI:DOI:
[12]
Gemini Team Google: Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Jack Krawczyk, Cosmo Du, Ed Chi, Heng-Tze Cheng, Eric Ni, Purvi Shah, Patrick Kane, Betty Chan, Manaal Faruqui, Aliaksei Severyn, Hanzhao Lin, YaGuang Li, Yong Cheng, Abe Ittycheriah, Mahdis Mahdieh, Mia Chen, Pei Sun, Dustin Tran, Sumit Bagri, Balaji Lakshminarayanan, Jeremiah Liu, Andras Orban, Fabian Güra, Hao Zhou, Xinying Song, Aurelien Boffy, Harish Ganapathy, Steven Zheng, HyunJeong Choe, Ágoston Weisz, Tao Zhu, Yifeng Lu, Siddharth Gopal, Jarrod Kahn, Maciej Kula, Jeff Pitman, Rushin Shah, Emanuel Taropa, Majd Al Merey, Martin Baeuml, Zhifeng Chen, Laurent El Shafey, Yujing Zhang, Olcan Sercinoglu, George Tucker, Enrique Piqueras, Maxim Krikun, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, Alexandre Frechette, Charlotte Smith, Laura Culp, Lev Proleev, Yi Luan, and Xi Chen. 2023. Gemini: A family of highly capable multimodal models. CoRR abs/2312.11805 (2023). DOI:DOI:arXiv:2312.11805
[13]
Anthropic. 2023. The Claude 3 Model Family: Opus, Sonnet, Haiku. Retrieved from https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf
[14]
Hira Javed Anubha Agrawal, Sakshi Saraswat. 2019. A pointer generator network model to automatic text summarization and headline generation. Int. J. Eng. Advan. Technol. 8, 5S3 (Sep. 2019), 447–451. DOI:DOI:
[15]
Yash Kumar Atri, Shraman Pramanick, Vikram Goyal, and Tanmoy Chakraborty. 2021. See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization. Knowl.-based Syst. 227 (2021), 107152. DOI:DOI:
[16]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), Yoshua Bengio and Yann LeCun (Eds.). Retrieved from http://arxiv.org/abs/1409.0473
[17]
Yu Bai, Yang Gao, and Heyan Huang. 2021. Cross-lingual abstractive summarization with limited parallel resources. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 6910–6924. DOI:DOI:
[18]
Vidhisha Balachandran, Hannaneh Hajishirzi, William W. Cohen, and Yulia Tsvetkov. 2022. Correcting diverse factual errors in abstractive summarization via post-editing and language model infilling. CoRR abs/2210.12378 (2022).
[19]
Vidhisha Balachandran, Artidoro Pagnoni, Jay Yoon Lee, Dheeraj Rajagopal, Jaime G. Carbonell, and Yulia Tsvetkov. 2021. StructSum: Summarization via structured representations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL’21), Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, 2575–2585. DOI:DOI:
[20]
J. Balaji, T. V. Geetha, and Ranjani Parthasarathi. 2016. Abstractive summarization: A hybrid approach for the compression of semantic graphs. Int. J. Semant. Web Inf. Syst. 12, 2 (Apr. 2016), 76–99. DOI:DOI:
[21]
Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. Association for Computational Linguistics, 65–72. Retrieved from https://aclanthology.org/W05-0909
[22]
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, and Hsiao-Wuen Hon. 2020. UniLMv2: Pseudo-masked language models for unified language model pre-training. In Proceedings of the 37th International Conference on Machine Learning (ICML’20)(Proceedings of Machine Learning Research, Vol. 119). PMLR, 642–652. Retrieved from http://proceedings.mlr.press/v119/bao20a.html
[23]
Tal Baumel, Matan Eyal, and Michael Elhadad. 2018. Query focused abstractive summarization: Incorporating query relevance, multi-document coverage, and summary length constraints into seq2seq models. CoRR abs/1801.07704 (2018).
[24]
Rupal Bhargava, Yashvardhan Sharma, and Gargi Sharma. 2016. ATSSI: Abstractive text summarization using sentiment infusion. Procedia Comput. Sci. 89 (Jan. 2016), 404–411. DOI:DOI:
[25]
Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, and Luo Si. 2020. PALM: Pre-training an autoencoding&autoregressive language model for context-conditioned generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 8681–8691. DOI:DOI:
[26]
Arthur Brazinskas, Mirella Lapata, and Ivan Titov. 2020. Few-shot learning for opinion summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 4119–4135. DOI:DOI:
[27]
Xiaoyan Cai, Kaile Shi, Yuehan Jiang, Libin Yang, and Sen Liu. 2021. HITS-based attentional neural model for abstractive summarization. Knowl.-based Syst. 222 (2021), 106996. DOI:DOI:
[28]
Meng Cao, Yue Dong, and Jackie Cheung. 2022. Hallucinated but factual! Inspecting the factuality of hallucinations in abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 3340–3354. DOI:DOI:
[29]
Meng Cao, Yue Dong, Jingyi He, and Jackie Chi Kit Cheung. 2023. Learning with rejection for abstractive text summarization. CoRR abs/2302.08531 (2023).
[30]
Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstractive summarization models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 6251–6258. DOI:DOI:
[31]
Shuyang Cao and Lu Wang. 2021. Attention head masking for inference time content selection in abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 5008–5016. DOI:DOI:
[32]
Shuyang Cao and Lu Wang. 2021. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 6633–6649. DOI:DOI:
[33]
Yue Cao, Xiaojun Wan, Jin-ge Yao, and Dian Yu. 2020. MultiSumm: Towards a unified model for multi-lingual abstractive summarization. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI’20), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’20). AAAI Press, 11–18. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/5328
[34]
Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), the 30th Innovative Applications of Artificial Intelligence (IAAI’18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’18), Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 4784–4791. Retrieved from https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16121
[35]
Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 1662–1675. DOI:DOI:
[36]
Hou Pong Chan, Lu Wang, and Irwin King. 2021. Controllable summarization with constrained Markov decision process. Trans. Assoc. Comput. Ling. 9 (2021), 1213–1232. DOI:DOI:
[37]
Y. Chang, H. Lei, X. Li, and Y. Huang. 2019. A reinforced improved attention model for abstractive text summarization. In Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation (PACLIC’19).362–369.
[38]
Kushal Chawla, Balaji Vasan Srinivasan, and Niyati Chhaya. 2019. Generating formality-tuned summaries using input-dependent rewards. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL’19), Mohit Bansal and Aline Villavicencio (Eds.). Association for Computational Linguistics, 833–842. DOI:DOI:
[39]
Kai Chen, Baotian Hu, Qingcai Chen, and Hong Yu. 2019. A neural abstractive summarization model guided with topic sentences. Aust. J. Intell. Inf. Process. Syst. 17, 1 (2019), 48–53. Retrieved from http://ajiips.com.au/papers/V17.1/v17n1_52-57.pdf
[40]
Sihao Chen, Fan Zhang, Kazoo Sone, and Dan Roth. 2021. Improving faithfulness in abstractive summarization with contrast candidate generation and selection. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 5935–5941. DOI:DOI:
[41]
Xuewen Chen, Jinlong Li, and Hai-Han Wang. 2019. Keyphrase guided beam search for neural abstractive text summarization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’19). IEEE, 1–9. DOI:DOI:
[42]
Xiuying Chen, Mingzhe Li, Xin Gao, and Xiangliang Zhang. 2022. Towards improving faithfulness in abstractive summarization. CoRR abs/2210.01877 (2022).
[43]
Yongchao Chen, Xin He, Guanghui Wang, and Junyang Yu. 2021. Improving text summarization using feature extraction approach based on pointer-generator with coverage. In Proceedings of the 18th International Conference on Web Information Systems and Applications (WISA’21)(Lecture Notes in Computer Science, Vol. 12999), Chunxiao Xing, Xiaoming Fu, Yong Zhang, Guigang Zhang, and Chaolemen Borjigin (Eds.). Springer, 489–496. DOI:DOI:
[44]
Yisong Chen and Qing Song. 2021. News text summarization method based on BART-TextRank model. In Proceedings of the IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC’21). IEEE. DOI:DOI:
[45]
Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18), Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, 675–686. DOI:DOI:
[46]
Jackie Chi Kit Cheung and Gerald Penn. 2014. Unsupervised sentence enhancement for automatic summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics, 775–786. DOI:DOI:
[47]
Hyungtak Choi, Lohith Ravuru, Tomasz Dryjanski, Sunghan Rye, Donghyun Lee, Hojung Lee, and Inchul Hwang. 2019. VAE-PGN based abstractive model in multi-stage architecture for text summarization. In Proceedings of the 12th International Conference on Natural Language Generation (INLG’19), Kees van Deemter, Chenghua Lin, and Hiroya Takamura (Eds.). Association for Computational Linguistics, 510–515. DOI:DOI:
[48]
YunSeok Choi, Dahae Kim, and Jee-Hyong Lee. 2018. Abstractive summarization by neural attention model with document content memory. In Proceedings of the Conference on Research in Adaptive and Convergent Systems (RACS’18), Chih-Cheng Hung and Lamjed Ben Said (Eds.). ACM, 11–16. DOI:DOI:
[49]
Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16), Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). The Association for Computational Linguistics, 93–98. DOI:DOI:
[50]
Prafulla Kumar Choubey, Alexander R. Fabbri, Jesse Vig, Chien-Sheng Wu, Wenhao Liu, and Nazneen Fatema Rajani. 2021. CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization. arXiv e-prints (2021), arXiv–2110.
[51]
Tanya Chowdhury, Sachin Kumar, and Tanmoy Chakraborty. 2020. Neural abstractive summarization with structural attention. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20), Christian Bessiere (Ed.). ijcai.org, 3716–3722. DOI:DOI:
[52]
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All that’s “human” is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 7282–7296. DOI:DOI:
[53]
Maximin Coavoux, Hady Elsahar, and Matthias Gallé. 2019. Unsupervised aspect-based multi-document abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization. Association for Computational Linguistics, 42–47. DOI:DOI:
[54]
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 615–621. DOI:DOI:
[55]
Divyanshu Daiya. 2020. Combining temporal event relations and pre-trained language models for text summarization. In Proceedings of the 19th IEEE International Conference on Machine Learning and Applications (ICMLA’20), M. Arif Wani, Feng Luo, Xiaolin Andy Li, Dejing Dou, and Francesco Bonchi (Eds.). IEEE, 641–646. DOI:DOI:
[56]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171–4186. DOI:DOI:
[57]
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’19), Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 13042–13054. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/c20bb2d9a50d5ac1f713f8b34d9aac5a-Abstract.html
[58]
Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu. 2020. Multi-fact correction in abstractive text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 9320–9331. DOI:DOI:
[59]
Yue Dong, John Wieting, and Pat Verga. 2022. Faithful to the document or to the world? Mitigating hallucinations via entity-linked knowledge in abstractive summarization. CoRR abs/2204.13761 (2022).
[60]
Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2021. GSum: A general framework for guided neural abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 4830–4842. DOI:DOI:
[61]
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. GLM: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 320–335. DOI:DOI:
[62]
H. P. Edmundson. 1969. New methods in automatic extracting. J. ACM 16, 2 (Apr. 1969), 264–285. DOI:DOI:
[63]
Matan Eyal, Tal Baumel, and Michael Elhadad. 2019. Question answering as an automatic evaluation metric for news article summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 3938–3948. DOI:DOI:
[64]
Alexander R. Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq R. Joty, Dragomir R. Radev, and Yashar Mehdad. 2021. Improving zero and few-shot abstractive summarization with intermediate fine-tuning and data augmentation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 704–717. DOI:DOI:
[65]
Angela Fan, David Grangier, and Michael Auli. 2018. Controllable abstractive summarization. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, (NMT@ACL’18)Alexandra Birch, Andrew M. Finch, Minh-Thang Luong, Graham Neubig, and Yusuke Oda (Eds.). Association for Computational Linguistics, 45–54. DOI:DOI:
[66]
Patrick Fernandes, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Structured neural summarization. In Proceedings of the 7th International Conference on Learning Representations (ICLR’19). OpenReview.net. Retrieved from https://openreview.net/forum?id=H1ersoRqtm
[67]
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, and Yejin Choi. 2021. Discourse understanding and factual consistency in abstractive summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL’21), Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, 435–447. DOI:DOI:
[68]
Kavita Ganesan, ChengXiang Zhai, and Jiawei Han. 2010. Opinosis: A graph based approach to abstractive summarization of highly redundant opinions. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING’10), Chu-Ren Huang and Dan Jurafsky (Eds.). Tsinghua University Press, 340–348. Retrieved from https://aclanthology.org/C10-1039/
[69]
Yang Gao, Yang Wang, Luyang Liu, Yi-Di Guo, and Heyan Huang. 2020. Neural abstractive summarization fusing by global generative topics. Neural Comput. Appl. 32, 9 (2020), 5049–5058. DOI:DOI:
[70]
Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. J. Artif. Intell. Res. 61 (2018), 65–170. DOI:DOI:
[71]
Sebastian Gehrmann, Yuntian Deng, and Alexander M. Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4098–4109. DOI:DOI:
[72]
Pierre-Etienne Genest and Guy Lapalme. 2012. Fully abstractive approach to guided summarization. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, 354–358. Retrieved from https://aclanthology.org/P12-2069/
[73]
Asish Ghoshal, Arash Einolghozati, Ankit Arun, Haoran Li, Lili Yu, Yashar Mehdad, Scott Wen-tau Yih, and Asli Celikyilmaz. 2022. Improving faithfulness of abstractive summarization by controlling confounding effect of irrelevant sentences. CoRR abs/2212.09726 (2022).
[74]
David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2003. English gigaword. Ling. Data Consort., Philad. 4, 1 (2003), 34.
[75]
Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18). Association for Computational Linguistics, 708–719. DOI:DOI:
[76]
Yong Guan, Shaoru Guo, Ru Li, Xiaoli Li, and Hu Zhang. 2021. Frame semantics guided network for abstractive sentence summarization. Knowl.-based Syst. 221 (2021), 106973. DOI:DOI:
[77]
Yong Guan, Shaoru Guo, Ru Li, Xiaoli Li, and Hu Zhang. 2021. Integrating semantic scenario and word relations for abstractive sentence summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 2522–2529. DOI:DOI:
[78]
Min Gui, Junfeng Tian, Rui Wang, and Zhenglu Yang. 2019. Attention optimization for abstractive document summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 1222–1228. DOI:DOI:
[79]
Chulaka Gunasekara, Guy Feigenblat, Benjamin Sznajder, Ranit Aharonov, and Sachindra Joshi. 2021. Using question answering rewards to improve abstractive summarization. In Findings of the Association for Computational Linguistics (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 518–526. DOI:DOI:
[80]
Som Gupta and Sanjai Kumar Gupta. 2019. Abstractive summarization: An overview of the state of the art. Expert Syst. Appl. 121 (2019), 49–65. DOI:DOI:
[81]
Xu-Wang Han, Hai-Tao Zheng, Jin-Yuan Chen, and Cong-Zhi Zhao. 2019. Diverse decoding for abstractive document summarization. Appl. Sci. 9, 3 (Jan. 2019), 386. DOI:DOI:
[82]
Puruso Muhammad Hanunggul and Suyanto Suyanto. 2019. The impact of local attention in LSTM for abstractive text summarization. In Proceedings of the International Seminar on Research of Information Technology and Intelligent Systems (ISRITI’19). 54–57. DOI:DOI:
[83]
Zepeng Hao, Yupu Guo, Cheng Gong, and Honghui Chen. 2019. Hybrid non-local network for abstractive summarization. IOP Conf. Series: Mater. Sci. Eng. 692, 1 (Nov. 2019), 012049. DOI:DOI:
[84]
Zepeng Hao, Jingzhou Ji, Tao Xie, and Bin Xue. 2020. Abstractive summarization model with a feature-enhanced Seq2Seq structure. In Proceedings of the 5th Asia-Pacific Conference on Intelligent Robot Systems (ACIRS’20). IEEE, 163–167. DOI:DOI:
[85]
Zepeng Hao, Taihua Shao, Shengwei Zhou, and Honghui Chen. 2019. Memory-enhanced abstractive summarization. J. Phys.: Conf. Series 1229, 1 (May 2019), 012063. DOI:DOI:
[86]
Hardy and Andreas Vlachos. 2018. Guided neural language generation for abstractive summarization using abstract meaning representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 768–773. DOI:DOI:
[87]
Donna Harman and Paul Over. 2004. The effects of human variation in DUC summarization evaluation. In Text Summarization Branches Out. Association for Computational Linguistics, 10–17. Retrieved from https://aclanthology.org/W04-1003
[88]
Junxian He, Wojciech Kryscinski, Bryan McCann, Nazneen Rajani, and Caiming Xiong. 2022. CTRLsum: Towards generic controllable text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’22), Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, 5879–5915. Retrieved from https://aclanthology.org/2022.emnlp-main.396
[89]
Pengcheng He, Baolin Peng, Liyang Lu, Song Wang, Jie Mei, Yang Liu, Ruochen Xu, Hany Hassan Awadalla, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, and Xuedong Huang. 2022. Z-Code++: A pre-trained language model optimized for abstractive summarization. CoRR abs/2208.09770 (2022).
[90]
Wan Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A unified model for extractive and abstractive summarization using inconsistency loss. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18), Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, 132–141. DOI:DOI:
[91]
Jie Huang and Yifan Jiang. 2021. A DAE-based approach for improving the grammaticality of summaries. In Proceedings of the International Conference on Computers and Automation (CompAuto’21). IEEE. DOI:DOI:
[92]
Luyang Huang, Shuyang Cao, Nikolaus Nova Parulian, Heng Ji, and Lu Wang. 2021. Efficient attentions for long document summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 1419–1436. DOI:DOI:
[93]
Luyang Huang, Lingfei Wu, and Lu Wang. 2020. Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5094–5107. DOI:DOI:
[94]
Yuxin Huang, Zhengtao Yu, Junjun Guo, Yan Xiang, Zhiqiang Yu, and Yantuan Xian. 2022. Abstractive document summarization via multi-template decoding. Appl. Intell. 52, 9 (Jan. 2022). DOI:DOI:
[95]
Neslihan Iskender, Tim Polzehl, and Sebastian Möller. 2021. Reliability of human evaluation for text summarization: Lessons learned and challenges ahead. In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval’21). Association for Computational Linguistics, 86–96. Retrieved from https://aclanthology.org/2021.humeval-1.10
[96]
Heewon Jang and Wooju Kim. 2021. Reinforced abstractive text summarization with semantic added reward. IEEE Access 9 (2021), 103804–103810. DOI:DOI:
[97]
Xin Ji and Wen Zhao. 2021. SKGSUM: Abstractive document summarization with semantic knowledge graphs. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’21). IEEE, 1–8. DOI:DOI:
[98]
Ruipeng Jia, Yannan Cao, Fang Fang, Jinpeng Li, Yanbing Liu, and Pengfei Yin. 2020. Enhancing textual representation for abstractive summarization: Leveraging masked decoder. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’20). IEEE, 1–8. DOI:DOI:
[99]
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. CoRR abs/2310.06825 (2023).
[100]
Yichen Jiang and Mohit Bansal. 2018. Closed-book training to improve summarization encoder memory. In Proceedings of the Conference on Empirical Methods in Natural Language (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4067–4077. DOI:DOI:
[101]
Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, and Jianfeng Gao. 2021. Enriching transformers with structured tensor-product representations for abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 4780–4793. DOI:DOI:
[102]
Dawei Jin, Ruizhi Kang, Hongjun Zhang, Wening Hao, and Gang Chen. 2020. Improving abstractive summarization via dilated convolution. J. Phys.: Conf. Series 1616, 1 (Aug. 2020), 012078. DOI:DOI:
[103]
Hanqi Jin, Tianming Wang, and Xiaojun Wan. 2020. SemSUM: Semantic dependency guided neural abstractive summarization. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI’20), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’20). AAAI Press, 8026–8033. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/6312
[104]
Sung-Guk Jo, Jeong-Jae Kim, and Byung-Won On. 2022. Learning cluster patterns for abstractive summarization. CoRR abs/2202.10967 (2022).
[105]
Praveen Kumar Katwe, Aditya Khamparia, Deepak Gupta, and Ashit Kumar Dutta. 2023. Methodical systematic review of abstractive summarization and natural language processing models for biomedical health informatics: Approaches, metrics and challenges. ACM Trans. Asian Low-Resour. Lang. Inf. Process. (May 2023). DOI:
[106]
Shirin Akther Khanam, Fei Liu, and Yi-Ping Phoebe Chen. 2021. Joint knowledge-powered topic level attention for a convolutional text summarization model. Knowl.-based Syst. 228 (2021), 107273. DOI:DOI:
[107]
Heechan Kim and Soowon Lee. 2019. A context based coverage model for abstractive document summarization. In Proceedings of the International Conference on Information and Communication Technology Convergence (ICTC’19). IEEE, 1129–1132. DOI:DOI:
[108]
Heechan Kim and Soowon Lee. 2019. Document summarization model based on general context in RNN. J. Inf. Process. Syst. 15, 6 (2019), 1378–1391. DOI:DOI:
[109]
Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, and Doug Downey. 2022. Don’t say what you don’t know: Improving the consistency of abstractive summarization by constraining beam search. CoRR abs/2203.08436 (2022).
[110]
Barbara Ann Kitchenham and Stuart Charters. 2007. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE 2007-001. Keele University and Durham University Joint Report. Retrieved from https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf
[111]
Huan Yee Koh, Jiaxin Ju, Ming Liu, and Shirui Pan. 2023. An empirical survey on long document summarization: Datasets, models, and metrics. ACM Comput. Surv. 55, 8 (2023), 154:1–154:35. DOI:DOI:
[112]
Panagiotis Kouris, Georgios Alexandridis, and Andreas Stafylopatis. 2019. Abstractive text summarization based on deep learning and semantic content generalization. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL’19), Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 5082–5092. DOI:DOI:
[113]
Panagiotis Kouris, Georgios Alexandridis, and Andreas Stafylopatis. 2021. Abstractive text summarization: Enhancing sequence-to-sequence models using word sense disambiguation and semantic content generalization. Comput. Ling. 47, 4 (2021), 813–859. DOI:DOI:
[114]
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 9332–9346. DOI:DOI:
[115]
Wojciech Kryscinski, Romain Paulus, Caiming Xiong, and Richard Socher. 2018. Improving abstraction in text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 1808–1817. DOI:DOI:
[116]
Hong-Seok Kwon, Byung-Hyun Go, Juhong Park, WonKee Lee, Yewon Jeong, and Jong-Hyeok Lee. 2021. Gated dynamic convolutions with deep layer fusion for abstractive document summarization. Comput. Speech Lang. 66 (2021), 101159. DOI:DOI:
[117]
Philippe Laban, Andrew Hsi, John F. Canny, and Marti A. Hearst. 2020. The summary loop: Learning to write abstractive summaries without examples. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5135–5150. DOI:DOI:
[118]
Faisal Ladhak, Esin Durmus, He He, Claire Cardie, and Kathleen R. McKeown. 2022. Faithful or extractive? On mitigating the faithfulness-abstractiveness trade-off in abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 1410–1421. DOI:DOI:
[119]
Tung Le and Le Minh Nguyen. 2018. Combined objective function in deep learning model for abstractive summarization. In Proceedings of the 9th International Symposium on Information and Communication Technology (SoICT’18). ACM, 84–91. DOI:DOI:
[120]
HyunSoo Lee, YunSeok Choi, and Jee-Hyong Lee. 2020. Attention history-based attention for abstractive text summarization. In Proceedings of the 35th ACM/SIGAPP Symposium on Applied Computing (SAC’20), Chih-Cheng Hung, Tomás Cerný, Dongwan Shin, and Alessio Bechini (Eds.). ACM, 1075–1081. DOI:DOI:
[121]
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 7871–7880. DOI:DOI:
[122]
Chenliang Li, Weiran Xu, Si Li, and Sheng Gao. 2018. Guiding generation for abstractive text summarization based on key information guide network. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 55–60. DOI:DOI:
[123]
Haoran Li, Song Xu, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, and Bowen Zhou. 2021. Learn to copy from the copying history: Correlational copy network for abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 4091–4101. DOI:DOI:
[124]
Haoran Li, Junnan Zhu, Jiajun Zhang, and Chengqing Zong. 2018. Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization. In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18), Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, 1430–1441. Retrieved from https://aclanthology.org/C18-1121/
[125]
Haoran Li, Junnan Zhu, Jiajun Zhang, Chengqing Zong, and Xiaodong He. 2020. Keywords-guided abstractive sentence summarization. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI’20), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’20). AAAI Press, 8196–8203. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/6333
[126]
Jinpeng Li, Chuang Zhang, Xiaojun Chen, Yanan Cao, and Ruipeng Jia. 2020. Improving abstractive summarization with iterative representation. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’20). IEEE, 1–8. DOI:DOI:
[127]
Jinpeng Li, Chuang Zhang, Xiaojun Chen, Yanan Cao, Pengcheng Liao, and Peng Zhang. 2019. Abstractive text summarization with multi-head attention. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’19). IEEE, 1–8. DOI:DOI:
[128]
Lei Li, Wei Liu, Marina Litvak, Natalia Vanetik, and Zuying Huang. 2019. In conclusion not repetition: Comprehensive abstractive summarization with diversified attention based on determinantal point processes. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL’19). Association for Computational Linguistics, 822–832. DOI:DOI:
[129]
Shuaimin Li and Jungang Xu. 2021. A two-step abstractive summarization model with asynchronous and enriched-information decoding. Neural Comput. Appl. 33, 4 (2021), 1159–1170. DOI:DOI:
[130]
Wei Li, Xinyan Xiao, Yajuan Lyu, and Yuanzhuo Wang. 2018. Improving neural abstractive document summarization with explicit information selection modeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 1787–1796. DOI:DOI:
[131]
Wei Li, Xinyan Xiao, Yajuan Lyu, and Yuanzhuo Wang. 2018. Improving neural abstractive document summarization with structural regularization. In Proceedings of the Conference on Empirical Methods in Natural Language (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4078–4087. DOI:DOI:
[132]
Zhenwen Li, Wenhao Wu, and Sujian Li. 2020. Composing elementary discourse units in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6191–6196. DOI:DOI:
[133]
Qingmin Liang, Ling Lu, Tianji Chang, and Wu Yang. 2020. CFCSS: Based on CF network convolutional Seq2Seq model for abstractive summarization. In Proceedings of the 15th IEEE Conference on Industrial Electronics and Applications (ICIEA’20). 1160–1164. DOI:DOI:ISSN: 2158-2297.
[134]
Xiaobo Liang, Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin, Wei Chen, Min Zhang, and Tie-Yan Liu. 2021. R-Drop: Regularized dropout for neural networks. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’21), Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.). 10890–10905. Retrieved from https://proceedings.neurips.cc/paper/2021/hash/5a66b9200f29ac3fa0ae244cc2a51b39-Abstract.html
[135]
Pengcheng Liao, Chuang Zhang, Xiaojun Chen, and Xiaofei Zhou. 2020. Improving abstractive text summarization with history aggregation. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’20). IEEE, 1–9. DOI:DOI:
[136]
Weizhi Liao, Yaheng Ma, Yanchao Yin, Guanglei Ye, and Dongzhou Zuo. 2021. Improving abstractive summarization based on dynamic residual network with reinforce dependency. Neurocomputing 448 (2021), 228–237. DOI:DOI:
[137]
Opher Lieber, Or Sharir, Barak Lenz, and Yoav Shoham. 2021. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs 1 (2021), 9.
[138]
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics, 74–81.
[139]
Wuhang Lin, Jianling Li, Zibo Yi, Bin Ji, Shasha Li, Jie Yu, and Jun Ma. 2021. A unified summarization model with semantic guide and keyword coverage mechanism. In Proceedings of the 30th International Conference on Artificial Neural Networks: Artificial Neural Networks and Machine Learning (ICANN’21)(Lecture Notes in Computer Science, Vol. 12895), Igor Farkas, Paolo Masulli, Sebastian Otte, and Stefan Wermter (Eds.). Springer, 333–344. DOI:DOI:
[140]
Fei Liu, Jeffrey Flanigan, Sam Thomson, Norman M. Sadeh, and Noah A. Smith. 2015. Toward abstractive summarization using semantic representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’15), Rada Mihalcea, Joyce Yue Chai, and Anoop Sarkar (Eds.). The Association for Computational Linguistics, 1077–1086. DOI:DOI:
[141]
Junpeng Liu, Yanyan Zou, Hainan Zhang, Hongshen Chen, Zhuoye Ding, Caixia Yuan, and Xiaojie Wang. 2021. Topic-aware contrastive learning for abstractive dialogue summarization. In Findings of the Association for Computational Linguistics (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 1229–1243. DOI:DOI:
[142]
Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018. Generative adversarial network for abstractive text summarization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), the 30th Innovative Applications of Artificial Intelligence (IAAI’18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’18), Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 8109–8110. Retrieved from https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16238
[143]
Qing Liu, Lei Chen, and Yuan Yuan. 2021. Abstractive summarization with word embedding prediction and scheduled sampling. In Proceedings of the 6th International Conference on Automation, Control and Robotics Engineering (CACRE’21), Fumin Zhang and Ying Zhao (Eds.). IEEE, 316–321. DOI:DOI:
[144]
Qing Liu, Lei Chen, Yuan Yuan, and Huarui Wu. 2021. History reuse and bag-of-words loss for long summary generation. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 2551–2560. DOI:DOI:
[145]
Shuaiqi Liu, Jiannong Cao, Ruosong Yang, and Zhiyuan Wen. 2022. Key phrase aware transformer for abstractive summarization. Inf. Process. Manag. 59, 3 (May 2022), 102913. DOI:DOI:
[146]
Wei Liu, Huanqin Wu, Wenjing Mu, Zhen Li, Tao Chen, and Dan Nie. 2021. CO2Sum: Contrastive learning for factual-consistent abstractive summarization. CoRR abs/2112.01147 (2021).
[147]
Xin Liu and Liutong Xv. 2019. Abstract summarization based on the combination of transformer and LSTM. In Proceedings of the International Conference on Intelligent Computing, Automation and Systems (ICICAS’19). 923–927. DOI:DOI:
[148]
Yizhu Liu, Qi Jia, and Kenny Q. Zhu. 2021. Keyword-aware abstractive summarization by extracting set-level intermediate summaries. In Proceedings of the Web Conference (WWW’21), Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia (Eds.). ACM/IW3C2, 3042–3054. DOI:DOI:
[149]
Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 3728–3738. DOI:DOI:
[150]
Yixin Liu and Pengfei Liu. 2021. SimCLS: A simple framework for contrastive learning of abstractive summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 1065–1072. DOI:DOI:
[151]
Yixin Liu, Pengfei Liu, Dragomir R. Radev, and Graham Neubig. 2022. BRIO: Bringing order to abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 2890–2903. DOI:DOI:
[152]
Yizhu Liu, Zhiyi Luo, and Kenny Q. Zhu. 2018. Controlling length in abstractive summarization using a convolutional neural network. In Proceedings of the Conference on Empirical Methods in Natural Language (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4110–4119. DOI:DOI:
[153]
Qian Lou, Ting Hua, Yen-Chang Hsu, Yilin Shen, and Hongxia Jin. 2022. DictFormer: Tiny transformer with shared dictionary. In Proceedings of the 10th International Conference on Learning Representations (ICLR’22). OpenReview.net. Retrieved from https://openreview.net/forum?id=GWQWAeE9EpB
[154]
Yao Lu, Linqing Liu, Zhile Jiang, Min Yang, and Randy Goebel. 2019. A multi-task learning framework for abstractive text summarization. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), the 31st Innovative Applications of Artificial Intelligence Conference (IAAI’19), the 9th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’19). AAAI Press, 9987–9988. DOI:DOI:
[155]
Hans Peter Luhn. 1958. The automatic creation of literature abstracts. IBM J. Res. Devel. 2, 2 (1958), 159–165.
[156]
Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15), Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). The Association for Computational Linguistics, 1412–1421. DOI:DOI:
[157]
Tinghuai Ma, Qian Pan, Huan Rong, Yurong Qian, Yuan Tian, and Najla Al-Nabhan. 2022. T-BERTSum: Topic-aware text summarization based on BERT. IEEE Trans. Comput. Soc. Syst. 9, 3 (2022), 879–890. DOI:
[158]
Ye Ma, Zixun Lan, Lu Zong, and Kaizhu Huang. 2021. Global-aware beam search for neural abstractive summarization. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’21), Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.). 16545–16557. Retrieved from https://proceedings.neurips.cc/paper/2021/hash/89d4402dc03d3b7318bbac10203034ab-Abstract.html
[159]
Qianren Mao, Jianxin Li, Hao Peng, Shizhu He, Lihong Wang, Philip S. Yu, and Zheng Wang. 2022. Fact-driven abstractive summarization by utilizing multi-granular multi-relational knowledge. IEEE/ACM Trans. Audio, Speech, Lang. Process. 30, 3 (2022), 1–1. DOI:DOI:
[160]
Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed Hassan Awadallah, and Dragomir R. Radev. 2022. DYLE: Dynamic latent extraction for abstractive long-input summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 1687–1698. DOI:DOI:
[161]
Andreas Marfurt and James Henderson. 2021. Sentence-level planning for especially abstractive summarization. In Proceedings of the 3rd Workshop on New Frontiers in Summarization. Association for Computational Linguistics. DOI:DOI:
[162]
Abu Kaisar Mohammad Masum, Sheikh Abujar, Md Ashraful Islam Talukder, A. K. M. Shahariar Azad Rabby, and Syed Akhter Hossain. 2019. Abstractive method of text summarization with sequence to sequence RNNs. In Proceedings of the 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT’19). IEEE, 1–5. DOI:DOI:
[163]
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan T. McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 1906–1919. DOI:DOI:
[164]
Julian J. McAuley and Jure Leskovec. 2013. From amateurs to connoisseurs: Modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd International World Wide Web Conference (WWW’13), Daniel Schwabe, Virgílio A. F. Almeida, Hartmut Glaser, Ricardo Baeza-Yates, and Sue B. Moon (Eds.). International World Wide Web Conferences Steering Committee/ACM, 897–908. DOI:DOI:
[165]
Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally typical sampling. Trans. Assoc. Comput. Ling. 11 (2023), 102–121. Retrieved from https://transacl.org/ojs/index.php/tacl/article/view/3993
[166]
George A Miller. 1995. WordNet: A lexical database for English. Commun. ACM 38, 11 (1995), 39–41.
[167]
Ibrahim F. Moawad and Mostafa Aref. 2012. Semantic graph reduction approach for abstractive text summarization. In Proceedings of the 7th International Conference on Computer Engineering Systems (ICCES’12). 132–138. DOI:DOI:
[168]
Shivangi Modi and Rachana Oza. 2018. Review on abstractive text summarization techniques (ATST) for single and multi documents. In Proceedings of the International Conference on Computing, Power and Communication Technologies (GUCON’18). IEEE, 1173–1176.
[169]
Dennis Singh Moirangthem and Minho Lee. 2020. Abstractive summarization of long texts by representing multiple compositionalities with temporal hierarchical pointer generator network. Neural Netw. 124 (2020), 1–11. DOI:DOI:
[170]
N. Moratanch and Chitrakala Gopalan.2019. A novel framework for semantic OrientedAbstractive text summarization. J. Web Eng. 17, 8 (2019), 675–716. DOI:DOI:
[171]
Pranav Ajit Nair, and Anil Kumar Singh and. 2021. Improving abstractive summarization with commonsense knowledge. In Proceedings of the Student Research Workshop Associated with RANLP’21. INCOMA Ltd.DOI:DOI:
[172]
Ramesh Nallapati, Bing Xiang, and Bowen Zhou. 2016. Sequence-to-sequence RNNs for text summarization. CoRR abs/1602.06023 (2016).
[173]
Ramesh Nallapati, Bowen Zhou, Cicero Nogueira dos Santos, Çaglar Gulçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL’16), Yoav Goldberg and Stefan Riezler (Eds.). ACL, 280–290. DOI:DOI:
[174]
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cícero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathy McKeown, and Bing Xiang. 2021. Entity-level factual consistency of abstractive text summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL’21), Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, 2727–2733. DOI:DOI:
[175]
Feng Nan, Cícero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen R. McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, and Bing Xiang. 2021. Improving factual consistency of abstractive summarization via question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 6881–6894. DOI:DOI:
[176]
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 1797–1807. DOI:DOI:
[177]
Shashi Narayan, Yao Zhao, Joshua Maynez, Gonçalo Simões, Vitaly Nikolaev, and Ryan T. McDonald. 2021. Planning with learned entity prompts for abstractive summarization. Trans. Assoc. Comput. Ling. 9 (2021), 1475–1492. DOI:DOI:
[178]
Preksha Nema, Mitesh M. Khapra, Anirban Laha, and Balaraman Ravindran. 2017. Diversity driven attention model for query-based abstractive summarization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1063–1072. DOI:DOI:
[179]
Jun-Ping Ng and Viktoria Abrecht. 2015. Better summarization evaluation with word embeddings for ROUGE. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). Association for Computational Linguistics, 1925–1930. DOI:DOI:
[180]
Israel C. T. Ngoko, Amlan Mukherjee, and Boniface Kabaso. 2018. Abstractive text summarization using recurrent neural networks: Systematic literature review. In Proceedings of the International Conference on Intellectual Capital, Knowledge Management & Organisational Learning (ICICKM’18). 435–439.
[181]
Minh-Phuc Nguyen and Nhi-Thao Tran. 2021. Improving abstractive summarization with segment-augmented and position-awareness. In Proceedings of the 5th International Conference on Arabic Computational Linguistics (ACLING’21)(Procedia Computer Science, Vol. 189), Khaled Shaalan and Samhaa R. El-Beltagy (Eds.). Elsevier, 167–174. DOI:DOI:
[182]
Toi Nguyen, Toai Le, and Nhi-Thao Tran. 2020. Abstractive sentence summarization with encoder-convolutional neural networks. In Proceedings of the 12th International Conference on Knowledge and Systems Engineering (KSE’20). IEEE, 13–18. DOI:DOI:
[183]
Thong Nguyen, Anh Tuan Luu, Truc Lu, and Tho Quan. 2021. Enriching and controlling global semantics for text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 9443–9456. DOI:DOI:
[184]
Wenbo Nie, Wei Zhang, Xinle Li, and Yao Yu. 2019. An abstractive summarizer based on improved pointer-generator network. In Proceedings of the 34th Youth Academic Annual Conference of Chinese Association of Automation (YAC’19). IEEE. DOI:DOI:
[185]
Nikola I. Nikolov and Richard H. R. Hahnloser. 2020. Abstractive document summarization without parallel data. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC’20), Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, 6638–6644. Retrieved from https://aclanthology.org/2020.lrec-1.819/
[186]
Jianwei Niu, Huan Chen, Qingjuan Zhao, Limin Su, and Mohammed Atiquzzaman. 2017. Multi-document abstractive summarization using chunk-graph and recurrent neural network. In Proceedings of the IEEE International Conference on Communications (ICC’17). IEEE, 1–6. DOI:DOI:
[187]
Jianwei Niu, Mingsheng Sun, Joel J. P. C. Rodrigues, and Xuefeng Liu. 2019. A novel attention mechanism considering decoder input for abstractive text summarization. In Proceedings of the IEEE International Conference on Communications (ICC’19). IEEE, 1–7. DOI:DOI:
[188]
Chitu Okoli and Kira Schabram. 2010. A guide to conducting a systematic literature review of information systems research. SSRN Electron. J. (2010). DOI:DOI:
[189]
OpenAI. 2023. GPT-4 technical report. CoRR abs/2303.08774 (2023).
[190]
Jessica Ouyang, Boya Song, and Kathy McKeown. 2019. A robust abstractive system for cross-lingual summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 2025–2031. DOI:DOI:
[191]
Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, Jennifer M. Tetzlaff, Elie A. Akl, Sue E. Brennan, Roger Chou, Julie Glanville, Jeremy M. Grimshaw, Asbjorn Hróbjartsson, Manoj M. Lalu, Tianjing Li, Elizabeth W. Loder, Evan Mayo-Wilson, Steve McDonald, Luke A. McGuinness, Lesley A. Stewart, James Thomas, Andrea C. Tricco, Vivian A. Welch, Penny Whiting, and David Moher. 2021. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 372 (2021).
[192]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. ACL, 311–318. DOI:DOI:
[193]
Ramakanth Pasunuru and Mohit Bansal. 2018. Multi-reward reinforced summarization with saliency and entailment. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 646–653. DOI:DOI:
[194]
Ramakanth Pasunuru, Han Guo, and Mohit Bansal. 2017. Towards improving abstractive summarization via entailment generation. In Proceedings of the Workshop on New Frontiers in Summarization (NFiS@EMNLP’17), Lu Wang, Jackie Chi Kit Cheung, Giuseppe Carenini, and Fei Liu (Eds.). Association for Computational Linguistics, 27–32. DOI:DOI:
[195]
Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In Proceedings of the 6th International Conference on Learning Representations (ICLR’18). OpenReview.net. Retrieved from https://openreview.net/forum?id=HkAClQgA-
[196]
Diogo Pernes, Afonso Mendes, and André F. T. Martins. 2022. Improving abstractive summarization with energy-based re-ranking. CoRR abs/2210.15553 (2022).
[197]
Daniele Pighin, Marco Cornolti, Enrique Alfonseca, and Katja Filippova. 2014. Modelling events through memory-based, open-IE patterns for abstractive summarization. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14). The Association for Computer Linguistics, 892–901. DOI:DOI:
[198]
Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 9308–9319. DOI:DOI:
[199]
Yulius Denny Prabowo, Achmad Imam Kristijantoro, H. L. H. S. Warnars, and Widodo Budiharto. 2021. Systematic literature review on abstractive text summarization using Kitchenham method. ICIC Express Lett., Part B: Applic. 12, 11 (2021), 1075–1080.
[200]
Chandra Prakash and Anupam Shukla. 2014. Human aided text summarizer SAAR using reinforcement learning. In Proceedings of the International Conference on Soft Computing and Machine Intelligence. 83–87. DOI:DOI:
[201]
Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. ProphetNet: Predicting future N-gram for sequence-to-sequence pre-training. In Findings of the Association for Computational Linguistics (EMNLP’20), Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 2401–2410. DOI:DOI:
[202]
Yifu Qiu and Shay B. Cohen. 2022. Abstractive summarization guided by latent hierarchical document structure. CoRR abs/2211.09458 (2022).
[203]
Dragomir R. Radev and Kathleen R. McKeown. 1998. Generating natural language summaries from multiple on-line sources. Comput. Ling. 24, 3 (1998), 469–500.
[204]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR abs/1910.10683 (2019).
[205]
Shaik Rafi and Ranjita Das. 2021. RNN encoder and decoder with teacher forcing attention mechanism for abstractive summarization. In Proceedings of the IEEE 18th India Council International Conference (INDICON’21). IEEE. DOI:DOI:
[206]
Md Motiur Rahman and Fazlul Hasan Siddiqui. 2019. An optimized abstractive text summarization model using peephole convolutional LSTM. Symmetry 11, 10 (2019), 1290. DOI:DOI:
[207]
Md. Motiur Rahman and Fazlul Hasan Siddiqui. 2020. Multi-layered attentional peephole convolutional LSTM for abstractive text summarization. ETRI J. 43, 2 (Dec. 2020), 288–298. DOI:DOI:
[208]
Dheeraj Rajagopal, Siamak Shakeri, Cícero Nogueira dos Santos, Eduard H. Hovy, and Chung-Ching Chang. 2022. Counterfactual data augmentation improves factuality of abstractive summarization. CoRR abs/2205.12416 (2022).
[209]
Gonçalo Raposo, Afonso Raposo, and Ana Sofia Carmo. 2022. Document-level abstractive summarization. CoRR abs/2212.03013 (2022).
[210]
Mathieu Ravaut, Shafiq R. Joty, and Nancy F. Chen. 2022. SummaReranker: A multi-task mixture-of-experts re-ranking framework for abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 4504–4524. DOI:DOI:
[211]
Rahul Rawat, Pranay Rawat, Vivek Elahi, and Amaan Elahi. 2021. Abstractive summarization on dynamically changing text. In Proceedings of the 5th International Conference on Computing Methodologies and Communication (ICCMC’21). IEEE. DOI:DOI:
[212]
Tohida Rehman, Suchandan Das, Debarshi Kumar Sanyal, and Samiran Chattopadhyay. 2023. Abstractive text summarization using attentive GRU based encoder-decoder. CoRR abs/2302.13117 (2023).
[213]
Banafsheh Rekabdar, Christos Mousas, and Bidyut Gupta. 2019. Generative adversarial network with policy gradient for text summarization. In Proceedings of the 13th IEEE International Conference on Semantic Computing (ICSC’19). IEEE, 204–207. DOI:DOI:
[214]
Weidong Ren, Hao Zhou, Gongshen Liu, and Fei Huan. 2021. ASM: Augmentation-based semantic mechanism on abstractive summarization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’21). IEEE, 1–8. DOI:DOI:
[215]
Gaetano Rossiello, Pierpaolo Basile, Giovanni Semeraro, Marco Di Ciano, and Gaetano Grasso. 2016. Improving neural abstractive text summarization with prior knowledge (position paper). In Proceedings of the AI*IA Workshop on Deep Understanding and Reasoning: A Challenge for Next-generation Intelligent Agents, co-located with the 15th International Conference of the Italian Association for Artificial Intelligence (AIxIA’16)(CEUR Workshop Proceedings, Vol. 1802), Federico Chesani, Paola Mello, and Michela Milano (Eds.). CEUR-WS.org, 13–18. Retrieved from http://ceur-ws.org/Vol-1802/paper2.pdf
[216]
Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15), Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). The Association for Computational Linguistics, 379–389. DOI:DOI:
[217]
Horacio Saggion and Guy Lapalme. 2002. Generating indicative-informative summaries with SumUM. Comput. Ling. 28, 4 (2002), 497–526. DOI:DOI:
[218]
Deepak Sahoo, Ashutosh Bhoi, and Rakesh Chandra Balabantaray. 2018. Hybrid approach to abstractive summarization. Procedia Comput. Sci. 132 (Jan. 2018), 1228–1237. DOI:DOI:
[219]
Ankit Sahu and Sriram G. Sanjeevi. 2021. Better fine-tuning with extracted important sentences for abstractive summarization. In Proceedings of the International Conference on Communication, Control and Information Sciences (ICCISc’21). IEEE. DOI:DOI:
[220]
Evan Sandhaus. 2008. The New York Times annotated corpus. Ling. Data Consort., Philad. 6, 12 (2008), e26752.
[221]
Ritesh Sarkhel, Moniba Keymanesh, Arnab Nandi, and Srinivasan Parthasarathy. 2020. Interpretable multi-headed attention for abstractive summarization at controllable lengths. In Proceedings of the 28th International Conference on Computational Linguistics (COLING’20), Donia Scott, Núria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, 6871–6882. DOI:DOI:
[222]
Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. 2020. Discriminative adversarial search for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning (ICML’20)(Proceedings of Machine Learning Research, Vol. 119). PMLR, 8555–8564. Retrieved from http://proceedings.mlr.press/v119/scialom20a.html
[223]
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1073–1083. DOI:DOI:
[224]
Elaheh ShafieiBavani, Mohammad Ebrahimi, Raymond K. Wong, and Fang Chen. 2018. A graph-theoretic summary evaluation for Rouge. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18), Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 762–767. DOI:DOI:
[225]
Eva Sharma, Luyang Huang, Zhe Hu, and Lu Wang. 2019. An entity-driven framework for abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 3278–3289. DOI:DOI:
[226]
Eva Sharma, Chen Li, and Lu Wang. 2019. BIGPATENT: A large-scale dataset for abstractive and coherent summarization. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL’19), Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 2204–2213. DOI:DOI:
[227]
Yunsheng Shi, Jun Meng, Jian Wang, Hongfei Lin, and Yumeng Li. 2018. A normalized encoder-decoder model for abstractive summarization using focal loss. In Proceedings of the 7th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC’18)(Lecture Notes in Computer Science, Vol. 11109), Min Zhang, Vincent Ng, Dongyan Zhao, Sujian Li, and Hongying Zan (Eds.). Springer, 383–392. DOI:DOI:
[228]
Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. 2022. Using DeepSpeed and megatron to train megatron-turing NLG 530B, a large-scale generative language model. CoRR abs/2201.11990 (2022).
[229]
Kaiqiang Song, Logan Lebanoff, Qipeng Guo, Xipeng Qiu, Xiangyang Xue, Chen Li, Dong Yu, and Fei Liu. 2020. Joint parsing and generation for abstractive summarization. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI’20), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’20). AAAI Press, 8894–8901. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/6419
[230]
Kaiqiang Song, Bingqing Wang, Zhe Feng, Ren Liu, and Fei Liu. 2020. Controlling the amount of verbatim copying in abstractive summarization. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI’20), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’20). AAAI Press, 8902–8909. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/6420
[231]
Kaiqiang Song, Lin Zhao, and Fei Liu. 2018. Structure-infused copy mechanisms for abstractive summarization. In Proceedings of the 27th International Conference on Computational Linguistics (COLING’18), Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, 1717–1729. Retrieved from https://aclanthology.org/C18-1146/
[232]
Shengli Song, Haitao Huang, and Tongxiao Ruan. 2019. Abstractive text summarization using LSTM-CNN based deep learning. Multim. Tools Appl. 78, 1 (2019), 857–875. DOI:DOI:
[233]
Sajad Sotudeh, Hanieh Deilamsalehy, Franck Dernoncourt, and Nazli Goharian. 2023. Curriculum-guided abstractive summarization. CoRR abs/2302.01342 (2023).
[234]
Arvind Krishna Sridhar and Erik Visser. 2022. Improved beam search for hallucination mitigation in abstractive summarization. CoRR abs/2212.02712 (2022).
[235]
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. 2020. Learning to summarize with human feedback. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’20), Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). Retrieved from https://proceedings.neurips.cc/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html
[236]
Jun Suzuki and Masaaki Nagata. 2017. Cutting-off redundant repeating generations for neural abstractive summarization. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17), Mirella Lapata, Phil Blunsom, and Alexander Koller (Eds.). Association for Computational Linguistics, 291–297. DOI:DOI:
[237]
Ayesha Ayub Syed, Ford Lumban Gaol, and Tokuro Matsuo. 2021. A survey of the state-of-the-art models in neural abstractive text summarization. IEEE Access 9 (2021), 13248–13265.
[238]
Gábor Szücs and Dorottya Huszti. 2019. Seq2seq deep learning method for summary generation by LSTM with two-way encoder and beam search decoder. In Proceedings of the 17th IEEE International Symposium on Intelligent Systems and Informatics (SISY’19). IEEE, 221–226. DOI:DOI:
[239]
Jiwei Tan, Xiaojun Wan, and Jianguo Xiao. 2017. Abstractive document summarization with a graph-based attentional neural model. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1171–1181. DOI:DOI:
[240]
Minakshi Tomer and Manoj Kumar. 2020. Improving text summarization using ensembled approach based on fuzzy with LSTM. Arab. J. Sci. Eng. 45, 12 (Aug. 2020), 10743–10754. DOI:DOI:
[241]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models. CoRR abs/2302.13971 (2023).
[242]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. CoRR abs/2307.09288 (2023).
[243]
Liam van der Poel, Ryan Cotterell, and Clara Meister. 2022. Mutual information alleviates hallucinations in abstractive summarization. CoRR abs/2210.13210 (2022).
[244]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Annual Conference on Neural Information Processing Systems, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998–6008. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
[245]
Tatiana Vodolazova and Elena Lloret. 2019. The impact of rule-based text generation on the quality of abstractive summaries. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP’19), Ruslan Mitkov and Galia Angelova (Eds.). INCOMA Ltd., 1275–1284. DOI:DOI:
[246]
David Wan and Mohit Bansal. 2022. FactPEGASUS: Factuality-aware pre-training and fine-tuning for abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’22), Marine Carpuat, Marie-Catherine de Marneffe, and Iván Vladimir Meza Ruíz (Eds.). Association for Computational Linguistics, 1010–1028. DOI:DOI:
[247]
David Wan, Mengwen Liu, Kathleen R. McKeown, Markus Dreyer, and Mohit Bansal. 2023. Faithfulness-aware decoding strategies for abstractive summarization. CoRR abs/2303.03278 (2023).
[248]
Xia Wan and Shenggen Ju. 2021. SA-HAVE: A self-attention based hierarchical VAEs network for abstractive summarization. J. Phys.: Conf. Series 2078, 1 (Nov. 2021), 012073. DOI:DOI:
[249]
Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5008–5020. DOI:DOI:
[250]
Fei Wang, Kaiqiang Song, Hongming Zhang, Lifeng Jin, Sangwoo Cho, Wenlin Yao, Xiaoyang Wang, Muhao Chen, and Dong Yu. 2022. Salience allocation as guidance for abstractive summarization. CoRR abs/2210.12330 (2022).
[251]
Guan Wang, Weihua Li, Edmund Lai, and Jianhua Jiang. 2022. KATSum: Knowledge-aware abstractive text summarization. CoRR abs/2212.03371 (2022).
[252]
Haonan Wang, Yang Gao, Yu Bai, Mirella Lapata, and Heyan Huang. 2021. Exploring explainable selection to control abstractive summarization. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21), the 33rd Conference on Innovative Applications of Artificial Intelligence (IAAI’21), the 11th Symposium on Educational Advances in Artificial Intelligence (EAAI’21). AAAI Press, 13933–13941. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17641
[253]
Jiaan Wang, Fandong Meng, Duo Zheng, Yunlong Liang, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2022. A survey on cross-lingual summarization. Trans. Assoc. Comput. Ling. 10 (2022), 1304–1323. Retrieved from https://transacl.org/ojs/index.php/tacl/article/view/4017
[254]
Kai Wang, Xiaojun Quan, and Rui Wang. 2019. BiSET: Bi-directional selective encoding with template for abstractive summarization. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL’19), Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 2153–2162. DOI:DOI:
[255]
Lihan Wang, Min Yang, Chengming Li, Ying Shen, and Ruifeng Xu. 2021. Abstractive text summarization with hierarchical multi-scale abstraction modeling and dynamic memory. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’21), Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). ACM, 2086–2090. DOI:DOI:
[256]
Liang Wang, Wei Zhao, Ruoyu Jia, Sujian Li, and Jingming Liu. 2019. Denoising based sequence-to-sequence pre-training for text generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 4001–4013. DOI:DOI:
[257]
Wenbo Wang, Yang Gao, Heyan Huang, and Yuxiang Zhou. 2019. Concept pointer network for abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 3074–3083. DOI:DOI:
[258]
Zhengjue Wang, Zhibin Duan, Hao Zhang, Chaojie Wang, Long Tian, Bo Chen, and Mingyuan Zhou. 2020. Friendly topic assistant for transformer based abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 485–497. DOI:DOI:
[259]
Adhika Pramita Widyassari, Supriadi Rustad, Guruh Fajar Shidik, Edi Noersasongko, Abdul Syukur, Affandy Affandy, and De Rosal Ignatius Moses Setiadi. 2020. Review of automatic text summarization techniques & methods. J. King Saud Univ.-Comput. Inf. Sci. (2020).
[260]
Chien-Sheng Wu, Linqing Liu, Wenhao Liu, Pontus Stenetorp, and Caiming Xiong. 2021. Controllable abstractive dialogue summarization with sketch supervision. In Findings of the Association for Computational Linguistics (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 5108–5122. DOI:DOI:
[261]
Jhen-Yi Wu, Ying-Jia Lin, and Hung-Yu Kao. 2022. Unsupervised single document abstractive summarization using semantic units. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (AACL/IJCNLP’22), Yulan He, Heng Ji, Yang Liu, Sujian Li, Chia-Hui Chang, Soujanya Poria, Chenghua Lin, Wray L. Buntine, Maria Liakata, Hanqi Yan, Zonghan Yan, Sebastian Ruder, Xiaojun Wan, Miguel Arana-Catania, Zhongyu Wei, Hen-Hsen Huang, Jheng-Long Wu, Min-Yuh Day, Pengfei Liu, and Ruifeng Xu (Eds.). Association for Computational Linguistics, 954–966. Retrieved from https://aclanthology.org/2022.aacl-main.69
[262]
Wenhao Wu, Wei Li, Jiachen Liu, Xinyan Xiao, Ziqiang Cao, Sujian Li, and Hua Wu. 2022. FRSUM: Towards faithful abstractive summarization via enhancing factual robustness. CoRR abs/2211.00294 (2022).
[263]
Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, and Haifeng Wang. 2021. BASS: Boosting abstractive summarization with unified semantic graph. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP’21), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 6052–6067. DOI:DOI:
[264]
Yixuan Wu and Kei Wakabayashi. 2020. Effect of semantic content generalization on pointer generator network in text summarization. In Proceedings of the 22nd International Conference on Information Integration and Web-based Applications & Services (iiWAS’20), Maria Indrawan-Santiago, Eric Pardede, Ivan Luiz Salvadori, Matthias Steinbauer, Ismail Khalil, and Gabriele Kotsis (Eds.). ACM, 72–76. DOI:DOI:
[265]
Xiujuan Xiang, Guangluan Xu, Xingyu Fu, Yang Wei, Li Jin, and Lei Wang. 2018. Skeleton to abstraction: An attentive information extraction schema for enhancing the saliency of text summarization. Information 9, 9 (2018), 217. DOI:DOI:
[266]
Dongling Xiao, Han Zhang, Yu-Kun Li, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE-GEN: An enhanced multi-flow pre-training and fine-tuning framework for natural language generation. CoRR abs/2001.11314 (2020).
[267]
Liqiang Xiao, Hao He, and Yaohui Jin. 2022. FusionSum: Abstractive summarization with sentence fusion and cooperative reinforcement learning. Knowl.-based Syst. 243 (2022), 108483. DOI:DOI:
[268]
Wen Xiao and Giuseppe Carenini. 2022. Entity-based SpanCopy for abstractive summarization to improve the factual consistency. CoRR abs/2209.03479 (2022).
[269]
Hao Xu, Yanan Cao, Ruipeng Jia, Yanbing Liu, and Jianlong Tan. 2018. Sequence generative adversarial network for long text summarization. In Proceedings of the IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI’18), Lefteri H. Tsoukalas, Éric Grégoire, and Miltiadis Alamaniotis (Eds.). IEEE, 242–248. DOI:DOI:
[270]
Haiyang Xu, Yun Wang, Kun Han, Baochang Ma, Junwen Chen, and Xiangang Li. 2020. Selective attention encoders by syntactic graph convolutional networks for document summarization. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’20). IEEE, 8219–8223. DOI:DOI:
[271]
Song Xu, Haoran Li, Peng Yuan, Youzheng Wu, Xiaodong He, and Bowen Zhou. 2020. Self-attention guided copy mechanism for abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 1355–1362. DOI:DOI:
[272]
Weiran Xu, Chenliang Li, Minghao Lee, and Chi Zhang. 2020. Multi-task learning for abstractive text summarization with key information guide network. EURASIP J. Adv. Signal Process. 2020, 1 (2020), 16. DOI:DOI:
[273]
Yumo Xu and Mirella Lapata. 2022. Document summarization with latent queries. Trans. Assoc. Comput. Ling. 10 (2022), 623–638. DOI:DOI:
[274]
Ying Xu, Jey Han Lau, Timothy Baldwin, and Trevor Cohn. 2017. Decoupling encoder and decoder networks for abstractive document summarization. In Proceedings of the Workshop on Summarization and Summary Evaluation Across Source Types and Genres (MultiLing@EACL’17), George Giannakopoulos, Elena Lloret, John M. Conroy, Josef Steinberger, Marina Litvak, Peter A. Rankel, and Benoît Favre (Eds.). Association for Computational Linguistics, 7–11. DOI:DOI:
[275]
Min Yang, Chengming Li, Ying Shen, Qingyao Wu, Zhou Zhao, and Xiaojun Chen. 2021. Hierarchical human-like deep neural networks for abstractive text summarization. IEEE Trans. Neural Netw. Learn. Syst. 32, 6 (2021), 2744–2757. DOI:DOI:
[276]
Min Yang, Qiang Qu, Wenting Tu, Ying Shen, Zhou Zhao, and Xiaojun Chen. 2019. Exploring human-like reading strategy for abstractive text summarization. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), the 31st Innovative Applications of Artificial Intelligence Conference (IAAI’19), the 9th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’19). AAAI Press, 7362–7369. DOI:DOI:
[277]
Kaichun Yao, Libo Zhang, Dawei Du, Tiejian Luo, Lili Tao, and Yanjun Wu. 2020. Dual encoding for abstractive text summarization. IEEE Trans. Cybern. 50, 3 (2020), 985–996. DOI:DOI:
[278]
Jaya Kumar Yogan, Ong Sing Goh, Basiron Halizah, Hea Choon Ngo, and C. Puspalata. 2016. A review on automatic text summarization approaches. J. Comput. Sci. 12, 4 (2016), 178–190.
[279]
Yongjian You, Weijia Jia, Tianyi Liu, and Wenmian Yang. 2019. Improving abstractive document summarization with salient information modeling. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL’19), Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 2132–2141. DOI:DOI:
[280]
QiAo Yuan, Pin Ni, Junru Liu, Xiangzhi Tong, Hanzhe Lu, Gangmin Li, and Steven Guan. 2021. An encoder-decoder architecture with graph convolutional networks for abstractive summarization. In Proceedings of the IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI’21). IEEE. DOI:DOI:
[281]
Farooq Zaman, Matthew Shardlow, Saeed-Ul Hassan, Naif Radi Aljohani, and Raheel Nawaz. 2020. HTSS: A novel hybrid text summarisation and simplification architecture. Inf. Process. Manag. 57, 6 (2020), 102351. DOI:DOI:
[282]
Haopeng Zhang, Semih Yavuz, Wojciech Kryscinski, Kazuma Hashimoto, and Yingbo Zhou. 2022. Improving the faithfulness of abstractive summarization via entity coverage control. In Findings of the Association for Computational Linguistics (NAACL’22), Marine Carpuat, Marie-Catherine de Marneffe, and Iván Vladimir Meza Ruíz (Eds.). Association for Computational Linguistics, 528–535. DOI:DOI:
[283]
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning (ICML’20)(Proceedings of Machine Learning Research, Vol. 119). PMLR, 11328–11339. Retrieved from http://proceedings.mlr.press/v119/zhang20ae.html
[284]
Mengli Zhang, Gang Zhou, Wanting Yu, and Wenfen Liu. 2021. FAR-ASS: Fact-aware reinforced abstractive sentence summarization. Inf. Process. Manag. 58, 3 (2021), 102478. DOI:DOI:
[285]
Mengli Zhang, Gang Zhou, Wanting Yu, Wenfen Liu, Ningbo Huang, and Ze Yu. 2022. FCSF-TABS: Two-stage abstractive summarization with fact-aware reinforced content selection and fusion. Neural Comput. Applic. 34, 1 (Jan. 2022). DOI:DOI:
[286]
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In Proceedings of the 8th International Conference on Learning Representations (ICLR’20). OpenReview.net. Retrieved from https://openreview.net/forum?id=SkeHuCVFDr
[287]
Xingxing Zhang, Yiran Liu, Xun Wang, Pengcheng He, Yang Yu, Si-Qing Chen, Wayne Xiong, and Furu Wei. 2022. Momentum calibration for text generation. CoRR abs/2212.04257 (2022).
[288]
Xuewen Zhang, Kui Meng, and Gongshen Liu. 2019. Hie-Transformer: A hierarchical hybrid transformer for abstractive article summarization. In Proceedings of the 26th International Conference on Neural Information Processing (ICONIP’19)(Lecture Notes in Computer Science, Vol. 11955), Tom Gedeon, Kok Wai Wong, and Minho Lee (Eds.). Springer, 248–258. DOI:DOI:
[289]
Shuai Zhao and Fucheng You. 2020. A topical keywords fusion based on transformer for text summarization. In Proceedings of the 13th International Conference on Intelligent Computation Technology and Automation (ICICTA’20). IEEE. DOI:DOI:
[290]
Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019. MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 563–578. DOI:DOI:
[291]
Wenjun Zhao, Meina Song, and E. Haihong. 2018. Summarization with highway condition radom pointer-generator network. In Proceedings of the International Conference on Algorithms, Computing and Artificial Intelligence (ACAI’18). ACM, 25:1–25:5. DOI:DOI:
[292]
Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J. Liu. 2022. Calibrating sequence likelihood improves conditional language generation. CoRR abs/2210.00045 (2022).
[293]
Changmeng Zheng, Yi Cai, Guanjie Zhang, and Qing Li. 2020. Controllable abstractive sentence summarization with guiding entities. In Proceedings of the 28th International Conference on Computational Linguistics (COLING’20), Donia Scott, Núria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, 5668–5678. DOI:DOI:
[294]
Qingyu Zhou, Nan Yang, Furu Wei, and Ming Zhou. 2017. Selective encoding for abstractive sentence summarization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1095–1104. DOI:DOI:
[295]
Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. 2021. Enhancing factual consistency of abstractive summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21), Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 718–733. DOI:DOI:
[296]
Junnan Zhu, Qian Wang, Yining Wang, Yu Zhou, Jiajun Zhang, Shaonan Wang, and Chengqing Zong. 2019. NCLS: Neural cross-lingual summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 3052–3062. DOI:DOI:
[297]
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15).
[298]
Yunqi Zhu, Wensheng Zhang, and Mingjin Zhu. 2022. Differentiable N-gram objective on abstractive summarization. CoRR abs/2202.04003 (2022).
[299]
Haojie Zhuang and Weibin Zhang. 2019. Generating semantically similar and human-readable summaries with generative adversarial networks. IEEE Access 7 (2019), 169426–169433. DOI:DOI:
[300]
Ekaterina Zolotareva, Tsegaye Misikir Tashu, and Tomás Horváth. 2020. Abstractive text summarization using transfer learning. In Proceedings of the 20th Conference Information Technologies: Applications and Theory (ITAT’20)(CEUR Workshop Proceedings, Vol. 2718), Martin Holena, Tomás Horváth, Alica Kelemenová, Frantisek Mráz, Dana Pardubská, Martin Plátek, and Petr Sosík (Eds.). CEUR-WS.org, 75–80. Retrieved from http://ceur-ws.org/Vol-2718/paper28.pdf
[301]
Yanyan Zou, Xingxing Zhang, Wei Lu, Furu Wei, and Ming Zhou. 2020. Pre-training for abstractive document summarization by reinstating source text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’20), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 3646–3660.
Index Terms
- Single-Document Abstractive Text Summarization: A Systematic Literature Review
Recommendations
Graph-based abstractive biomedical text summarization
Graphical abstractDisplay Omitted
Highlights- A graph generation and frequent itemset mining approach have been used for the generation of extractive summaries.
- The T5 model has been adopted to generate abstractive summaries in the biomedical domain.
- The ROUGE metric has been ...
AbstractSummarization is the process of compressing a text to obtain its important informative parts. In recent years, various methods have been presented to extract important parts of textual documents to present them in a summarized form. The first ...
Sentiment diversification for short review summarization
WI '17: Proceedings of the International Conference on Web IntelligenceWith the abundance of reviews published on the Web about a given product, consumers are looking for ways to view major opinions that can be presented in a quick and succinct way. Reviews contain many different opinions, making the ability to show a ...
Assessing Abstractive and Extractive Methods for Automatic News Summarization
DocEng '24: Proceedings of the ACM Symposium on Document Engineering 2024Automatic Text Summarization (ATS) is a research area that originated in the late 1950s and has gained increasing importance with the surge of text data available today. ATS approaches are generally classified into extractive and abstractive methods. ...
Comments
Information & Contributors
Information
Published In
March 2025
984 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 November 2024
Online AM: 18 October 2024
Accepted: 16 September 2024
Revised: 03 September 2024
Received: 12 July 2022
Published in CSUR Volume 57, Issue 3
Check for updates
Author Tags
Qualifiers
- Survey
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 1,753Total Downloads
- Downloads (Last 12 months)1,753
- Downloads (Last 6 weeks)440
Reflects downloads up to 27 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in