Summarization Papers
Organized by Xiachong Feng.
Contributor
Yichong Huang, Haozheng Yang, Jiaan Wang
Summarization Learning Route
Summarization Learning Route (with link)
Trending
Presentations && Notes
- Dialogue Summarization (2022.1)
- Cross-lingual Summarization
- 如何把DialoGPT用到对话摘要任务？@ ACL 2021
- 对话摘要最新进展简述
- Dialogue Summarization (2021.5)
- 融入常识知识的生成式对话摘要
- 会议摘要有难度？快来引入对话篇章结构信息
- 文本摘要论文列表(Chinese)
- 事实感知的生成式文本摘要(Chinese)
- 多模态摘要简述(Chinese)
- 文本摘要简述
- Multi-modal Summarization
- ACL20 Summarization
- 文本摘要简述 (Chinese)
- ACL19 Summarization
- Brief intro to summarization (Chinese)
- EMNLP19 Summarization (Chinese)
- ACL19-A Simple Theoretical Model of Importance for Summarization
- ACL19-Multimodal Abstractive Summarization for How2 Videos
Big Model Era
- TempoSum: Evaluating the Temporal Generalization of Abstractive Summarization Chi Seng Cheang, Hou Pong Chan, Derek F. Wong, Xuebo Liu, Zhaocong Li, Yanming Sun, Shudong Liu, Lidia S. Chao `` [pdf] [data]
[Abs] Recent pre-trained language models (PLMs) achieve promising results in existing abstractive summarization datasets. However, existing summarization benchmarks overlap in time with the standard pre-training corpora and finetuning datasets. Hence, the strong performance of PLMs may rely on the parametric knowledge that is memorized during pre-training and fine-tuning. Moreover, the knowledge memorized by PLMs may quickly become outdated, which affects the generalization performance of PLMs on future data. In this work, we propose TempoSum, a novel benchmark that contains data samples from 2010 to 2022, to understand the temporal generalization ability of abstractive summarization models. Through extensive human evaluation, we show that parametric knowledge stored in summarization models significantly affects the faithfulness of the generated summaries on future data. Moreover, existing faithfulness enhancement methods cannot reliably improve the faithfulness of summarization models on future data. Finally, we discuss several recommendations to the research community on how to evaluate and improve the temporal generalization capability of text summarization models.
- RadAdapt: Radiology Report Summarization via Lightweight Domain Adaptation of Large Language Models Dave Van Veen, Cara Van Uden, Maayane Attias, Anuj Pareek, Christian Bluethgen, Malgorzata Polacin, Wah Chiu, Jean-Benoit Delbrouck, Juan Manuel Zambrano Chaves, Curtis P. Langlotz, Akshay S. Chaudhari, John Pauly [pdf]
[Abs] We systematically investigate lightweight strategies to adapt large language models (LLMs) for the task of radiology report summarization (RRS). Specifically, we focus on domain adaptation via pretraining (on natural language, biomedical text, and clinical text) and via prompting (zero-shot, in-context learning) or parameter-efficient fine-tuning (prefix tuning, LoRA). Our results on the MIMIC-III dataset consistently demonstrate best performance by maximally adapting to the task via pretraining on clinical text and parameter-efficient fine-tuning on RRS examples. Importantly, this method fine-tunes a mere 0.32% of parameters throughout the model, in contrast to end-to-end fine-tuning (100% of parameters). Additionally, we study the effect of in-context examples and out-of-distribution (OOD) training before concluding with a radiologist reader study and qualitative analysis. Our findings highlight the importance of domain adaptation in RRS and provide valuable insights toward developing effective natural language processing solutions for clinical tasks.
- ImpressionGPT: An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT Chong Ma, Zihao Wu, Jiaqi Wang, Shaochen Xu, Yaonai Wei, Zhengliang Liu, Lei Guo, Xiaoyan Cai, Shu Zhang, Tuo Zhang, Dajiang Zhu, Dinggang Shen, Tianming Liu, Xiang Li [pdf]
[Abs] The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians, and it is typically written by radiologists based on the 'Findings' section. However, writing numerous impressions can be laborious and error-prone for radiologists. Although recent studies have achieved promising results in automatic impression generation using large-scale medical text data for pre-training and fine-tuning pre-trained language models, such models often require substantial amounts of medical text data and have poor generalization performance. While large language models (LLMs) like ChatGPT have shown strong generalization capabilities and performance, their performance in specific domains, such as radiology, remains under-investigated and potentially limited. To address this limitation, we propose ImpressionGPT, which leverages the in-context learning capability of LLMs by constructing dynamic contexts using domain-specific, individualized data. This dynamic prompt approach enables the model to learn contextual knowledge from semantically similar examples from existing data. Additionally, we design an iterative optimization algorithm that performs automatic evaluation on the generated impression results and composes the corresponding instruction prompts to further optimize the model. The proposed ImpressionGPT model achieves state-of-the-art performance on both MIMIC-CXR and OpenI datasets without requiring additional training data or fine-tuning the LLMs. This work presents a paradigm for localizing LLMs that can be applied in a wide range of similar application scenarios, bridging the gap between general-purpose LLMs and the specific language processing needs of various domains.
- Extractive Summarization via ChatGPT for Faithful Summary Generation Haopeng Zhang, Xiao Liu, Jiawei Zhang [pdf]
[Abs] Extractive summarization is a crucial task in natural language processing that aims to condense long documents into shorter versions by directly extracting sentences. The recent introduction of ChatGPT has attracted significant interest in the NLP community due to its remarkable performance on a wide range of downstream tasks. However, concerns regarding factuality and faithfulness have hindered its practical applications for summarization systems. This paper first presents a thorough evaluation of ChatGPT's performance on extractive summarization and compares it with traditional fine-tuning methods on various benchmark datasets. Our experimental analysis reveals that ChatGPT's extractive summarization performance is still inferior to existing supervised systems in terms of ROUGE scores. In addition, we explore the effectiveness of in-context learning and chain-of-thought reasoning for enhancing its performance. Furthermore, we find that applying an extract-then-generate pipeline with ChatGPT yields significant performance improvements over abstractive baselines in terms of summary faithfulness. These observations highlight potential directions for enhancing ChatGPT's capabilities for faithful text summarization tasks using two-stage approaches.
- Human-like Summarization Evaluation with ChatGPT Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, Xiaojun Wan [pdf]
[Abs] Evaluating text summarization is a challenging problem, and existing evaluation metrics are far from satisfactory. In this study, we explored ChatGPT's ability to perform human-like summarization evaluation using four human evaluation methods on five datasets. We found that ChatGPT was able to complete annotations relatively smoothly using Likert scale scoring, pairwise comparison, Pyramid, and binary factuality evaluation. Additionally, it outperformed commonly used automatic evaluation metrics on some datasets. Furthermore, we discussed the impact of different prompts, compared its performance with that of human evaluation, and analyzed the generated explanations and invalid responses.
- Cross-Lingual Summarization via ChatGPT Jiaan Wang, Yunlong Liang, Fandong Meng, Zhixu Li, Jianfeng Qu, Jie Zhou [pdf]
[Abs] Given a document in a source language, cross-lingual summarization (CLS) aims to generate a summary in a different target language. Recently, the emergence of ChatGPT has attracted wide attention from the computational linguistics community. However, it is not yet known the performance of ChatGPT on CLS. In this report, we empirically use various prompts to guide ChatGPT to perform zero-shot CLS from different paradigms (i.e., end-to-end and pipeline), and provide a preliminary evaluation on its generated summaries.We find that ChatGPT originally prefers to produce lengthy summaries with more detailed information. But with the help of an interactive prompt, ChatGPT can balance between informativeness and conciseness, and significantly improve its CLS performance. Experimental results on three widely-used CLS datasets show that ChatGPT outperforms the advanced GPT 3.5 model (i.e., text-davinci-003). In addition, we provide qualitative case studies to show the superiority of ChatGPT on CLS.
- Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen, Wei Cheng [pdf]
[Abs] Text summarization has been a crucial problem in natural language processing (NLP) for several decades. It aims to condense lengthy documents into shorter versions while retaining the most critical information. Various methods have been proposed for text summarization, including extractive and abstractive summarization. The emergence of large language models (LLMs) like GPT3 and ChatGPT has recently created significant interest in using these models for text summarization tasks. Recent studies \cite{goyal2022news, zhang2023benchmarking} have shown that LLMs-generated news summaries are already on par with humans. However, the performance of LLMs for more practical applications like aspect or query-based summaries is underexplored. To fill this gap, we conducted an evaluation of ChatGPT's performance on four widely used benchmark datasets, encompassing diverse summaries from Reddit posts, news articles, dialogue meetings, and stories. Our experiments reveal that ChatGPT's performance is comparable to traditional fine-tuning methods in terms of Rouge scores. Moreover, we highlight some unique differences between ChatGPT-generated summaries and human references, providing valuable insights into the superpower of ChatGPT for diverse text summarization tasks. Our findings call for new directions in this area, and we plan to conduct further research to systematically examine the characteristics of ChatGPT-generated summaries through extensive human evaluation.
- News Summarization and Evaluation in the Era of GPT-3 Tanya Goyal, Junyi Jessy Li, Greg Durrett [pdf] [code]
[Abs] The recent success of zero- and few-shot prompting with models like GPT-3 has led to a paradigm shift in NLP research. In this paper, we study its impact on text summarization, focusing on the classic benchmark domain of news summarization. First, we investigate how zero-shot GPT-3 compares against fine-tuned models trained on large summarization datasets. We show that not only do humans overwhelmingly prefer GPT-3 summaries, but these also do not suffer from common dataset-specific issues such as poor factuality. Next, we study what this means for evaluation, particularly the role of gold standard test sets. Our experiments show that both reference-based and reference-free automatic metrics, e.g. recently proposed QA- or entailment-based factuality approaches, cannot reliably evaluate zero-shot summaries. Finally, we discuss future research challenges beyond generic summarization, specifically, keyword- and aspect-based summarization, showing how dominant fine-tuning approaches compare to zero-shot prompting. To support further research, we release: (a) a corpus of 10K generated summaries from fine-tuned and zero-shot models across 4 standard summarization benchmarks, (b) 1K human preference judgments and rationales comparing different systems for generic- and keyword-based summarization.
- Benchmarking Large Language Models for News Summarization Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, Tatsunori B. Hashimoto [pdf]
[Abs] Large language models (LLMs) have shown promise for automatic summarization but the reasons behind their successes are poorly understood. By conducting a human evaluation on ten LLMs across different pretraining methods, prompts, and model scales, we make two important observations. First, we find instruction tuning, and not model size, is the key to the LLM's zero-shot summarization capability. Second, existing studies have been limited by low-quality references, leading to underestimates of human performance and lower few-shot and finetuning performance. To better evaluate LLMs, we perform human evaluation over high-quality summaries we collect from freelance writers. Despite major stylistic differences such as the amount of paraphrasing, we find that LMM summaries are judged to be on par with human written summaries.
- Is ChatGPT a General-Purpose Natural Language Processing Task Solver? Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, Diyi Yang [pdf]
[Abs] Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i.e., without adaptation on downstream data. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community due to the fact that it can generate high-quality responses to human input and self-correct previous mistakes based on subsequent conversations. However, it is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot. In this work, we empirically analyze the zero-shot learning ability of ChatGPT by evaluating it on 20 popular NLP datasets covering 7 representative task categories. With extensive empirical studies, we demonstrate both the effectiveness and limitations of the current version of ChatGPT. We find that ChatGPT performs well on many tasks favoring reasoning capabilities (e.g., arithmetic reasoning) while it still faces challenges when solving specific tasks such as sequence tagging. We additionally provide in-depth analysis through qualitative case studies.
- A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung `` [pdf]
[Abs] This paper proposes a framework for quantitatively evaluating interactive LLMs such as ChatGPT using publicly available data sets. We carry out an extensive technical evaluation of ChatGPT using 21 data sets covering 8 different common NLP application tasks. We evaluate the multitask, multilingual and multi-modal aspects of ChatGPT based on these data sets and a newly designed multimodal dataset. We find that ChatGPT outperforms LLMs with zero-shot learning on most tasks and even outperforms fine-tuned models on some tasks. We find that it is better at understanding non-Latin script languages than generating them. It is able to generate multimodal content from textual prompts, via an intermediate code generation step. Moreover, we find that ChatGPT is 64.33% accurate on average in 10 different reasoning categories under logical reasoning, non-textual reasoning, and commonsense reasoning, hence making it an unreliable reasoner. It is, for example, better at deductive than inductive reasoning. ChatGPT suffers from hallucination problems like other LLMs and it generates more extrinsic hallucinations from its parametric memory as it does not have access to an external knowledge base. Finally, the interactive feature of ChatGPT enables human collaboration with the underlying LLM to improve its performance, i.e, 8% ROUGE-1 on summarization and 2% ChrF++ on machine translation, in a multi-turn "prompt engineering" fashion.
Decomposed
- Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees Swarnadeep Saha, Shiyue Zhang, Peter Hase, Mohit Bansal
ICLR 2023[pdf] [code]
[Abs] Current abstractive summarization models either suffer from a lack of clear interpretability or provide incomplete rationales by only highlighting parts of the source document. To this end, we propose the Summarization Program (SP), an interpretable modular framework consisting of an (ordered) list of binary trees, each encoding the step-by-step generative process of an abstractive summary sentence from the source document. A Summarization Program contains one root node per summary sentence, and a distinct tree connects each summary sentence (root node) to the document sentences (leaf nodes) from which it is derived, with the connecting nodes containing intermediate generated sentences. Edges represent different modular operations involved in summarization such as sentence fusion, compression, and paraphrasing. We first propose an efficient best-first search method over neural modules, SP-Search that identifies SPs for human summaries by directly optimizing for ROUGE scores. Next, using these programs as automatic supervision, we propose seq2seq models that generate Summarization Programs, which are then executed to obtain final summaries. We demonstrate that SP-Search effectively represents the generative process behind human summaries using modules that are typically faithful to their intended behavior. We also conduct a simulation study to show that Summarization Programs improve the interpretability of summarization models by allowing humans to better simulate model reasoning. Summarization Programs constitute a promising step toward interpretable and modular abstractive summarization, a complex task previously addressed primarily through blackbox end-to-end neural systems. Supporting code available at this https URL
Benchmark
- Benchmarking Large Language Models for News Summarizatio Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, Tatsunori B. Hashimoto [pdf]
[Abs] Large language models (LLMs) have shown promise for automatic summarization but the reasons behind their successes are poorly understood. By conducting a human evaluation on ten LLMs across different pretraining methods, prompts, and model scales, we make two important observations. First, we find instruction tuning, and not model size, is the key to the LLM's zero-shot summarization capability. Second, existing studies have been limited by low-quality references, leading to underestimates of human performance and lower few-shot and finetuning performance. To better evaluate LLMs, we perform human evaluation over high-quality summaries we collect from freelance writers. Despite major stylistic differences such as the amount of paraphrasing, we find that LMM summaries are judged to be on par with human written summaries.
- MuLD: The Multitask Long Document Benchmark G Thomas Hudson, Noura Al Moubayed [pdf] [data]
- EXPLAINABOARD: An Explainable Leaderboard for NLP Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Graham Neubig [pdf] [ExplainaBoard]
- GLGE: A New General Language Generation Evaluation Benchmark Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan [pdf] [benchmark]
Survey
- A Survey on Biomedical Text Summarization with Pre-trained Language Model Qianqian Xie, Zheheng Luo, Benyou Wang, Sophia Ananiadou [pdf] [code]
[Abs] The exponential growth of biomedical texts such as biomedical literature and electronic health records (EHRs), provides a big challenge for clinicians and researchers to access clinical information efficiently. To address the problem, biomedical text summarization has been proposed to support clinical information retrieval and management, aiming at generating concise summaries that distill key information from single or multiple biomedical documents. In recent years, pre-trained language models (PLMs) have been the de facto standard of various natural language processing tasks in the general domain. Most recently, PLMs have been further investigated in the biomedical field and brought new insights into the biomedical text summarization task. In this paper, we systematically summarize recent advances that explore PLMs for biomedical text summarization, to help understand recent progress, challenges, and future directions. We categorize PLMs-based approaches according to how they utilize PLMs and what PLMs they use. We then review available datasets, recent approaches and evaluation metrics of the task. We finally discuss existing challenges and promising future directions. To facilitate the research community, we line up open resources including available datasets, recent approaches, codes, evaluation metrics, and the leaderboard in a public project: this https URL.
- A Survey on Medical Document Summarization Raghav Jain, Anubhav Jangra, Sriparna Saha, Adam Jatowt [pdf]
[Abs] The internet has had a dramatic effect on the healthcare industry, allowing documents to be saved, shared, and managed digitally. This has made it easier to locate and share important data, improving patient care and providing more opportunities for medical studies. As there is so much data accessible to doctors and patients alike, summarizing it has become increasingly necessary - this has been supported through the introduction of deep learning and transformer-based networks, which have boosted the sector significantly in recent years. This paper gives a comprehensive survey of the current techniques and trends in medical summarization
- Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directions Qi Jia, Siyu Ren, Yizhu Liu, Kenny Q. Zhu [pdf]
[Abs] Abstractive dialogue summarization is to generate a concise and fluent summary covering the salient information in a dialogue among two or more interlocutors. It has attracted great attention in recent years based on the massive emergence of social communication platforms and an urgent requirement for efficient dialogue information understanding and digestion. Different from news or articles in traditional document summarization, dialogues bring unique characteristics and additional challenges, including different language styles and formats, scattered information, flexible discourse structures and unclear topic boundaries. This survey provides a comprehensive investigation on existing work for abstractive dialogue summarization from scenarios, approaches to evaluations. It categorizes the task into two broad categories according to the type of input dialogues, i.e., open-domain and task-oriented, and presents a taxonomy of existing techniques in three directions, namely, injecting dialogue features, designing auxiliary training tasks and using additional data.A list of datasets under different scenarios and widely-accepted evaluation metrics are summarized for completeness. After that, the trends of scenarios and techniques are summarized, together with deep insights on correlations between extensively exploited features and different scenarios. Based on these analyses, we recommend future directions including more controlled and complicated scenarios, technical innovations and comparisons, publicly available datasets in special domains, etc.
- A Survey of Automatic Text Summarization Using Graph Neural Networks Marco Ferdinand Salchner, Adam Jatowt
COLING 2022[pdf]
[Abs] Although automatic text summarization (ATS) has been researched for several decades, the application of graph neural networks (GNNs) to this task started relatively recently. In this survey we provide an overview on the rapidly evolving approach of using GNNs for the task of automatic text summarization. In particular we provide detailed information on the functionality of GNNs in the context of ATS, and a comprehensive overview of models utilizing this approach.
- A Survey on Cross-Lingual Summarization Jiaan Wang, Fandong Meng, Duo Zheng, Yunlong Liang, Zhixu Li, Jianfeng Qu, Jie Zhou
TACL 2022[pdf]
[Abs] Cross-lingual summarization is the task of generating a summary in one language (e.g., English) for the given document(s) in a different language (e.g., Chinese). Under the globalization background, this task has attracted increasing attention of the computational linguistics community. Nevertheless, there still remains a lack of comprehensive review for this task. Therefore, we present the first systematic critical review on the datasets, approaches, and challenges in this field. Specifically, we carefully organize existing datasets and approaches according to different construction methods and solution paradigms, respectively. For each type of datasets or approaches, we thoroughly introduce and summarize previous efforts and further compare them with each other to provide deeper analyses. In the end, we also discuss promising directions and offer our thoughts to facilitate future research. This survey is for both beginners and experts in cross-lingual summarization, and we hope it will serve as a starting point as well as a source of new ideas for researchers and engineers interested in this area.
- An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics uan Yee Koh, Jiaxin Ju, Ming Liu, Shirui Pan
ACM Computing Surveys[pdf]
[Abs] Long documents such as academic articles and business reports have been the standard format to detail out important issues and complicated subjects that require extra attention. An automatic summarization system that can effectively condense long documents into short and concise texts to encapsulate the most important information would thus be significant in aiding the reader's comprehension. Recently, with the advent of neural architectures, significant research efforts have been made to advance automatic text summarization systems, and numerous studies on the challenges of extending these systems to the long document domain have emerged. In this survey, we provide a comprehensive overview of the research on long document summarization and a systematic evaluation across the three principal components of its research setting: benchmark datasets, summarization models, and evaluation metrics. For each component, we organize the literature within the context of long document summarization and conduct an empirical analysis to broaden the perspective on current research progress. The empirical analysis includes a study on the intrinsic characteristics of benchmark datasets, a multi-dimensional analysis of summarization models, and a review of the summarization evaluation metrics. Based on the overall findings, we conclude by proposing possible directions for future exploration in this rapidly growing field.
- Multi-document Summarization via Deep Learning Techniques: A Survey Congbo Ma, Wei Emma Zhang, Mingyu Guo, Hu Wang, QUAN Z. Sheng [pdf]
- Embedding Knowledge for Document Summarization: A Survey Yutong Qu, Wei Emma Zhang, Jian Yang, Lingfei Wu, Jia Wu, Xindong Wu [pdf]
- A Survey on Dialogue Summarization: Recent Advances and New Frontiers Xiachong Feng, Xiaocheng Feng, Bing Qin
IJCAI 2022, Survey Track[pdf] - Automatic Text Summarization Methods: A Comprehensive Review Divakar Yadav, Jalpa Desai, Arun Kumar Yadav [pdf]
- Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods Wei Li, Wenhao Wu, Moye Chen, Jiachen Liu, Xinyan Xiao, Hua Wu [pdf]
- Recent Advances in Neural Text Generation: A Task-Agnostic Survey Chen Tang, Frank Guerin, Yucheng Li, Chenghua Lin [pdf]
- Survey of Hallucination in Natural Language Generation Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, Pascale Fung [pdf]
- A Survey on Retrieval-Augmented Text Generation Huayang Li, Yixuan Su, Deng Cai, Yan Wang, Lemao Liu [pdf]
- A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, Dawei Song [pdf]
- A Survey of Pretrained Language Models Based Text Generation Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen [pdf]
- A Comprehensive Review on Summarizing Financial News Using Deep Learning Saurabh Kamal, Sahil Sharma [pdf]
- A Survey on Multi-modal Summarization Anubhav Jangra, Adam Jatowt, Sriparna Saha, Mohammad Hasanuzzaman [pdf]
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig [pdf]
- Pretrained Language Models for Text Generation: A Survey Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen
IJCAI21[pdf] - A Survey of Recent Abstract Summarization Techniques Diyah Puspitaningrum
ICICT21[pdf] - A Survey of the State-of-the-Art Models in Neural Abstractive Text Summarization AYESHA AYUB SYED, FORD LUMBAN GAOL, TOKURO MATSUO [pdf]
- Automatic summarization of scientific articles: A survey Nouf Ibrahim Altmami, Mohamed El Bachir Menai
Journal of King Saud University - Computer and Information Sciences[pdf] - Multi-document Summarization via Deep Learning Techniques: A Survey Congbo Ma, Wei Emma Zhang, Mingyu Guo, Hu Wang, Quan Z. Sheng [pdf]
- Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and Challenges Dima Suleiman, Arafat A. Awajan [pdf]
- A Survey of Knowledge-Enhanced Text Generation Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, Meng Jiang [pdf]
- From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information Shen Gao, Xiuying Chen, Zhaochun Ren, Dongyan Zhao, Rui Yan
IJCAI20[pdf] - Neural Abstractive Text Summarization with Sequence-to-Sequence Models Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, Chandan K. Reddy [pdf]
- A Survey on Neural Network-Based Summarization Methods Yue Dong [pdf]
- Automated text summarisation and evidence-based medicine: A survey of two domains Abeed Sarker, Diego Molla, Cecile Paris [pdf]
- Automatic Keyword Extraction for Text Summarization: A Survey Santosh Kumar Bharti, Korra Sathya Babu [pdf]
- Text Summarization Techniques: A Brief Survey Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saeid Safaei, Elizabeth D. Trippe, Juan B. Gutierrez, Krys Kochut [pdf]
- Recent automatic text summarization techniques: a survey Mahak Gambhir, Vishal Gupta [pdf]
Toolkit
- Summary Workbench: Unifying Application and Evaluation of Text Summarization Models Shahbaz Syed, Dominik Schwabe, Martin Potthast
EMNLP 2022 Demo[pdf] [demo]
[Abs] This paper presents Summary Workbench, a new tool for developing and evaluating text summarization models. New models and evaluation measures can be easily integrated as Docker-based plugins, allowing to examine the quality of their summaries against any input and to evaluate them using various evaluation measures. Visual analyses combining multiple measures provide insights into the models' strengths and weaknesses. The tool is hosted at \url{this https URL} and also supports local deployment for private resources.
- iFacetSum: Coreference-based Interactive Faceted Summarization for Multi-Document Exploration Eran Hirsch, Alon Eirew, Ori Shapira, Avi Caciularu, Arie Cattan, Ori Ernst, Ramakanth Pasunuru, Hadar Ronen, Mohit Bansal, Ido Dagan
EMNLP 2021[pdf] [demo] - SummerTime: Text Summarization Toolkit for Non-experts Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng, Yusen Zhang, Tao Yu, Ahmed Hassan Awadallah, Dragomir Radev
EMNLP 2021 Demo Track[pdf] [Demo] - Summary Explorer: Visualizing the State of the Art in Text Summarization Shahbaz Syed, Tariq Yousef, Khalid Al-Khatib, Stefan Jänicke, Martin Potthast [pdf] [web]
- fastnlp/fastSum [code]
- Graph4NLP [code] [summarization]
- CTRLsum: Towards Generic Controllable Text Summarization [pdf] [code]
EMNLP 2022
[Abs] Current summarization systems yield generic summaries that are disconnected from users’ preferences and expectations. To address this limitation, we present CTRLsum, a generic framework to control generated summaries through a set of keywords. During training keywords are extracted automatically without requiring additional human annotations. At test time CTRLsum features a control function to map control signal to keywords; through engineering the control function, the same trained model is able to be applied to control summaries on various dimensions, while neither affecting the model training process nor the pretrained models. We additionally explore the combination of keywords and text prompts for more control tasks. Experiments demonstrate the effectiveness of CTRLsum on three domains of summarization datasets and five control tasks: (1) entity-centric and (2) length-controllable summarization, (3) contribution summarization on scientific papers, (4) invention purpose summarization on patent filings, and (5) question-guided summarization on news articles. Moreover, when used in a standard, unconstrained summarization setting, CTRLsum is comparable or better than strong pretrained systems.
- OpenNMT-py: Open-Source Neural Machine Translation [pdf] [code]
- Fairseq: Facebook AI Research Sequence-to-Sequence Toolkit written in Python. [code]
- LeafNATS: An Open-Source Toolkit and Live Demo System for Neural Abstractive Text Summarization Tian Shi, Ping Wang, Chandan K. Reddy
NAACL19[pdf] [code] - TransformerSum [code]
Analysis
- Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation Raymond Li, Wen Xiao, Linzi Xing, Lanjun Wang, Gabriel Murray, Giuseppe Carenini
EMNLP 2022[pdf] [code]
[Abs] The multi-head self-attention mechanism of the transformer model has been thoroughly investigated recently. In one vein of study, researchers are interested in understanding why and how transformers work. In another vein, researchers propose new attention augmentation methods to make transformers more accurate, efficient and interpretable. In this paper, we combine these two lines of research in a human-in-the-loop pipeline to first discover important task-specific attention patterns. Then those patterns are injected, not only to smaller models, but also to the original model. The benefits of our pipeline and discovered patterns are demonstrated in two case studies with extractive summarization and topic segmentation. After discovering interpretable patterns in BERT-based models fine-tuned for the two downstream tasks, experiments indicate that when we inject the patterns into attention heads, the models show considerable improvements in accuracy and efficiency.
- Analyzing Multi-Task Learning for Abstractive Text Summarization Frederic Kirstein, Jan Philip Wahle, Terry Ruas, Bela Gipp `` [pdf]
[Abs] Despite the recent success of multi-task learning and pre-finetuning for natural language understanding, few works have studied the effects of task families on abstractive text summarization. Task families are a form of task grouping during the pre-finetuning stage to learn common skills, such as reading comprehension. To close this gap, we analyze the influence of multi-task learning strategies using task families for the English abstractive text summarization task. We group tasks into one of three strategies, i.e., sequential, simultaneous, and continual multi-task learning, and evaluate trained models through two downstream tasks. We find that certain combinations of task families (e.g., advanced reading comprehension and natural language inference) positively impact downstream performance. Further, we find that choice and combinations of task families influence downstream performance more than the training scheme, supporting the use of task families for abstractive text summarization.
- On Decoding Strategies for Neural Text Generators Gian Wiher, Clara Meister, Ryan Cotterell [pdf]
- Training Dynamics for Text Summarization Models Tanya Goyal, Jiacheng Xu, Junyi Jessy Li, Greg Durrett [https://arxiv.org/abs/2110.08370]
- Does Summary Evaluation Survive Translation to Other Languages? Neslihan Iskender, Oleg Vasilyev, Tim Polzehl, John Bohannon, Sebastian Möller [pdf]
- How well do you know your summarization datasets? Priyam Tejaswin, Dhruv Naik, Pengfei Liu
Findings of ACL 2021[pdf] [code] - Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution Jiacheng Xu, Greg Durrett
ACL2021[pdf] [code] - To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text Matt Wilber, William Timkey, Marten Van Schijndel
Findings of ACL 2021[pdf] [code] - What Makes a Good Summary? Reconsidering the Focus of Automatic Summarization Maartje ter Hoeve, Julia Kiseleva, Maarten de Rijke [pdf]
- Intrinsic Evaluation of Summarization Datasets Rishi Bommasani, Claire Cardie
EMNLP20[pdf] - Metrics also Disagree in the Low Scoring Range: Revisiting Summarization Evaluation Metrics Manik Bhandari, Pranav Gour, Atabak Ashfaq, Pengfei Liu
COLING20 Short[pdf] [code] - At Which Level Should We Extract? An Empirical Analysis on Extractive Document Summarization Qingyu Zhou, Furu Wei, Ming Zhou
COLING20[pdf] - Corpora Evaluation and System Bias detection in Multi Document Summarization Alvin Dey, Tanya Chowdhury, Yash Kumar, Tanmoy Chakraborty
Findings of EMNLP[pdf] - Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries Daniel Deutsch, Dan Roth [pdf] [code]
- Understanding Neural Abstractive Summarization Models via Uncertainty Jiacheng Xu, Shrey Desai, Greg Durrett
EMNLP20 Short[pdf] [code] - Re-evaluating Evaluation in Text Summarization Manik Bhandari, Pranav Gour, Atabak Ashfaq, Pengfei Liu, Graham Neubig
EMNLP20[pdf] [code] - CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems Yiran Chen, Pengfei Liu, Ming Zhong, Zi-Yi Dou, Danqing Wang, Xipeng Qiu, Xuanjing Huang
EMNLP20[pdf] [code] - What Have We Achieved on Text Summarization? Dandan Huang, Leyang Cui, Sen Yang, Guangsheng Bao, Kun Wang, Jun Xie, Yue Zhang
EMNLP20[pdf] - Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization Zhengyuan Liu, Ke Shi, Nancy F. Chen
Findings of EMNLP20[pdf] - Extractive Summarization as Text Matching Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, Xuanjing Huang
ACL20[pdf] [code] - Neural Text Summarization: A Critical Evaluation Wojciech Kryściński, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher
EMNLP19[pdf] - Earlier Isn’t Always Better:Sub-aspect Analysis on Corpus and System Biases in Summarization Taehee Jung, Dongyeop Kang, Lucas Mentch, Eduard Hovy
EMNLP19[pdf] [code] - A Closer Look at Data Bias in Neural Extractive Summarization Models Ming Zhong, Danqing Wang, Pengfei Liu, Xipeng Qiu, Xuanjing Huang
EMNLP19 Workshop[pdf] - Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses Matt Grenander, Yue Dong, Jackie Chi Kit Cheung, Annie Louis
EMNLP19 Short[pdf] - Searching for Effective Neural Extractive Summarization: What Works and What's Next Ming Zhong, Pengfei Liu, Danqing Wang, Xipeng Qiu, Xuanjing Huang
ACL19[pdf] [code] - Content Selection in Deep Learning Models of Summarization Chris Kedzie, Kathleen McKeown, Hal Daumé III
EMNLP18[pdf] [code]
Thesis
- Principled Approaches to Automatic Text Summarization Maxime Peyrard [pdf]
- Neural Text Summarization and Generation Piji Li [pdf]
Theory
- Bayesian Active Summarization Alexios Gidiotis, Grigorios Tsoumakas [pdf]
- RefSum: Refactoring Neural Summarization Yixin Liu, Zi-Yi Dou, Pengfei Liu
NAACL21[pdf] [code] - Principled Approaches to Automatic Text Summarization Maxime Peyrard [pdf]
- KLearn: Background Knowledge Inference from Summarization Data Maxime Peyrard, Robert West
Findings of EMNLP20[pdf] [code] - A Simple Theoretical Model of Importance for Summarization Maxime Peyrard
ACL19[pdf] - BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle Peter West, Ari Holtzman, Jan Buys, Yejin Choi
EMNLP19[pdf] [code]
Dataset
Dialogue
Dataset
- MeetingBank: A Benchmark Dataset for Meeting Summarization Yebowen Hu, Timothy Ganter, Hanieh Deilamsalehy, Franck Dernoncourt, Hassan Foroosh, Fei Liu
ACL 2023[pdf] [data]
[Abs] As the number of recorded meetings increases, it becomes increasingly important to utilize summarization technology to create useful summaries of these recordings. However, there is a crucial lack of annotated meeting corpora for developing this technology, as it can be hard to collect meetings, especially when the topics discussed are confidential. Furthermore, meeting summaries written by experienced writers are scarce, making it hard for abstractive summarizers to produce sensible output without a reliable reference. This lack of annotated corpora has hindered the development of meeting summarization technology. In this paper, we present MeetingBank, a new benchmark dataset of city council meetings over the past decade. MeetingBank is unique among other meeting corpora due to its divide-and-conquer approach, which involves dividing professionally written meeting minutes into shorter passages and aligning them with specific segments of the meeting. This breaks down the process of summarizing a lengthy meeting into smaller, more manageable tasks. The dataset provides a new testbed of various meeting summarization systems and also allows the public to gain insight into how council decisions are made. We make the collection, including meeting video links, transcripts, reference summaries, agenda, and other metadata, publicly available to facilitate the development of better meeting summarization techniques.
- ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts Rajdeep Mukherjee, Abhinav Bohra, Akash Banerjee, Soumya Sharma, Manjunath Hegde, Afreen Shaikh, Shivani Shrivastava, Koustuv Dasgupta, Niloy Ganguly, Saptarshi Ghosh, Pawan Goyal
EMNLP 2022[pdf] [data]
[Abs] Despite tremendous progress in automatic summarization, state-of-the-art methods are predominantly trained to excel in summarizing short newswire articles, or documents with strong layout biases such as scientific articles or government reports. Efficient techniques to summarize financial documents, including facts and figures, have largely been unexplored, majorly due to the unavailability of suitable datasets. In this work, we present ECTSum, a new dataset with transcripts of earnings calls (ECTs), hosted by publicly traded companies, as documents, and short experts-written telegram-style bullet point summaries derived from corresponding Reuters articles. ECTs are long unstructured documents without any prescribed length limit or format. We benchmark our dataset with state-of-the-art summarizers across various metrics evaluating the content quality and factual consistency of the generated summaries. Finally, we present a simple-yet-effective approach, ECT-BPS, to generate a set of bullet points that precisely capture the important facts discussed in the calls.
- TODSum: Task-Oriented Dialogue Summarization with State Tracking Lulu Zhao, Fujia Zheng, Keqing He, Weihao Zeng, Yuejie Lei, Huixing Jiang, Wei Wu, Weiran Xu, Jun Guo, Fanyu Meng [pdf]
- TWEETSUMM - A Dialog Summarization Dataset for Customer Service Guy Feigenblat, Chulaka Gunasekara, Benjamin Sznajder, Sachindra Joshi, David Konopnicki, Ranit Aharonov
Findings of EMNLP 2021[pdf] [data] - ForumSum: A Multi-Speaker Conversation Summarization Dataset Misha Khalman, Yao Zhao, Mohammad Saleh
EMNLP 2021 Findings[pdf] [data] - CSDS: A Fine-grained Chinese Dataset for Customer Service Dialogue Summarization Haitao Lin, Liqun Ma, Junnan Zhu, Lu Xiang, Yu Zhou, Jiajun Zhang, Chengqing Zong
EMNLP 2021[pdf] [data] - EmailSum: Abstractive Email Thread Summarization Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, Mohit Bansal
ACL 2021[pdf] [data] - DialSumm: A Real-Life Scenario Dialogue Summarization Dataset Yulong Chen, Yang Liu, Liang Chen, Yue Zhang
Findings of ACL21[pdf] [data] - ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining Alexander R. Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang, Haoran Li, Yashar Mehdad, Dragomir Radev
ACL 2021[pdf] [code] - MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization Chenguang Zhu, Yang Liu, Jie Mei, Michael Zeng
NAACL21[pdf] [code] - QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev
NAACL21[pdf] [data] - Storytelling with Dialogue: A Critical Role Dungeons and Dragons Dataset Revanth Rameshkumar, Peter Bailey
ACL20[pdf] [data] - SumTitles: a Summarization Dataset with Low Extractiveness Valentin Malykh, Konstantin Chernis, Ekaterina Artemova, Irina Piontkovskaya
COLING20[pdf] [code] - Summarizing Medical Conversations via Identifying Important Utterances Yan Song, Yuanhe Tian, Nan Wang, Fei Xia
COLING20[pdf] [code] - GupShup: Summarizing Open-Domain Code-Switched Conversations Laiba Mehnaz, Debanjan Mahata, Rakesh Gosangi, Uma Sushmitha Gunturi, Riya Jain, Gauri Gupta, Amardeep Kumar, Isabelle G. Lee, Anish Acharya, Rajiv Ratn Shah
EMNLP 2021[pdf][code] - SummScreen: A Dataset for Abstractive Screenplay Summarization Mingda Chen, Zewei Chu, Sam Wiseman, Kevin Gimpel
ACL 2022[pdf] [data]
[Abs] We introduce SummScreen, a summarization dataset comprised of pairs of TV series transcripts and human written recaps. The dataset provides a challenging testbed for abstractive summarization for several reasons. Plot details are often expressed indirectly in character dialogues and may be scattered across the entirety of the transcript. These details must be found and integrated to form the succinct plot descriptions in the recaps. Also, TV scripts contain content that does not directly pertain to the central plot but rather serves to develop characters or provide comic relief. This information is rarely contained in recaps. Since characters are fundamental to TV series, we also propose two entity-centric evaluation metrics. Empirically, we characterize the dataset by evaluating several methods, including neural models and those based on nearest neighbors. An oracle extractive approach outperforms all benchmarked models according to automatic metrics, showing that the neural models are unable to fully exploit the input transcripts. Human evaluation and qualitative analysis reveal that our non-oracle models are competitive with their oracle counterparts in terms of generating faithful plot events and can benefit from better content selectors. Both oracle and non-oracle models generate unfaithful facts, suggesting future research directions.
- SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization Bogdan Gliwa, Iwona Mochol, Maciej Biesek, Aleksander Wawer
EMNLP19[pdf] [data] - Dial2Desc: End-to-end Dialogue Description Generation Haojie Pan, Junpei Zhou, Zhou Zhao, Yan Liu, Deng Cai, Min Yang [pdf]
- The AMI meeting corpus: A pre-announcement Carletta, Jean and Ashby, Simone and Bourban, Sebastien and Flynn, Mike and Guillemot, Mael and Hain, Thomas and Kadlec, Jaroslav and Karaiskos, Vasilis and Kraaij, Wessel and Kronenthal, Melissa and others [pdf]
- The ICSI meeting corpus Janin, Adam and Baron, Don and Edwards, Jane and Ellis, Dan and Gelbart, David and Morgan, Nelson and Peskin, Barbara and Pfau, Thilo and Shriberg, Elizabeth and Stolcke, Andreas and others [pdf]
Email Summarization
- Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization Kexun Zhang, Jiaao Chen, Diyi Yang
Findings of ACL 2022[pdf] - EmailSum: Abstractive Email Thread Summarization Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, Mohit Bansal
ACL 2021[pdf] [data] - Smart To-Do: Automatic Generation of To-Do Items from Emails Sudipto Mukherjee, Subhabrata Mukherjee, Marcello Hasegawa, Ahmed Hassan Awadallah, Ryen White
ACL 2020[pdf] [code] [bib] - Identifying Implicit Quotes for Unsupervised Extractive Summarization of Conversations Ryuji Kano, Yasuhide Miura, Tomoki Taniguchi, Tomoko Ohkuma
AACL20[pdf] - This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation Rui Zhang, Joel Tetreault
ACL 2019[pdf] [data] [bib] - Building a Dataset for Summarization and Keyword Extraction from Emails Vanessa Loza, Shibamouli Lahiri, Rada Mihalcea, Po-Hsiang Lai
LREC 2014[pdf] - A Publicly Available Annotated Corpus for Supervised Email Summarization Jan Ulrich, Gabriel Murray, Giuseppe Carenini
AAAI 2008[pdf] - Summarizing Email Conversations with Clue Words Giuseppe Carenini, Raymond T. Ng, Xiaodong Zhou
WWW 2007[pdf] - Task-focused Summarization of Email Simon H. Corston-Oliver Eric Ringger Michael Gamon Richard Campbell
ACL 2004[pdf] - Summarizing email threads Owen Rambow, Lokesh Shrestha, John Chen, Chirsty Lauridsen
NAACL 2004[pdf] [bib] - Facilitating email thread access by extractive summary generation Ani Nenkova
Recent advances in natural language processing III: selected papers from RANLP[pdf] - Summarizing Archived Discussions: A Beginning Paula S. Newman, John C. Blitzer
Proceedings of the 8th international conference on Intelligent user interfaces[pdf] - Combining linguistic and machine learning techniques for email summarization Smaranda Muresan, Evelyne Tzoukermann, Judith L. Klavans
Proceedings of the ACL 2001 Workshop on Computational Natural Language Learning (ConLL) 2001[pdf] [bib]
Meeting Summarization
- Learning to Rank Utterances for Query-Focused Meeting Summarization Xingxian Liu, Yajing Xu
Findings of ACL 2023[pdf]
[Abs] Query-focused meeting summarization(QFMS) aims to generate a specific summary for the given query according to the meeting transcripts. Due to the conflict between long meetings and limited input size, previous works mainly adopt extract-then-summarize methods, which use extractors to simulate binary labels or ROUGE scores to extract utterances related to the query and then generate a summary. However, the previous approach fails to fully use the comparison between utterances. To the extractor, comparison orders are more important than specific scores. In this paper, we propose a Ranker-Generator framework. It learns to rank the utterances by comparing them in pairs and learning from the global orders, then uses top utterances as the generator’s input. We show that learning to rank utterances helps to select utterances related to the query effectively, and the summarizer can benefit from it. Experimental results on QMSum show that the proposed model outperforms all existing multi-stage models with fewer parameters.
- ExplainMeetSum: A Dataset for Explainable Meeting Summarization Aligned with Human Intent Hyun Kim, Minsoo Cho, Seung-Hoon Na
ACL 2023[pdf] [code]
[Abs] To enhance the explainability of meeting summarization, we construct a new dataset called “ExplainMeetSum,” an augmented version of QMSum, by newly annotating evidence sentences that faithfully “explain” a summary. Using ExplainMeetSum, we propose a novel multiple extractor guided summarization, namely Multi-DYLE, which extensively generalizes DYLE to enable using a supervised extractor based on human-aligned extractive oracles. We further present an explainability-aware task, named “Explainable Evidence Extraction” (E3), which aims to automatically detect all evidence sentences that support a given summary. Experimental results on the QMSum dataset show that the proposed Multi-DYLE outperforms DYLE with gains of up to 3.13 in the ROUGE-1 score. We further present the initial results on the E3 task, under the settings using separate and joint evaluation metrics.
- VCSUM: A Versatile Chinese Meeting Summarization Dataset Han Wu, Mingjie Zhan, Haochen Tan, Zhaohui Hou, Ding Liang, Linqi Song
Findings of ACL 2023[pdf] [data]
[Abs] Compared to news and chat summarization, the development of meeting summarization is hugely decelerated by the limited data. To this end, we introduce a versatile Chinese meeting summarization dataset, dubbed VCSum, consisting of 239 real-life meetings, with a total duration of over 230 hours. We claim our dataset is versatile because we provide the annotations of topic segmentation, headlines, segmentation summaries, overall meeting summaries, and salient sentences for each meeting transcript. As such, the dataset can adapt to various summarization tasks or methods, including segmentation-based summarization, multi-granularity summarization and retrieval-then-generate summarization. Our analysis confirms the effectiveness and robustness of VCSum. We also provide a set of benchmark models regarding different downstream summarization tasks on VCSum to facilitate further research.
- Query-Utterance Attention with Joint modeling for Query-Focused Meeting Summarization Xingxian Liu, Bin Duan, Bo Xiao, Yajing Xu
ICASSP 2023[pdf]
[Abs] Query-focused meeting summarization (QFMS) aims to generate summaries from meeting transcripts in response to a given query. Previous works typically concatenate the query with meeting transcripts and implicitly model the query relevance only at the token level with attention mechanism. However, due to the dilution of key query-relevant information caused by long meeting transcripts, the original transformer-based model is insufficient to highlight the key parts related to the query. In this paper, we propose a query-aware framework with joint modeling token and utterance based on Query-Utterance Attention. It calculates the utterance-level relevance to the query with a dense retrieval module. Then both token-level query relevance and utterance-level query relevance are combined and incorporated into the generation process with attention mechanism explicitly. We show that the query relevance of different granularities contributes to generating a summary more related to the query. Experimental results on the QMSum dataset show that the proposed model achieves new state-of-the-art performance.
- Meeting Decision Tracker: Making Meeting Minutes with De-Contextualized Utterances Shumpei Inoue, Hy Nguyen, Pham Viet Hoang, Tsungwei Liu, Minh-Tien Nguyen
AACL-IJCNLP 2022[pdf] [demo]
[Abs] Meetings are a universal process to make decisions in business and project collaboration. The capability to automatically itemize the decisions in daily meetings allows for extensive tracking of past discussions. To that end, we developed Meeting Decision Tracker, a prototype system to construct decision items comprising decision utterance detector (DUD) and decision utterance rewriter (DUR). We show that DUR makes a sizable contribution to improving the user experience by dealing with utterance collapse in natural conversation. An introduction video of our system is also available at this https URL.
- ESSumm: Extractive Speech Summarization from Untranscribed Meeting Jun Wang
Interspeech 2022[pdf]
[Abs] In this paper, we propose a novel architecture for direct extractive speech-to-speech summarization, ESSumm, which is an unsupervised model without dependence on intermediate transcribed text. Different from previous methods with text presentation, we are aimed at generating a summary directly from speech without transcription. First, a set of smaller speech segments are extracted based on speech signal's acoustic features. For each candidate speech segment, a distance-based summarization confidence score is designed for latent speech representation measure. Specifically, we leverage the off-the-shelf self-supervised convolutional neural network to extract the deep speech features from raw audio. Our approach automatically predicts the optimal sequence of speech segments that capture the key information with a target summary length. Extensive results on two well-known meeting datasets (AMI and ICSI corpora) show the effectiveness of our direct speech-based method to improve the summarization quality with untranscribed data. We also observe that our unsupervised speech-based method even performs on par with recent transcript-based summarization approaches, where extra speech recognition is required.
- Abstractive Meeting Summarization: A Survey Virgile Rennard, Guokan Shang, Julie Hunter, Michalis Vazirgiannis [pdf]
[Abs] Recent advances in deep learning, and especially the invention of encoder-decoder architectures, has significantly improved the performance of abstractive summarization systems. While the majority of research has focused on written documents, we have observed an increasing interest in the summarization of dialogues and multi-party conversation over the past few years. A system that could reliably transform the audio or transcript of a human conversation into an abridged version that homes in on the most important points of the discussion would be valuable in a wide variety of real-world contexts, from business meetings to medical consultations to customer service calls. This paper focuses on abstractive summarization for multi-party meetings, providing a survey of the challenges, datasets and systems relevant to this task and a discussion of promising directions for future study.
- ALIGNMEET: A Comprehensive Tool for Meeting Annotation, Alignment, and Evaluation Peter Polák, Muskaan Singh, Anna Nedoluzhko, Ondřej Bojar
LREC 2022[pdf] [data] - TANet: Thread-Aware Pretraining for Abstractive Conversational Summarization Ze Yang, Liran Wang, Zhoujin Tian, Wei Wu, Zhoujun Li
Findings of NAACL 2022[pdf]
[Abs] Although pre-trained language models (PLMs) have achieved great success and become a milestone in NLP, abstractive conversational summarization remains a challenging but less studied task. The difficulty lies in two aspects. One is the lack of large-scale conversational summary data. Another is that applying the existing pre-trained models to this task is tricky because of the structural dependence within the conversation and its informal expression, etc. In this work, we first build a large-scale (11M) pretraining dataset called RCSum, based on the multi-person discussions in the Reddit community. We then present TANet, a thread-aware Transformer-based network. Unlike the existing pre-trained models that treat a conversation as a sequence of sentences, we argue that the inherent contextual dependency among the utterances plays an essential role in understanding the entire conversation and thus propose two new techniques to incorporate the structural information into our model. The first is thread-aware attention which is computed by taking into account the contextual dependency within utterances. Second, we apply thread prediction loss to predict the relations between utterances. We evaluate our model on four datasets of real conversations, covering types of meeting transcripts, customer-service records, and forum threads. Experimental results demonstrate that TANet achieves a new state-of-the-art in terms of both automatic evaluation and human judgment.
- Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents Yusen Zhang, Ansong Ni, Ziming Mao, Chen Henry Wu, Chenguang Zhu, Budhaditya Deb, Ahmed H. Awadallah, Dragomir Radev, Rui Zhang
ACL 2022[pdf] [code]
[Abs] Text summarization helps readers capture salient information from documents, news, interviews, and meetings. However, most state-of-the-art pretrained language models (LM) are unable to efficiently process long text for many summarization tasks. In this paper, we propose SummN, a simple, flexible, and effective multi-stage framework for input texts that are longer than the maximum context length of typical pretrained LMs. SummN first splits the data samples and generates a coarse summary in multiple stages and then produces the final fine-grained summary based on it. Our framework can process input text of arbitrary length by adjusting the number of stages while keeping the LM input size fixed. Moreover, it can deal with both single-source documents and dialogues, and it can be used on top of different backbone abstractive summarization models. To the best of our knowledge, SummN is the first multi-stage split-then-summarize framework for long input summarization. Our experiments demonstrate that SummN outperforms previous state-of-the-art methods by improving ROUGE scores on three long meeting summarization datasets AMI, ICSI, and QMSum, two long TV series datasets from SummScreen, and a long document summarization dataset GovReport. Our data and code are available at https://github.com/psunlpgroup/Summ-N.
- Exploring Neural Models for Query-Focused Summarization Jesse Vig, Alexander R. Fabbri, Wojciech Kryściński [pdf] [code]
- Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment MengNan Qi, Hao Liu, YuZhuo Fu, Ting Liu
EMNLP 2021 Findings[pdf] - Meeting Summarization with Pre-training and Clustering Methods Andras Huebner, Wei Ji, Xiang Xiao [pdf] [code]
- Context or No Context? A preliminary exploration of human-in-the-loop approach for Incremental Temporal Summarization in meetings Nicole Beckage, Shachi H Kumar, Saurav Sahay, Ramesh Manuvinakurike
EMNLP 2021| newsum[pdf] - RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization Chenxin An, Ming Zhong, Zhichao Geng, Jianqiang Yang, Xipeng Qiu [pdf]
- An Exploratory Study on Long Dialogue Summarization: What Works and What's Next Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev
Findings of EMNLP 2021 Short[pdf] - DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization Ming Zhong, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
AAAI 2022[pdf] [code] - Dynamic Sliding Window for Meeting Summarization Zhengyuan Liu, Nancy F. Chen
SummDial@SIGDial 2021[pdf] - MeetSum: Transforming Meeting Transcript Summarization using Transformers! Nima Sadri, Bohan Zhang, Bihan Liu [pdf]
- Incremental temporal summarization in multiparty meetings Ramesh Manuvinakurike, Saurav Sahay, Wenda Chen, Lama Nachman
SIGIR 2021[pdf] - Abstractive Spoken Document Summarization using Hierarchical Model with Multi-stage Attention Diversity Optimization Potsawee Manakul, Mark J. F. Gales, Linlin Wang
INTERSPEECH 2020[pdf] [code] - What are meeting summaries? An analysis of human extractive summaries in meeting corpus Fei Liu, Yang Liu
SIGDIAL 2008[pdf] - Exploring Speaker Characteristics for Meeting Summarization Fei Liu, Yang Liu
INTERSPEECH 2010[pdf] - Automatic meeting summarization and topic detection system Tai-Chia Huang, Chia-Hsuan Hsieh, Hei-Chia Wang [pdf]
- A keyphrase based approach to interactive meeting summarization Korbinian Riedhammer, Benoit Favre, Dilek Hakkani-T¨ur
2008 IEEE Spoken Language Technology Workshop[pdf] - A global optimization framework for meeting summarization Dan Gillick, Korbinian Riedhammerm, Benoit Favre, Dilek Hakkani-Tur
2009 IEEE International Conference on Acoustics, Speech and Signal Processing[pdf] - Evaluating the effectiveness of features and sampling in extractive meeting summarization Shasha Xie, Yang Liu, Hui Lin
SLT 2008[pdf] - Abstractive Meeting Summarization Using Dependency Graph Fusion Siddhartha Banerjee, Prasenjit Mitra, Kazunari Sugiyama
WWW 2015[pdf] - Automatic Community Creation for Abstractive Spoken Conversation Summarization Karan Singla, Evgeny Stepanov, Ali Orkan Bayer, Giuseppe Carenini, Giuseppe Riccardi
ACL 2017 workshop[pdf] [bib] - Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Jean-Pierre Tixier, Polykarpos Meladianos, Michalis Vazirgiannis, Jean-Pierre Lorré
ACL18[pdf] [code] - Abstractive meeting summarization based on an attentional neural model Nouha Dammak, Yassine BenAyed [pdf]
- A Study of Text Summarization Techniques for Generating Meeting Minutes Tu My Doan, Francois Jacquenet, Christine Largeron, Marc Bernard
RCIS 2020[pdf] - Meeting Summarization, A Challenge for Deep Learning Francois Jacquenet, Marc Bernard, Christine Largeron
IWANN 2019[pdf] - Generating Abstractive Summaries from Meeting Transcripts Siddhartha Banerjee, Prasenjit Mitra, Kazunari Sugiyama
Proceedings of the 2015 ACM Symposium on Document Engineering, DocEng' 2015[pdf] - Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation Paul Tardy, David Janiszek, Yannick Estève, Vincent Nguyen
LREC 2020[pdf] [bib] - Dialogue Discourse-Aware Graph Model and Data Augmentation for Meeting Summarization Xiachong Feng, Xiaocheng Feng, Bing Qin, Xinwei Geng
IJCAI21[pdf] [code] - How Domain Terminology Affects Meeting Summarization Performance Jia Jin Koay, Alexander Roustai, Xiaojin Dai, Dillon Burns, Alec Kerrigan, Fei Liu
COLING20 Short[pdf] [code] - How to Interact and Change? Abstractive Dialogue Summarization with Dialogue Act Weight and Topic Change Info Jiasheng Di, Xiao Wei, Zhenyu Zhang
KSEM 2020[pdf] [code] - Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts Chih-Wen Goo, Yun-Nung Chen
SLT18[pdf] [code] - A Sliding-Window Approach to Automatic Creation of Meeting Minutes Jia Jin Koay, Alexander Roustai, Xiaojin Dai, Fei Liu [pdf]
- Hierarchical Learning for Generation with Long Source Sequences Tobias Rohde, Xiaoxia Wu, Yinhan Liu [pdf] [[code]](https://g