Extractive Opinion Summarization in Quantized Transformer Spaces

Stefanos Angelidis; Reinald Kim Amplayo; Yoshihiko Suhara; Xiaolan Wang; Mirella Lapata

doi:10.1162/tacl_a_00366

journal article Open Access Jan 01, 2021

Extractive Opinion Summarization in Quantized Transformer Spaces

Stefanos Angelidis Reinald Kim Amplayo Yoshihiko Suhara Xiaolan Wang Mirella Lapata

Transactions of the Association for Computational Linguistics Vol. 9 pp. 277-293 · MIT Press

View at Publisher Save 10.1162/tacl_a_00366

Abstract

Abstract
We present the Quantized Transformer (QT), an unsupervised system for extractive opinion summarization. QT is inspired by Vector- Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables controllable summarization without further training, by utilizing properties of the quantized space to extract aspect-specific summaries. We also make publicly available Space, a large-scale evaluation benchmark for opinion summarizers, comprising general and aspect-specific summaries for 50 hotels. Experiments demonstrate the promise of our approach, which is validated by human studies where judges showed clear preference for our method over competitive baselines.

Topics

No keywords indexed for this article. Browse by subject →

References

45

[1]

Amplayo "Informative and controllable opinion summarization" arXiv preprint arXiv:1909.02322v1 (2019)

[2]

Amplayo "Unsupervised opinion summarization with noising and denoising" (2020) 10.18653/v1/2020.acl-main.175

[3]

Angelidis "Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised" (2018) 10.18653/v1/d18-1403

[4]

Bengio "Estimating or propagating gradients through stochastic neurons for conditional computation" arXiv preprint arXiv: 1308.3432v1 (2013)

[5]

Bražinskas "Unsupervised opinion summarization as copycat-review generation" (2020) 10.18653/v1/2020.acl-main.461

[6]

Cao "Ranking with recursive neural networks and its application to multi-document summarization" (2015)

[7]

Carenini "Multi-document summarization of evaluative text" (2006)

[8]

Cheng "Neural summarization by extracting sentences and words" (2016) 10.18653/v1/p16-1046

[9]

Chu "MeanSum: A neural model for unsupervised multi-document abstractive summarization" (2019)

[10]

Coavoux "Unsupervised aspect-based multi-document abstractive summarization" (2019) 10.18653/v1/d19-5405

[11]

Devlin "BERT: Pre-training of deep bidirectional transformers for language understanding" (2019)

[12]

Fabbrizio "A hybrid approach to multi-document summarization of opinions in reviews" (2014) 10.3115/v1/w14-4408

[13]

Erkan "Lexrank: Graph-based lexical centrality as salience in text summarization" Journal Of Artificial Intelligence Research (2004) 10.1613/jair.1523

[14]

Ganesan "Opinosis: A graph based approach to abstractive summarization of highly redundant opinions" (2010)

[15]

Holtzman "The curious case of neural text degeneration" (2020)

[16]

Minqing "Mining and summarizing customer reviews" (2004)

[17]

Tien "Opinions summarization: Aspect similarity recognition relaxes the constraint of predefined aspects" (2019)

[18]

Isonuma "Unsupervised neural single-document summarization of reviews via learning latent discourse structure and its ranking" (2019) 10.18653/v1/p19-1206

[19]

Kaiser "Discrete autoencoders for sequence models" ArXiv (2018)

[20]

Kingma "Auto-encoding variational Bayes" CoRR (2014)

[21]

Koehn "Statistical significance tests for machine translation evaluation" (2004)

[22]

Liu "Generating Wikipedia by summarizing long sequences" (2018)

[23]

Louviere (2015) 10.1017/cbo9781107337855

[24]

Maddison "The concrete distribution: A continuous relaxation of discrete random variables" (2017)

[25]

Marcheggiani "Hierarchical multi-label conditional random fields for aspect-oriented opinion mining" (2014) 10.1007/978-3-319-06028-6_23

[26]

Miao "Snippext: Semi-supervised opinion mining with augmented data" (2020) 10.1145/3366423.3380144

[27]

Narayan "Ranking sentences for extractive summarization with reinforcement learning" (2018)

[28]

van den Oord "Neural discrete representation learning" (2017)

[29]

Pan "Large scale abstractive multi-review summarization (LSARS) via aspect alignment" (2020) 10.1145/3397271.3401439

[30]

Bo "Opinion mining and sentiment analysis" Foundations and Trends in Information Retrieval (2008)

[31]

Perez-Beltrachini "Generating summaries with topic templates and structured convolutional decoders" (2019) 10.18653/v1/p19-1504

[32]

Peyrard "A simple theoretical model of importance for summarization" (2019) 10.18653/v1/p19-1101

[33]

Radev "Evaluation challenges in large-scale document summarization" (2003)

[34]

Radford "Learning to generate reviews and discovering sentiment" arXiv preprint arXiv:1704.01444v2 (2017)

[35]

Rohrbach "Object hallucination in image captioning" (2018) 10.18653/v1/d18-1437

[36]

Roy "Unsupervised paraphrasing without translation" (2019) 10.18653/v1/p19-1605

[37]

Roy "Theory and experiments on vector quantized autoencoders" arXiv preprint arXiv:1805.11063v2 (2018)

[38]

See "Get to the point: Summarization with pointer-generator networks" (2017) 10.18653/v1/p17-1099

[39]

Suhara "OpinionDigest: A simple framework for opinion summarization" (2020) 10.18653/v1/2020.acl-main.513

[40]

Tian "Aspect and opinion aware abstractive review summarization with reinforced hard typed decoder" (2019) 10.1145/3357384.3358142

[41]

van der Maaten "Visualizing data using t-SNE" Journal of Machine Learning Research (2008)

[42]

Vaswani "Attention is all you need" (2017)

[43]

Wang "Latent aspect rating analysis on review text data: A rating regression approach" (2010) 10.1145/1835804.1835903

[44]

Wang "ExtremeReader: An interactive explorer for customizable and explainable review summarization" (2020) 10.1145/3366424.3383535

[45]

Yasunaga "Graph-based neural multi-document summarization" (2017) 10.18653/v1/k17-1045

Metrics

41

Citations

45

References

Details

Published: Jan 01, 2021
Vol/Issue: 9
Pages: 277-293
License: View

Authors

S

Stefanos Angelidis

University of Edinburgh, United Kingdom. s.angelidis@ed.ac.uk

R

Reinald Kim Amplayo

University of Edinburgh, United Kingdom. reinald.kim@ed.ac.uk

Y

Yoshihiko Suhara

Megagon Labs, United States. yoshi@megagon.ai

X

Xiaolan Wang

Megagon Labs, United States. xiaolan@megagon.ai

M

Mirella Lapata

University of Edinburgh, United Kingdom. mlap@inf.ed.ac.uk

Cite This Article

Stefanos Angelidis, Reinald Kim Amplayo, Yoshihiko Suhara, et al. (2021). Extractive Opinion Summarization in Quantized Transformer Spaces. Transactions of the Association for Computational Linguistics, 9, 277-293. https://doi.org/10.1162/tacl_a_00366

Extractive Opinion Summarization in Quantized Transformer Spaces

You May Also Like