[1]XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16(1):142-151.[doi:10.11992/tis.202012024]
Copy

An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 1 Page number: 142-151 Column: 吴文俊人工智能科学技术奖论坛 Public date: 2021-01-05
Title:
An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention
Author(s):
XIAO Yuhan1;  LIN Huiping1;  WANG Quanbin2;  TAN Ying2
1. School of Software and Microelectronics, Peking University, Beijing 102600, China;
2. School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China
Keywords:
sentiment analysis; aspect; attention-over-attention; BERT pretrained model; global feature; local feature; deep learning; machine learning
CLC:
TP391
DOI:
10.11992/tis.202012024
Abstract:
Aspect-based sentiment analysis is of great significance to making full use of product reviews to analyze potential user needs. The current research work still has deficiencies. Many studies ignore the importance of local features centered on aspects and fail to handle emotional disturbances effectively. To address these problems, this article proposes a dual features attention-over-attention model with BERT (DFAOA-BERT). For the first time, an AOA (attention-over-attention) mechanism is combined with the BERT pretrained model. DFAOA-BERT also designs global and local feature extractors to fully capture an effective semantic association between aspects and context. According to the experimental results, DFAOA-BERT performs well on three public datasets: restaurant and laptop review datasets from SemEval 2014 Task 4 and the ACL-14 Twitter social review dataset. The effectiveness experiment of submodules also fully proves that each part of DFAOA-BERT makes a significant contribution to the excellent performance.
References:
[1] LIU Bing. Sentiment analysis and opinion mining[J]. Synthesis lectures on human language technologies, 2012, 5(1):1-167.
[2] PANG B, LEE L. Opinion mining and sentiment analysis[M]. Foundations and Trends in Information Retrieval, 2008:1-135.
[3] HUANG Binxuan, OU Yanglan, CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks[C]//Proceedings of the 11th International Conference on Social, Cultural, and Behavioral Modeling. Washington, USA, 2018:197-206.
[4] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014:3104-3112.
[5] TANG D, QIN B, FENG X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics. Osaka, Japan, 2016:3298-3307.
[6] MA D, LI S, ZHANG X, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intel-ligence. Melbourne, Australia, 2017:4068-4074.
[7] XU Weidi, TAN Ying. Semi-supervised target-oriented sentiment classification[J]. Neurocomputing, 2019, 337:120-128.
[8] PETERS M, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.New Orleans, Louisiana, 2018:2227-2237.
[9] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Los Angeles, USA, 2017:6000-6010.
[10] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative p-re-training[EB/OL].[2019-5-10]. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[11] DEVLIN J, CHANG M- W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, USA,2019:4171-4186.
[12] KARIMI A, ROSSI L, PRATI A. Adversarial training for aspect-based sentiment analysis with BERT[EB/OL].[2019-5-10]. https://arxiv.org/abs/2001.11316.
[13] SONG Youwei, WANG Jiahai, JIANG Tao, et al. Att-entional encoder network for targeted sentiment classi-fication[EB/OL].[2019-5-10]. https://arxiv.org/abs/1902.09314.
[14] JIANG Long, YU Mo, ZHOU Ming, et al. Target-dependent twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Portland, USA, 2011:151-160.
[15] CUI Y, CHEN Z, WEI S, et al. Attention-over-attention neural networks for reading comprehension[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver, Canada, 2017:593-602.
[16] PENNINGTON J, SOCHER R, MANNING C. Glove:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, 2014:1532-1543.
[17] ZENG Biqing, YANG Heng, XU Ruyang, et al. LCF:a local context focus mechanism for aspect-based sentiment classification[J]. Applied sciences, 2019, 9(16):1-22.
[18] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task 4:aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland, 2014:27-35.
[19] DONG Li, WEI Furu, TAN Chuanqi, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, Maryland, 2014:49-54.
[20] KIRITCHENKO S, ZHU Xiaodan, CHERRY C, et al. NRC-Canada-2014:detecting aspects and sentiment in customer reviews[C]//Proceedings of the 8th International Workshop on Semantic Evaluation. Dublin, Ireland, 2014:437-442.
[21] CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017:452-461.
[22] XU H, LIU B, SHU L, et al. BERT post-training for review reading comprehension and aspect-based sentiment analysis[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, USA, 2019:2324-2335.
[23] GLOROT X, BENGIO Y. Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Chia Laguna Resort, Italy, 2010:249-256.
[24] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014, 15(1):1929-1958.
[25] KINGMA D P, BA J. Adam:a method for stochastic optimization[C]//The 3rd International Conference for Learning Representations, San Diego. http://arxiv.org/abs/1412.6980.
[26] LI Xin, BING Lidong, LI Piji, et al. A unified model for opinion target extraction and target sentiment prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1):6714-6721.
[27] HU M, PENG Y, HUANG Z, et al. Open-domain targeted sentiment analysis via span-based extraction and classification[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence, Italy, 2019:537 -546.
Similar References:

Memo

-

Last Update: 2021-02-25

Copyright © CAAI Transactions on Intelligent Systems