[1]肖宇晗,林慧苹,汪权彬,等.基于双特征嵌套注意力的方面词情感分析算法[J].智能系统学报,2021,16(1):142-151.[doi:10.11992/tis.202012024]
XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16(1):142-151.[doi:10.11992/tis.202012024]
点击复制
XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16(1):142-151.[doi:10.11992/tis.202012024]
基于双特征嵌套注意力的方面词情感分析算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第1期
页码:
142-151
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2021-01-05
- Title:
- An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention
- Keywords:
- sentiment analysis; aspect; attention-over-attention; BERT pretrained model; global feature; local feature; deep learning; machine learning
- 分类号:
- TP391
- 摘要:
- 针对目前方面词情感分析方法忽视了以方面词为核心的局部特征的重要性，并难以有效减小情感干扰项的负面噪声的问题，本文提出了一种带有基于变换器的双向编码器表示技术（bi-directional encoder representations from transformers，BERT）加持的双特征嵌套注意力模型(dual features attention-over-attention with BERT，DFAOA-BERT)，首次将AOA(attention-over-attention)与BERT预训练模型结合，并设计了全局与局部特征提取器，能够充分捕捉方面词和语境的有效语义关联。实验结果表明：DFAOA-BERT在SemEval 2014任务4中的餐馆评论、笔记本评论和ACL-14 Twitter社交评论这3个公开数据集上均表现优异，而子模块的有效性实验，也充分证明了DFAOA-BERT各个部分的设计合理性。
- Abstract:
- Aspect-based sentiment analysis is of great significance to making full use of product reviews to analyze potential user needs. The current research work still has deficiencies. Many studies ignore the importance of local features centered on aspects and fail to handle emotional disturbances effectively. To address these problems, this article proposes a dual features attention-over-attention model with BERT (DFAOA-BERT). For the first time, an AOA (attention-over-attention) mechanism is combined with the BERT pretrained model. DFAOA-BERT also designs global and local feature extractors to fully capture an effective semantic association between aspects and context. According to the experimental results, DFAOA-BERT performs well on three public datasets: restaurant and laptop review datasets from SemEval 2014 Task 4 and the ACL-14 Twitter social review dataset. The effectiveness experiment of submodules also fully proves that each part of DFAOA-BERT makes a significant contribution to the excellent performance.
- 参考文献/References:
-
[1] LIU Bing. Sentiment analysis and opinion mining[J]. Synthesis lectures on human language technologies, 2012, 5(1):1-167.
[2] PANG B, LEE L. Opinion mining and sentiment analysis[M]. Foundations and Trends in Information Retrieval, 2008:1-135.
[3] HUANG Binxuan, OU Yanglan, CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks[C]//Proceedings of the 11th International Conference on Social, Cultural, and Behavioral Modeling. Washington, USA, 2018:197-206.
[4] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014:3104-3112.
[5] TANG D, QIN B, FENG X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics. Osaka, Japan, 2016:3298-3307.
[6] MA D, LI S, ZHANG X, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intel-ligence. Melbourne, Australia, 2017:4068-4074.
[7] XU Weidi, TAN Ying. Semi-supervised target-oriented sentiment classification[J]. Neurocomputing, 2019, 337:120-128.
[8] PETERS M, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.New Orleans, Louisiana, 2018:2227-2237.
[9] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Los Angeles, USA, 2017:6000-6010.
[10] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative p-re-training[EB/OL].[2019-5-10]. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[11] DEVLIN J, CHANG M- W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, USA,2019:4171-4186.
[12] KARIMI A, ROSSI L, PRATI A. Adversarial training for aspect-based sentiment analysis with BERT[EB/OL].[2019-5-10]. https://arxiv.org/abs/2001.11316.
[13] SONG Youwei, WANG Jiahai, JIANG Tao, et al. Att-entional encoder network for targeted sentiment classi-fication[EB/OL].[2019-5-10]. https://arxiv.org/abs/1902.09314.
[14] JIANG Long, YU Mo, ZHOU Ming, et al. Target-dependent twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Portland, USA, 2011:151-160.
[15] CUI Y, CHEN Z, WEI S, et al. Attention-over-attention neural networks for reading comprehension[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver, Canada, 2017:593-602.
[16] PENNINGTON J, SOCHER R, MANNING C. Glove:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, 2014:1532-1543.
[17] ZENG Biqing, YANG Heng, XU Ruyang, et al. LCF:a local context focus mechanism for aspect-based sentiment classification[J]. Applied sciences, 2019, 9(16):1-22.
[18] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task 4:aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland, 2014:27-35.
[19] DONG Li, WEI Furu, TAN Chuanqi, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, Maryland, 2014:49-54.
[20] KIRITCHENKO S, ZHU Xiaodan, CHERRY C, et al. NRC-Canada-2014:detecting aspects and sentiment in customer reviews[C]//Proceedings of the 8th International Workshop on Semantic Evaluation. Dublin, Ireland, 2014:437-442.
[21] CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017:452-461.
[22] XU H, LIU B, SHU L, et al. BERT post-training for review reading comprehension and aspect-based sentiment analysis[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, USA, 2019:2324-2335.
[23] GLOROT X, BENGIO Y. Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Chia Laguna Resort, Italy, 2010:249-256.
[24] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014, 15(1):1929-1958.
[25] KINGMA D P, BA J. Adam:a method for stochastic optimization[C]//The 3rd International Conference for Learning Representations, San Diego. http://arxiv.org/abs/1412.6980.
[26] LI Xin, BING Lidong, LI Piji, et al. A unified model for opinion target extraction and target sentiment prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1):6714-6721.
[27] HU M, PENG Y, HUANG Z, et al. Open-domain targeted sentiment analysis via span-based extraction and classification[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence, Italy, 2019:537 -546.
- 相似文献/References:
-
[1]赵文清,侯小可,沙海虹.语义规则在微博热点话题情感分析中的应用[J].智能系统学报,2014,9(1):121.[doi:10.3969/j.issn.1673-4785.201208020]
ZHAO Wenqing,HOU Xiaoke,SHA Haihong.Application of semantic rules to sentiment analysis of microblog hot topics[J].CAAI Transactions on Intelligent Systems,2014,9():121.[doi:10.3969/j.issn.1673-4785.201208020]
[2]吴钟强,张耀文,商琳.基于语义特征的多视图情感分类方法[J].智能系统学报,2017,12(5):745.[doi:10.11992/tis.201706026]
WU Zhongqiang,ZHANG Yaowen,SHANG Lin.Multi-view sentiment classification of microblogs based on semantic features[J].CAAI Transactions on Intelligent Systems,2017,12():745.[doi:10.11992/tis.201706026]
[3]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15():460.[doi:10.11992/tis.201812017]
[4]张铭泉,周辉,曹锦纲.基于注意力机制的双BERT有向情感文本分类研究[J].智能系统学报,2022,17(6):1220.[doi:10.11992/tis.202112038]
ZHANG Mingquan,ZHOU Hui,CAO Jingang.Dual BERT directed sentiment text classification based on attention mechanism[J].CAAI Transactions on Intelligent Systems,2022,17():1220.[doi:10.11992/tis.202112038]
[5]胡文彬,陈龙,黄贤波,等.融合交叉注意力的突发事件多模态中文反讽识别模型[J].智能系统学报,2024,19(2):392.[doi:10.11992/tis.202212011]
HU Wenbin,CHEN Long,HUANG Xianbo,et al.A multimodal Chinese sarcasm detection model for emergencies based on cross attention[J].CAAI Transactions on Intelligent Systems,2024,19():392.[doi:10.11992/tis.202212011]
[6]程艳,胡建生,赵松华,等.融合Transformer和交互注意力网络的方面级情感分类模型[J].智能系统学报,2024,19(3):728.[doi:10.11992/tis.202303016]
CHENG Yan,HU Jiansheng,ZHAO Songhua,et al.Aspect-level sentiment classification model combining Transformer and interactive attention network[J].CAAI Transactions on Intelligent Systems,2024,19():728.[doi:10.11992/tis.202303016]
备注/Memo
收稿日期:2020-12-15。
基金项目:国家重点研发计划项目(2018AAA0102301, 2018AAA0100302, 2018YFB1702900)；国家自然科学基金项目(62076010)
作者简介:肖宇晗，硕士研究生，主要研究方向为深度学习、数据挖掘和自然语言处理;林慧苹，副教授，博士，主要研究方向为大数据分析、企业信息服务，主持和参与国家863计划、国家自然科学基金项目、国家重点研发计划项目等多项。发表学术论文20余篇;谭营，教授，博士生导师，主要研究方向为智能科学、计算智能与群体智能、机器学习、人工神经网络、群体机器人、大数据挖掘。烟花算法发明人。吴文俊人工智能科学技术成就奖创新三等奖获得者。发表学术论文 330 余篇，出版学术专著 12 部。
通讯作者:谭营. E-mail：ytan@pku.edu.cn
更新日期/Last Update:
2021-02-25