[1]程艳,胡建生,赵松华,等.融合Transformer和交互注意力网络的方面级情感分类模型[J].智能系统学报,2024,19(3):728-737.[doi:10.11992/tis.202303016]
CHENG Yan,HU Jiansheng,ZHAO Songhua,et al.Aspect-level sentiment classification model combining Transformer and interactive attention network[J].CAAI Transactions on Intelligent Systems,2024,19(3):728-737.[doi:10.11992/tis.202303016]
点击复制
CHENG Yan,HU Jiansheng,ZHAO Songhua,et al.Aspect-level sentiment classification model combining Transformer and interactive attention network[J].CAAI Transactions on Intelligent Systems,2024,19(3):728-737.[doi:10.11992/tis.202303016]
融合Transformer和交互注意力网络的方面级情感分类模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第3期
页码:
728-737
栏目:
学术论文—自然语言处理与理解
出版日期:
2024-05-05
- Title:
- Aspect-level sentiment classification model combining Transformer and interactive attention network
- Keywords:
- aspect term; sentiment classification; recurrent neural network; transformer; interactive attention network; BERT; local feature; deep learning
- 分类号:
- TP391
- 文献标志码:
- 2023-11-13
- 摘要:
- 现有的大多数研究者使用循环神经网络与注意力机制相结合的方法进行方面级情感分类任务。然而，循环神经网络不能并行计算，并且模型在训练过程中会出现截断的反向传播、梯度消失和梯度爆炸等问题，传统的注意力机制可能会给句子中重要情感词分配较低的注意力权重。针对上述问题，该文提出了一种融合Transformer和交互注意力网络的方面级情感分类模型。首先利用BERT(bidirectional encoder representation from Transformers)预训练模型来构造词嵌入向量，然后使用Transformer编码器对输入的句子进行并行编码，接着使用上下文动态掩码和上下文动态权重机制来关注与特定方面词有重要语义关系的局部上下文信息。最后在5个英文数据集和4个中文评论数据集上的实验结果表明，该文所提模型在准确率和F1上均表现最优。
- Abstract:
- At present, most researchers use a combination of recurrent neural networks and attention mechanisms for aspect-level sentiment classification tasks. However, the recurrent neural network cannot be computed in parallel, and the models encounter problems, such as truncated backpropagation, gradient vanishing, and gradient exploration, in the training process. Traditional attention mechanisms may assign reduced attention weights to important sentiment words in sentences. An aspect-level sentiment classification model combining Transformer and interactive attention network is proposed to solve these problems. In this approach, the pretrained model, which considers bidirectional encoder representation from Transformers (BERT), is initially used to construct word embedding vectors. Then, Transformer encoders are used to perform parallel encoding for input sentences. Subsequently, the contextual dynamic mask-off code and the contextual dynamic weighting mechanisms are applied to focus on local context information semantically relevant to specific aspect words. Finally, the model is tested on five English datasets and four Chinese review datasets. Experimental results demonstrate that the proposed model outperforms others in terms of accuracy and F1.
- 参考文献/References:
-
[1] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task 4: aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin: Association for Computational Linguistics, 2014: 27–35.
[2] LIU Bing. Sentiment analysis and opinion mining[J]. Synthesis lectures on human language technologies, 2012, 5(1): 1–167.
[3] DENG Dong, JING Liping, YU Jian, et al. Sentiment lexicon construction with hierarchical supervision topic model[J]. IEEE/ACM transactions on audio, speech, and language processing, 2019, 27(4): 704–718.
[4] PANG Bo, LEE L, VAITHYANATHAN S. Thumbs up? : sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - EMNLP ’02. Morristown: Association for Computational Linguistics, 2002: 79–86.
[5] JIANG Long, YU Mo, ZHOU Ming, et al. Target-dependent Twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1. Portland: ACM, 2011: 151–160.
[6] TANG D, QIN B, FENG X, et al. Effective LSTMs for Target-Dependent Sentiment Classification[C]//Proceedings of the 26th International Conference on Computational Linguistics. Stroudsburg: ACL, 2016: 3298–3307.
[7] MA Yukun, PENG Haiyun, KHAN T, et al. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis[J]. Cognitive computation, 2018, 10(4): 639–650.
[8] GU S, ZHANG L, HOU Y, et al. A position-aware bidirectional attention network for aspect-level sentiment analysis[C]//Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg: ACL, 2018: 774–784.
[9] TANG Duyu, QIN Bing, LIU Ting. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 214–224.
[10] XUE Wei, LI Tao. Aspect based sentiment analysis with gated convolutional networks[EB/OL]. (2018-05-18)[2023-03-08]. http://arxiv.org/abs/1805.07043.
[11] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM, 2017: 6000–6010.
[12] CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 452–461.
[13] WANG Yequan, HUANG Minlie, ZHU Xiaoyan, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 606–615.
[14] MA Dehong, LI Sujian, ZHANG Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne: ACM, 2017: 4068–4074.
[15] SONG Youwei, WANG Jiahai, JIANG Tao, et al. Attentional encoder network for targeted sentiment classification[EB/OL]. (2019–02–09)[2023–03–08]. http://arxiv.org/abs/1902.09314.
[16] XU Qiannan, ZHU Li, DAI Tao, et al. Aspect-based sentiment classification with multi-attention network[J]. Neurocomputing, 2020, 388(C): 135–143.
[17] 肖宇晗, 林慧苹, 汪权彬, 等. 基于双特征嵌套注意力的方面词情感分析算法[J]. 智能系统学报, 2021, 16(1): 142–151
XIAO Yuhan, LIN Huiping, WANG Quanbin, et al. An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J]. CAAI transactions on intelligent systems, 2021, 16(1): 142–151
[18] 曾义夫, 蓝天, 吴祖峰, 等. 基于双记忆注意力的方面级别情感分类模型[J]. 计算机学报, 2019, 42(8): 1845–1857
ZENG Yifu, LAN Tian, WU Zufeng, et al. Bi-memory based attention model for aspect level sentiment classification[J]. Chinese journal of computers, 2019, 42(8): 1845–1857
[19] ZENG Biqing, YANG Heng, XU Ruyang, et al. LCF: a local context focus mechanism for aspect-based sentiment classification[J]. Applied sciences, 2019, 9(16): 3389.
[20] PENG Haiyun, MA Yukun, LI Yang, et al. Learning multi-grained aspect target sequence for Chinese sentiment analysis[J]. Knowledge based systems, 2018, 148: 167–176.
[21] ZHAO Yanyan, QIN Bing, LIU Ting. Creating a fine-grained corpus for Chinese sentiment analysis[J]. IEEE intelligent systems, 2015, 30(1): 36–43.
[22] LI Shen, ZHAO Zhe, HU Renfen, et al. Analogical reasoning on Chinese morphological and semantic relations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg: Association for Computational Linguistics, 2018: 138–143.
[23] FAN Feifan, FENG Yansong, ZHAO Dongyan. Multi-grained attention network for aspect-level sentiment classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2018: 3433–3442.
[24] LI Xin, BING Lidong, LAM W, et al. Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2018: 946–956.
[25] CHEN F, YUAN Z, HUANG Y. Multi-source data fusion for aspect-level sentiment classification[J]. Knowledge-based systems, 2020, 187: 104831.
[26] ZENG Jiandian, LIU Tianyi, JIA Weijia, et al. Relation construction for aspect-level sentiment classification[J]. Information sciences, 2022, 586: 209–223.
[27] VASWANI A, BENGIO S, BREVDO E, et al. Tensor2Tensor for neural machine translation[C]//Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Boston: Association for Machine Translation in the Americas, 2018: 193–199.
- 相似文献/References:
-
[1]夏睿,宗成庆.情感文本分类混合模型及特征扩展策略[J].智能系统学报,2011,6(6):483.
XIA Rui,ZONG Chengqing.A hybrid approach to sentiment classification and feature expansion strategy[J].CAAI Transactions on Intelligent Systems,2011,6():483.
[2]郝洁,谢珺,苏婧琼,等.基于词加权LDA算法的无监督情感分类[J].智能系统学报,2016,11(4):539.[doi:10.11992/tis.201606007]
HAO Jie,XIE Jun,SU Jingqiong,et al.An unsupervised approach for sentiment classification based on weighted latent dirichlet allocation[J].CAAI Transactions on Intelligent Systems,2016,11():539.[doi:10.11992/tis.201606007]
[3]高庆吉,赵志华,徐达,等.语音情感识别研究综述[J].智能系统学报,2020,15(1):1.[doi:10.11992/tis.201904065]
GAO Qingji,ZHAO Zhihua,XU Da,et al.Review on speech emotion recognition research[J].CAAI Transactions on Intelligent Systems,2020,15():1.[doi:10.11992/tis.201904065]
[4]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15():460.[doi:10.11992/tis.201812017]
[5]肖宇晗,林慧苹,汪权彬,等.基于双特征嵌套注意力的方面词情感分析算法[J].智能系统学报,2021,16(1):142.[doi:10.11992/tis.202012024]
XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16():142.[doi:10.11992/tis.202012024]
备注/Memo
收稿日期:2023-03-08。
基金项目:国家自然科学基金项目（62167006，61967011）；江西省科技创新基地-智能信息处理与情感计算江西省重点实验室（原江西省智能教育省重点实验室）项目（20212BCD42001）；江西省 03 专项及 5G 项目（20212ABC03A22）；江西省主要学科学术和技术带头人培养计划-领军人才项目（20213BCJL22047）；江西省自然科学基金项目（20212BAB202017）.
作者简介:程艳，教授，博士生导师，主要研究方向为人工智能、智能信息处理、情感分析。发表学术论文70余篇，出版学术专著2部。E-mail：chyan88888@jxnu.edu.cn;胡建生，硕士研究生，主要研究方向为情感分析、深度学习。E-mail：1875560437@qq.com;赵松华，硕士研究生，主要研究方向为知识追踪、教育数据挖掘。E-mail：3140851935@qq.com
通讯作者:程艳. E-mail：chyan88888@jxnu.edu.cn
更新日期/Last Update:
1900-01-01