[1]CHENG Yan,HU Jiansheng,ZHAO Songhua,et al.Aspect-level sentiment classification model combining Transformer and interactive attention network[J].CAAI Transactions on Intelligent Systems,2024,19(3):728-737.[doi:10.11992/tis.202303016]
Copy
Aspect-level sentiment classification model combining Transformer and interactive attention network
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
19
Number of periods:
2024 3
Page number:
728-737
Column:
学术论文—自然语言处理与理解
Public date:
2024-05-05
- Title:
- Aspect-level sentiment classification model combining Transformer and interactive attention network
- Keywords:
- aspect term; sentiment classification; recurrent neural network; transformer; interactive attention network; BERT; local feature; deep learning
- CLC:
- TP391
- DOI:
- 10.11992/tis.202303016
- Abstract:
- At present, most researchers use a combination of recurrent neural networks and attention mechanisms for aspect-level sentiment classification tasks. However, the recurrent neural network cannot be computed in parallel, and the models encounter problems, such as truncated backpropagation, gradient vanishing, and gradient exploration, in the training process. Traditional attention mechanisms may assign reduced attention weights to important sentiment words in sentences. An aspect-level sentiment classification model combining Transformer and interactive attention network is proposed to solve these problems. In this approach, the pretrained model, which considers bidirectional encoder representation from Transformers (BERT), is initially used to construct word embedding vectors. Then, Transformer encoders are used to perform parallel encoding for input sentences. Subsequently, the contextual dynamic mask-off code and the contextual dynamic weighting mechanisms are applied to focus on local context information semantically relevant to specific aspect words. Finally, the model is tested on five English datasets and four Chinese review datasets. Experimental results demonstrate that the proposed model outperforms others in terms of accuracy and F1.
- References:
-
[1] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task 4: aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin: Association for Computational Linguistics, 2014: 27–35.
[2] LIU Bing. Sentiment analysis and opinion mining[J]. Synthesis lectures on human language technologies, 2012, 5(1): 1–167.
[3] DENG Dong, JING Liping, YU Jian, et al. Sentiment lexicon construction with hierarchical supervision topic model[J]. IEEE/ACM transactions on audio, speech, and language processing, 2019, 27(4): 704–718.
[4] PANG Bo, LEE L, VAITHYANATHAN S. Thumbs up? : sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - EMNLP ’02. Morristown: Association for Computational Linguistics, 2002: 79–86.
[5] JIANG Long, YU Mo, ZHOU Ming, et al. Target-dependent Twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1. Portland: ACM, 2011: 151–160.
[6] TANG D, QIN B, FENG X, et al. Effective LSTMs for Target-Dependent Sentiment Classification[C]//Proceedings of the 26th International Conference on Computational Linguistics. Stroudsburg: ACL, 2016: 3298–3307.
[7] MA Yukun, PENG Haiyun, KHAN T, et al. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis[J]. Cognitive computation, 2018, 10(4): 639–650.
[8] GU S, ZHANG L, HOU Y, et al. A position-aware bidirectional attention network for aspect-level sentiment analysis[C]//Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg: ACL, 2018: 774–784.
[9] TANG Duyu, QIN Bing, LIU Ting. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 214–224.
[10] XUE Wei, LI Tao. Aspect based sentiment analysis with gated convolutional networks[EB/OL]. (2018-05-18)[2023-03-08]. http://arxiv.org/abs/1805.07043.
[11] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM, 2017: 6000–6010.
[12] CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 452–461.
[13] WANG Yequan, HUANG Minlie, ZHU Xiaoyan, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 606–615.
[14] MA Dehong, LI Sujian, ZHANG Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne: ACM, 2017: 4068–4074.
[15] SONG Youwei, WANG Jiahai, JIANG Tao, et al. Attentional encoder network for targeted sentiment classification[EB/OL]. (2019–02–09)[2023–03–08]. http://arxiv.org/abs/1902.09314.
[16] XU Qiannan, ZHU Li, DAI Tao, et al. Aspect-based sentiment classification with multi-attention network[J]. Neurocomputing, 2020, 388(C): 135–143.
[17] 肖宇晗, 林慧苹, 汪权彬, 等. 基于双特征嵌套注意力的方面词情感分析算法[J]. 智能系统学报, 2021, 16(1): 142–151
XIAO Yuhan, LIN Huiping, WANG Quanbin, et al. An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J]. CAAI transactions on intelligent systems, 2021, 16(1): 142–151
[18] 曾义夫, 蓝天, 吴祖峰, 等. 基于双记忆注意力的方面级别情感分类模型[J]. 计算机学报, 2019, 42(8): 1845–1857
ZENG Yifu, LAN Tian, WU Zufeng, et al. Bi-memory based attention model for aspect level sentiment classification[J]. Chinese journal of computers, 2019, 42(8): 1845–1857
[19] ZENG Biqing, YANG Heng, XU Ruyang, et al. LCF: a local context focus mechanism for aspect-based sentiment classification[J]. Applied sciences, 2019, 9(16): 3389.
[20] PENG Haiyun, MA Yukun, LI Yang, et al. Learning multi-grained aspect target sequence for Chinese sentiment analysis[J]. Knowledge based systems, 2018, 148: 167–176.
[21] ZHAO Yanyan, QIN Bing, LIU Ting. Creating a fine-grained corpus for Chinese sentiment analysis[J]. IEEE intelligent systems, 2015, 30(1): 36–43.
[22] LI Shen, ZHAO Zhe, HU Renfen, et al. Analogical reasoning on Chinese morphological and semantic relations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg: Association for Computational Linguistics, 2018: 138–143.
[23] FAN Feifan, FENG Yansong, ZHAO Dongyan. Multi-grained attention network for aspect-level sentiment classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2018: 3433–3442.
[24] LI Xin, BING Lidong, LAM W, et al. Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2018: 946–956.
[25] CHEN F, YUAN Z, HUANG Y. Multi-source data fusion for aspect-level sentiment classification[J]. Knowledge-based systems, 2020, 187: 104831.
[26] ZENG Jiandian, LIU Tianyi, JIA Weijia, et al. Relation construction for aspect-level sentiment classification[J]. Information sciences, 2022, 586: 209–223.
[27] VASWANI A, BENGIO S, BREVDO E, et al. Tensor2Tensor for neural machine translation[C]//Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Boston: Association for Machine Translation in the Americas, 2018: 193–199.
- Similar References:
Memo
-
Last Update:
1900-01-01