[1]ZHANG Mingquan,ZHOU Hui,CAO Jingang.Dual BERT directed sentiment text classification based on attention mechanism[J].CAAI Transactions on Intelligent Systems,2022,17(6):1220-1227.[doi:10.11992/tis.202112038]
Copy

Dual BERT directed sentiment text classification based on attention mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 6 Page number: 1220-1227 Column: 学术论文—自然语言处理与理解 Public date: 2022-11-05
Title:
Dual BERT directed sentiment text classification based on attention mechanism
Author(s):
ZHANG Mingquan1; 2;  ZHOU Hui1; 2;  CAO Jingang1; 2
1. School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, China;
2. Intelligent Computing of Complex Energy System Engineering Research Center of the Ministry of Education, North China Electric Power University, Baoding 071003, China
Keywords:
sentiment analysis; bidirectional encoder representation from transform neural network; pretraining model; attention mechanism; deep learning; machine learning; text classification; neural network
CLC:
TP391
DOI:
10.11992/tis.202112038
Abstract:
Understanding the emotional relationships between different political entities in political news texts is a new research topic in the text classification field in computational social science. Traditional methods of sentiment analysis cannot be applied to political news texts because they do not consider the direction of emotional expression between entities. This study proposes a dual BERT-directed sentiment text classification model based on the attention mechanism, which consists of four modules: input module, sentiment analysis module, political entity direction module, and classification module. The structure of the sentiment analysis module and the political entity direction module are identical. Both employ the BERT pretraining model to embed the input information, a three-layer neural network to extract the emotional information or emotional direction information between entities, and an attention mechanism to combine these two kinds of information to classify political news texts. Experiments on comparable data sets show that the model outperforms existing models.
References:
[1] BING Liu. Sentiment analysis and opinion mining[M]. Switzerland: Springer Cham, 2012.
[2] HUTTO C, GILBERT E. Vader: a parsimonious rule-based model for sentiment analysis of social media text[C]//Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media. Michigan: AAAI, 2014: 216-225.
[3] PARK K, CHA M, RHIM E. Positivity bias in customer satisfaction ratings[C]//WWW’18: Companion Proceedings of the The Web Conference 2018. New York: ACM, 2018: 631-638.
[4] YOUNG L, SOROKA S. Affective news: the automated coding of sentiment in political texts[J]. Political communication, 2012, 29(2): 205–231.
[5] O’CONNOR B, BALASUBRAMANYAN R, ROUTLEDGE B R, et al. From tweets to polls: linking text sentiment to public opinion time series[C]//Fourth International AAAI Conference on Weblogs and Social Media. Washington: AAAI, 2010: 122-129.
[6] CERON A, CURINI L, IACUS S M, et al. Every tweet counts? How sentiment analysis of social media can improve our knowledge of citizens’ political preferences with an application to Italy and France[J]. New media & society, 2014, 16(2): 340–358.
[7] UNANKARD S, LI Xue, SHARAF M, et al. Predicting elections from social networks based on sub-event detection and sentiment analysis[M]//Web Information Systems Engineering-WISE 2014. Cham: Springer International Publishing, 2014: 1-16.
[8] JOO J, LI Weixin, STEEN F F, et al. Visual persuasion: inferring communicative intents of images[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 216-223.
[9] XUE Di, HONG Zheng, GUO Shize, et al. Personality recognition on social media with label distribution learning[J]. IEEE access, 2017, 5: 13478–13488.
[10] CHEN Danni, PARK K, JOO J. Understanding gender stereotypes and electoral success from visual self-presentations of politicians in social media[C]// Joint Workshop on Aesthetic and Technical Quality Assessment of Multimedia and Media Analytics for Societal Trends. New York: ACM, 2020: 21-25.
[11] 蔡庆平, 马海群. 基于Word2Vec和CNN的产品评论细粒度情感分析模型[J]. 图书情报工作, 2020, 64(6): 49–58
CAI Qingping, MA Haiqun. A fine-grained sentiment analysis model for product reviews based on Word2Vec and CNN[J]. Library and information service, 2020, 64(6): 49–58
[12] 张宜浩, 朱小飞, 徐传运, 等. 基于用户评论的深度情感分析和多视图协同融合的混合推荐方法[J]. 计算机学报, 2019, 42(6): 1316–1333
ZHANG Yihao, ZHU Xiaofei, XU Chuanyun, et al. Hybrid recommendation approach based on deep sentiment analysis of user reviews and multi-view collaborative fusion[J]. Chinese journal of computers, 2019, 42(6): 1316–1333
[13] VAN HEE C, LEFEVER E, HOSTE V. SemEval-2018 task 3: irony detection in English tweets[C]//Proceedings of The 12th International Workshop on Semantic Evaluation. New Orleans: ACM, 2018: 39-50.
[14] ZAMPIERI M, MALMASI S, NAKOV P, et al. Predicting the type and target of offensive posts in social media[C]//Proceedings of the 2019 Conference of the North. Minneapolis: ACL, 2019: 1415–1420.
[15] CHE Wanxiang, ZHAO Yanyan, GUO Honglei, et al. Sentence compression for aspect-based sentiment analysis[J]. IEEE-ACM transactions on audio speech and language processing, 2015, 23(12): 2111–2124.
[16] CORTIS K, FREITAS A, DAUDERT T, et al. Semeval-2017 task 5: fine-grained sentiment analysis on financial microblogs and news[C]//Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Vancouver: ACL, 2017: 519-535.
[17] BALAHUR A, STEINBERGER R, KABADJOV M, et al. Sentiment analysis in the news[EB/OL]. (2013-09-24)[ 2021-12-18].https://arxiv.org/abs/1309.6202.
[18] DEVLIN J, CHANG MING-WEI, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018-10-11)[ 2021-12-18].https://arxiv.org/abs/1810.04805.
[19] MA Dehong, LI Sujian, ZHANG Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[EB/OL]. (2017-09-04)[ 2021-12-18].https://arxiv.org/abs/1709.00893.
[20] 翟学明, 魏巍. 混合神经网络和条件随机场相结合的文本情感分析[J]. 智能系统学报, 2021, 16(2): 202–209
ZHAI Xueming, WEI Wei. Text sentiment analysis combining hybrid neural network and conditional random field[J]. CAAI transactions on intelligent systems, 2021, 16(2): 202–209
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Conference and Workshop on Neural Information Processing Systems. Long Beach: MIT, 2017: 5998-6008.
[22] PARK K, PAN ZHUFENG, JOO J. Who blames or endorses whom? entity-to-entity directed sentiment extraction in news text[EB/OL]. (2021-06-02)[2021-12-18].https://arxiv.org/abs/2106.01033.
[23] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[EB/OL]. (2016-07-06)[2021-12-18].https://arxiv.org/abs/1607.01759.
[24] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//NIPS’12: Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 1. New York: ACM, 2012: 1097-1105.
[25] ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. (2014-09-08)[2021-12-18].https://arxiv.org/abs/1409.2329.
[26] LIANG Shuailong, NICOL O, ZHANG Yue. Who blames whom in a crisis? detecting blame ties from news articles using neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 655-662.
[27] LIU Yinhan, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2021-12-18].https://arxiv.org/abs/1907.11692.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems