[1]张铭泉,周辉,曹锦纲.基于注意力机制的双BERT有向情感文本分类研究[J].智能系统学报,2022,17(6):1220-1227.[doi:10.11992/tis.202112038]
ZHANG Mingquan,ZHOU Hui,CAO Jingang.Dual BERT directed sentiment text classification based on attention mechanism[J].CAAI Transactions on Intelligent Systems,2022,17(6):1220-1227.[doi:10.11992/tis.202112038]
点击复制
ZHANG Mingquan,ZHOU Hui,CAO Jingang.Dual BERT directed sentiment text classification based on attention mechanism[J].CAAI Transactions on Intelligent Systems,2022,17(6):1220-1227.[doi:10.11992/tis.202112038]
基于注意力机制的双BERT有向情感文本分类研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第6期
页码:
1220-1227
栏目:
学术论文—自然语言处理与理解
出版日期:
2022-11-05
- Title:
- Dual BERT directed sentiment text classification based on attention mechanism
- Keywords:
- sentiment analysis; bidirectional encoder representation from transform neural network; pretraining model; attention mechanism; deep learning; machine learning; text classification; neural network
- 分类号:
- TP391
- 文献标志码:
- 2022-08-22
- 摘要:
- 在计算社会科学中，理解政治新闻文本中不同政治实体间的情感关系是文本分类领域一项新的研究内容。传统的情感分析方法没有考虑实体之间情感表达的方向，不适用于政治新闻文本领域。针对这一问题，本文提出了一种基于注意力机制的双变换神经网络的双向编码表示(bi-directional encoder representations from transformers, BERT)有向情感文本分类模型。该模型由输入模块、情感分析模块、政治实体方向模块和分类模块四部分组成。情感分析模块和政治实体方向模块具有相同结构，都先采用BERT预训练模型对输入信息进行词嵌入，再采用三层神经网络分别提取实体之间的情感信息和情感方向信息，最后使用注意力机制将两种信息融合，实现对政治新闻文本的分类。在相关数据集上进行实验，结果表明该模型优于现有模型。
- Abstract:
- Understanding the emotional relationships between different political entities in political news texts is a new research topic in the text classification field in computational social science. Traditional methods of sentiment analysis cannot be applied to political news texts because they do not consider the direction of emotional expression between entities. This study proposes a dual BERT-directed sentiment text classification model based on the attention mechanism, which consists of four modules: input module, sentiment analysis module, political entity direction module, and classification module. The structure of the sentiment analysis module and the political entity direction module are identical. Both employ the BERT pretraining model to embed the input information, a three-layer neural network to extract the emotional information or emotional direction information between entities, and an attention mechanism to combine these two kinds of information to classify political news texts. Experiments on comparable data sets show that the model outperforms existing models.
- 参考文献/References:
-
[1] BING Liu. Sentiment analysis and opinion mining[M]. Switzerland: Springer Cham, 2012.
[2] HUTTO C, GILBERT E. Vader: a parsimonious rule-based model for sentiment analysis of social media text[C]//Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media. Michigan: AAAI, 2014: 216-225.
[3] PARK K, CHA M, RHIM E. Positivity bias in customer satisfaction ratings[C]//WWW’18: Companion Proceedings of the The Web Conference 2018. New York: ACM, 2018: 631-638.
[4] YOUNG L, SOROKA S. Affective news: the automated coding of sentiment in political texts[J]. Political communication, 2012, 29(2): 205–231.
[5] O’CONNOR B, BALASUBRAMANYAN R, ROUTLEDGE B R, et al. From tweets to polls: linking text sentiment to public opinion time series[C]//Fourth International AAAI Conference on Weblogs and Social Media. Washington: AAAI, 2010: 122-129.
[6] CERON A, CURINI L, IACUS S M, et al. Every tweet counts? How sentiment analysis of social media can improve our knowledge of citizens’ political preferences with an application to Italy and France[J]. New media & society, 2014, 16(2): 340–358.
[7] UNANKARD S, LI Xue, SHARAF M, et al. Predicting elections from social networks based on sub-event detection and sentiment analysis[M]//Web Information Systems Engineering-WISE 2014. Cham: Springer International Publishing, 2014: 1-16.
[8] JOO J, LI Weixin, STEEN F F, et al. Visual persuasion: inferring communicative intents of images[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 216-223.
[9] XUE Di, HONG Zheng, GUO Shize, et al. Personality recognition on social media with label distribution learning[J]. IEEE access, 2017, 5: 13478–13488.
[10] CHEN Danni, PARK K, JOO J. Understanding gender stereotypes and electoral success from visual self-presentations of politicians in social media[C]// Joint Workshop on Aesthetic and Technical Quality Assessment of Multimedia and Media Analytics for Societal Trends. New York: ACM, 2020: 21-25.
[11] 蔡庆平, 马海群. 基于Word2Vec和CNN的产品评论细粒度情感分析模型[J]. 图书情报工作, 2020, 64(6): 49–58
CAI Qingping, MA Haiqun. A fine-grained sentiment analysis model for product reviews based on Word2Vec and CNN[J]. Library and information service, 2020, 64(6): 49–58
[12] 张宜浩, 朱小飞, 徐传运, 等. 基于用户评论的深度情感分析和多视图协同融合的混合推荐方法[J]. 计算机学报, 2019, 42(6): 1316–1333
ZHANG Yihao, ZHU Xiaofei, XU Chuanyun, et al. Hybrid recommendation approach based on deep sentiment analysis of user reviews and multi-view collaborative fusion[J]. Chinese journal of computers, 2019, 42(6): 1316–1333
[13] VAN HEE C, LEFEVER E, HOSTE V. SemEval-2018 task 3: irony detection in English tweets[C]//Proceedings of The 12th International Workshop on Semantic Evaluation. New Orleans: ACM, 2018: 39-50.
[14] ZAMPIERI M, MALMASI S, NAKOV P, et al. Predicting the type and target of offensive posts in social media[C]//Proceedings of the 2019 Conference of the North. Minneapolis: ACL, 2019: 1415–1420.
[15] CHE Wanxiang, ZHAO Yanyan, GUO Honglei, et al. Sentence compression for aspect-based sentiment analysis[J]. IEEE-ACM transactions on audio speech and language processing, 2015, 23(12): 2111–2124.
[16] CORTIS K, FREITAS A, DAUDERT T, et al. Semeval-2017 task 5: fine-grained sentiment analysis on financial microblogs and news[C]//Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Vancouver: ACL, 2017: 519-535.
[17] BALAHUR A, STEINBERGER R, KABADJOV M, et al. Sentiment analysis in the news[EB/OL]. (2013-09-24)[ 2021-12-18].https://arxiv.org/abs/1309.6202.
[18] DEVLIN J, CHANG MING-WEI, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018-10-11)[ 2021-12-18].https://arxiv.org/abs/1810.04805.
[19] MA Dehong, LI Sujian, ZHANG Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[EB/OL]. (2017-09-04)[ 2021-12-18].https://arxiv.org/abs/1709.00893.
[20] 翟学明, 魏巍. 混合神经网络和条件随机场相结合的文本情感分析[J]. 智能系统学报, 2021, 16(2): 202–209
ZHAI Xueming, WEI Wei. Text sentiment analysis combining hybrid neural network and conditional random field[J]. CAAI transactions on intelligent systems, 2021, 16(2): 202–209
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Conference and Workshop on Neural Information Processing Systems. Long Beach: MIT, 2017: 5998-6008.
[22] PARK K, PAN ZHUFENG, JOO J. Who blames or endorses whom? entity-to-entity directed sentiment extraction in news text[EB/OL]. (2021-06-02)[2021-12-18].https://arxiv.org/abs/2106.01033.
[23] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[EB/OL]. (2016-07-06)[2021-12-18].https://arxiv.org/abs/1607.01759.
[24] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//NIPS’12: Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 1. New York: ACM, 2012: 1097-1105.
[25] ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. (2014-09-08)[2021-12-18].https://arxiv.org/abs/1409.2329.
[26] LIANG Shuailong, NICOL O, ZHANG Yue. Who blames whom in a crisis? detecting blame ties from news articles using neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 655-662.
[27] LIU Yinhan, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2021-12-18].https://arxiv.org/abs/1907.11692.
- 相似文献/References:
-
[1]赵文清,侯小可,沙海虹.语义规则在微博热点话题情感分析中的应用[J].智能系统学报,2014,9(1):121.[doi:10.3969/j.issn.1673-4785.201208020]
ZHAO Wenqing,HOU Xiaoke,SHA Haihong.Application of semantic rules to sentiment analysis of microblog hot topics[J].CAAI Transactions on Intelligent Systems,2014,9():121.[doi:10.3969/j.issn.1673-4785.201208020]
[2]吴钟强,张耀文,商琳.基于语义特征的多视图情感分类方法[J].智能系统学报,2017,12(5):745.[doi:10.11992/tis.201706026]
WU Zhongqiang,ZHANG Yaowen,SHANG Lin.Multi-view sentiment classification of microblogs based on semantic features[J].CAAI Transactions on Intelligent Systems,2017,12():745.[doi:10.11992/tis.201706026]
[3]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15():460.[doi:10.11992/tis.201812017]
[4]肖宇晗,林慧苹,汪权彬,等.基于双特征嵌套注意力的方面词情感分析算法[J].智能系统学报,2021,16(1):142.[doi:10.11992/tis.202012024]
XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16():142.[doi:10.11992/tis.202012024]
[5]胡文彬,陈龙,黄贤波,等.融合交叉注意力的突发事件多模态中文反讽识别模型[J].智能系统学报,2024,19(2):392.[doi:10.11992/tis.202212011]
HU Wenbin,CHEN Long,HUANG Xianbo,et al.A multimodal Chinese sarcasm detection model for emergencies based on cross attention[J].CAAI Transactions on Intelligent Systems,2024,19():392.[doi:10.11992/tis.202212011]
备注/Memo
收稿日期:2021-12-18。
基金项目:中央高校基本科研业务费专项资金项目（2021MS092）.
作者简介:张铭泉，副教授，博士，主要研究方向为计算机组成、机器学习、模式识别。发表学术论文20篇;周辉，硕士研究生，主要研究方向为深度学习和自然语言处理;曹锦纲，博士研究生，讲师，主要研究方向为图像处理和模式识别。发表学术论文10篇
通讯作者:曹锦纲.E-mail:caojg168@126.com
更新日期/Last Update:
1900-01-01