[1]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460-467.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15(3):460-467.[doi:10.11992/tis.201812017]
点击复制
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15(3):460-467.[doi:10.11992/tis.201812017]
层次化双注意力神经网络模型的情感分析研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
15
期数:
2020年第3期
页码:
460-467
栏目:
学术论文—自然语言处理与理解
出版日期:
2020-05-05
- Title:
- Hierarchical double-attention neural networks for sentiment classification
- Keywords:
- sentiment analysis; attention mechanism; convolutional neural network (CNN); sentiment classification; recurrent neural network (RNN); word vector; deep learning; feature selection
- 分类号:
- TP391
- 摘要:
- 在篇章级的情感分类中由于篇章级文本较长，特征提取较普通句子级分析相对较难，大多方法使用层次化的模型进行篇章文本的情感分析，但目前的层次化模型多以循环神经网络和注意力机制为主，单一的循环神经网络结构提取的特征不够明显。本文针对篇章级的情感分类任务，提出一种层次化双注意力神经网络模型。首先对卷积神经网络进行改进，构建词注意力卷积神经网络。然后模型从两个层次依次提取篇章特征，第一层次使注意力卷积神经网络发现每个句子中的重要词汇，提取句子的词特征，构建句子特征向量；第二层次以循环神经网络获取整个篇章的语义表示，全局注意力机制发现篇章中每个句子的重要性，分配以不同的权重，最后构建篇章的整体语义表示。在IMDB、YELP 2013、YELP 2014数据集上的实验表明，模型较当前最好的模型更具优越性。
- Abstract:
- In sentiment classification, feature extraction in the document level is more difficult than the analysis in the common sentence level because of the length of the text. Most methods apply a hierarchical model to the sentiment analysis of text in the document level. However, most existing hierarchical methods mainly focus on a recurrent neural network (RNN) and attention mechanism, and the feature extracted by a single RNN is unclear. To solve the sentiment classification problem in the document level, we propose a hierarchical double-attention neural network model. In the first step, we improve a convolutional neural network (CNN), construct a word attention CNN, and then extract the features of the chapter from two levels. In the first level, the attention CNN can identify important words and phrases from every sentence, extract the word feature of the sentence, and construct the feature vector of the sentence. In the second level, the semantic meaning of the document is derived by the RNN. The global attention mechanism can find the importance of every sentence in the document, attribute different weights to them, and construct the whole semantic representation of the document. The experiment results on IMDB, YELP 2013, and YELP 2014 datasets show that our model achieves a more significant improvement than all state-of-the-art methods.
- 参考文献/References:
-
[1] PANG Bo, LEE L, VAITHYANATHAN S. Thumbs up?: sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Stroudsburg, USA, 2002: 79-86.
[2] LU Yue, CASTELLANOS M, DAYAL U, et al. Automatic construction of a context-aware sentiment lexicon: an optimization approach[C]//Proceedings of the 20th International Conference on World Wide Web. Hyderabad, India, 2011: 347-356.
[3] WANG Sida, MANNING C D. Baselines and bigrams: simple, good sentiment and topic classification[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers. Jeju Island, Korea, 2012: 90-94.
[4] KIRITCHENKO S, ZHU Xiaodan, MOHAMMAD S M. Sentiment analysis of short informal texts[J]. Journal of artificial intelligence research, 2014, 50: 723-762.
[5] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]// Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, USA, 2016: 260-270.
[6] SHEN Dinghan, MIN M R, LI Yitong, et al. Adaptive convolutional filter generation for natural language understanding. [J]. arXiv: 1709.08294, 2017.
[7] WANG Shuohang, JIANG Jing. Machine comprehension using match-LSTM and answer pointer[C]//Proceedings of International Conference on Learning Representations. Toulon, France, 2017: 1-15.
[8] WANG Wenhui, YANG Nan, WEI Furu, et al. Gated self-matching networks for reading comprehension and question answering[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2017: 189-198.
[9] KUMAR A, IRSOY O, ONDRUSKA P, et al. Ask me anything: dynamic memory networks for natural language processing[C]//Proceedings of the 33rd International Conference on Machine Learning. New York, USA, 2016: 1378-1387.
[10] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar, 2014: 1746-1751.
[11] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P, et al. A convolutional neural network for modelling sentences[C]//Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, USA, 2014: 655-665.
[12] ZHANG Xiang, ZHAO Junbo, LECUN Y. Character-level convolutional networks for text classification[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015: 649-657.
[13] TANG Duyu, QIN Bing, LIU Ting. Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 1422-1432.
[14] YANG Zichao, YANG Diyi, DYER C, et al. Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, USA, 2016: 1480-1489.
[15] ADI Y, KERMANY E, BELINKOV Y, et al. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks[C]//Proceedings of International Conference on Learning Representations. Toulon, France, 2017: 1608-1622.
[16] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st Conference on Neural Information Processing Systems. Long Beach, USA, 2017: 5998-6008.
[17] WAN Xiaojun. Co-training for cross-lingual sentiment classification[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Suntec, Singapore, 2009: 235-243.
[18] ZAGIBALOV T, CARROLL J. Automatic seed word selection for unsupervised sentiment classification of Chinese text[C]//Proceedings of the 22nd International Conference on Computational Linguistics. Manchester, United Kingdom, 2008: 1073-1080.
[19] LIU Jiangming, ZHANG Yue. Attention modeling for targeted sentiment[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, Spain, 2017: 572-577.
[20] ZHANG Lei, WANG Shuai, LIU Bing. Deep learning for sentiment analysis: a survey[J]. WIREs data mining and knowledge discovery, 2018, 8(4): e1253.
[21] CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017: 452-461.
[22] TANG Duyu, WEI Furu, YANG Nan, et al. Learning sentiment-specific word embedding for twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, USA, 2014: 1555-1565.
[23] JOHNSON R, ZHANG Tong. Effective use of word order for text categorization with convolutional neural networks[C]//Proceedings of 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Denver, USA, 2015: 103-112.
[24] SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment Treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, USA, 2013: 1631-1642.
[25] XU K, BA J, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille, France, 2015: 2048-2057.
[26] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations, 2014. San Diego, USA, 2015: 473-488.
[27] LUONG T, PHAM H, MANNING C D, et al. Effective approaches to attention-based neural machine translation[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 1412-1421.
[28] ZHOU Xinjie, WAN Xiaojun, XIAO Jianguo. Attention-based LSTM network for cross-lingual sentiment classification[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. Austin, USA, 2016: 247-256.
[29] ALLAMANIS M, PENG Hao, SUTTON C A. A convolutional attention network for extreme summarization of source code[C]//Proceedings of 2016 International Conference on Machine Learning. New York, USA, 2016: 2091-2100.
[30] YIN Wenpeng, SCHüTZE H, XIANG Bing, et al. ABCNN: attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the association for computational linguistics, 2016, 4: 259-272.
[31] WANG Linlin, CAO Zhu, DE MELO G, et al. Relation classification via multi-level attention CNNs[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 1298-1307.
[32] CHEN Huimin, SUN Maosong, TU Cunchao, et al. Neural sentiment classification with user and product attention [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, USA, 2016: 1650-1659.
[33] FAN Rongen, CHANG Kaiwei, HSIEH C J, et al. LIBLINEAR: a library for large linear classification[J]. Journal of machine learning research, 2008, 9: 1871-1874.
[34] LE Q, MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on Machine Learning. Beijing, China, 2014: 1188-1196.
- 相似文献/References:
-
[1]赵文清,侯小可,沙海虹.语义规则在微博热点话题情感分析中的应用[J].智能系统学报,2014,9(1):121.[doi:10.3969/j.issn.1673-4785.201208020]
ZHAO Wenqing,HOU Xiaoke,SHA Haihong.Application of semantic rules to sentiment analysis of microblog hot topics[J].CAAI Transactions on Intelligent Systems,2014,9():121.[doi:10.3969/j.issn.1673-4785.201208020]
[2]吴钟强,张耀文,商琳.基于语义特征的多视图情感分类方法[J].智能系统学报,2017,12(5):745.[doi:10.11992/tis.201706026]
WU Zhongqiang,ZHANG Yaowen,SHANG Lin.Multi-view sentiment classification of microblogs based on semantic features[J].CAAI Transactions on Intelligent Systems,2017,12():745.[doi:10.11992/tis.201706026]
[3]申凯,王晓峰,杨亚东.基于双向消息链路卷积网络的显著性物体检测[J].智能系统学报,2019,14(6):1152.[doi:10.11992/tis.201812003]
SHEN Kai,WANG Xiaofeng,YANG Yadong.Salient object detection based on bidirectional message link convolution neural network[J].CAAI Transactions on Intelligent Systems,2019,14():1152.[doi:10.11992/tis.201812003]
[4]赵文清,程幸福,赵振兵,等.注意力机制和Faster RCNN相结合的绝缘子识别[J].智能系统学报,2020,15(1):92.[doi:10.11992/tis.201907023]
ZHAO Wenqing,CHENG Xingfu,ZHAO Zhenbing,et al.Insulator recognition based on attention mechanism and Faster RCNN[J].CAAI Transactions on Intelligent Systems,2020,15():92.[doi:10.11992/tis.201907023]
[5]申翔翔,侯新文,尹传环.深度强化学习中状态注意力机制的研究[J].智能系统学报,2020,15(2):317.[doi:10.11992/tis.201809033]
SHEN Xiangxiang,HOU Xinwen,YIN Chuanhuan.State attention in deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2020,15():317.[doi:10.11992/tis.201809033]
[6]莫宏伟,田朋.基于注意力融合的图像描述生成方法[J].智能系统学报,2020,15(4):740.[doi:10.11992/tis.201910039]
MO Hongwei,TIAN Peng.An image caption generation method based on attention fusion[J].CAAI Transactions on Intelligent Systems,2020,15():740.[doi:10.11992/tis.201910039]
[7]肖宇晗,林慧苹,汪权彬,等.基于双特征嵌套注意力的方面词情感分析算法[J].智能系统学报,2021,16(1):142.[doi:10.11992/tis.202012024]
XIAO Yuhan,LIN Huiping,WANG Quanbin,et al.An algorithm for aspect-based sentiment analysis based on dual features attention-over-attention[J].CAAI Transactions on Intelligent Systems,2021,16():142.[doi:10.11992/tis.202012024]
[8]鲍维克,袁春.面向推荐系统的分期序列自注意力网络[J].智能系统学报,2021,16(2):353.[doi:10.11992/tis.202005028]
BAO Weike,YUAN Chun.Recommendation system with long-term and short-term sequential self-attention network[J].CAAI Transactions on Intelligent Systems,2021,16():353.[doi:10.11992/tis.202005028]
[9]洪恺临,曹江涛,姬晓飞.改进Center-Net网络的自主喷涂机器人室内窗户检测[J].智能系统学报,2021,16(3):425.[doi:10.11992/tis.202005016]
HONG Kailin,CAO Jiangtao,JI Xiaofei.Indoor window detection of autonomous spraying robot based on improved CenterNet network[J].CAAI Transactions on Intelligent Systems,2021,16():425.[doi:10.11992/tis.202005016]
[10]张勇,高大林,巩敦卫,等.用于关系抽取的注意力图长短时记忆神经网络[J].智能系统学报,2021,16(3):518.[doi:10.11992/tis.202008036]
ZHANG Yong,GAO Dalin,GONG Dunwei,et al.Attention graph long short-term memory neural network for relation extraction[J].CAAI Transactions on Intelligent Systems,2021,16():518.[doi:10.11992/tis.202008036]
[11]张铭泉,周辉,曹锦纲.基于注意力机制的双BERT有向情感文本分类研究[J].智能系统学报,2022,17(6):1220.[doi:10.11992/tis.202112038]
ZHANG Mingquan,ZHOU Hui,CAO Jingang.Dual BERT directed sentiment text classification based on attention mechanism[J].CAAI Transactions on Intelligent Systems,2022,17():1220.[doi:10.11992/tis.202112038]
[12]胡文彬,陈龙,黄贤波,等.融合交叉注意力的突发事件多模态中文反讽识别模型[J].智能系统学报,2024,19(2):392.[doi:10.11992/tis.202212011]
HU Wenbin,CHEN Long,HUANG Xianbo,et al.A multimodal Chinese sarcasm detection model for emergencies based on cross attention[J].CAAI Transactions on Intelligent Systems,2024,19():392.[doi:10.11992/tis.202212011]
备注/Memo
收稿日期:2018-12-15。
基金项目:国家自然科学基金项目(61772211，61503143)
作者简介:曾碧卿，教授，博士，主要研究方向为认知计算和自然语言处理。获发明专利6项，发表学术论文100余篇，出版学术专著2部;韩旭丽，硕士研究生，主要研究方向为自然语言处理、情感分析。发表学术论文10篇;王盛玉，硕士研究生，主要研究方向为自然语言处理、情感分析。发表学术论文6篇
通讯作者:曾碧卿.E-mail:zengbiqing0528@163.com
更新日期/Last Update:
1900-01-01