[1]肖融,孔亮,张岩.基于文本的新闻事件多版本发现模型[J].智能系统学报,2012,7(4):307-314.
XIAO Rong,KONG Liang,ZHANG Yan.A text clustering model for diverse versions discovery[J].CAAI Transactions on Intelligent Systems,2012,7(4):307-314.
点击复制
XIAO Rong,KONG Liang,ZHANG Yan.A text clustering model for diverse versions discovery[J].CAAI Transactions on Intelligent Systems,2012,7(4):307-314.
基于文本的新闻事件多版本发现模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
7
期数:
2012年第4期
页码:
307-314
栏目:
学术论文—自然语言处理与理解
出版日期:
2012-08-25
- Title:
- A text clustering model for diverse versions discovery
- 文章编号:
- 1673-4785(2012)04-0307-08
- 分类号:
- TP18
- 文献标志码:
- A
- 摘要:
- 信息时代的发展让越来越多的新闻事件充斥人们的生活，对于一件特定的新闻事件，目前已有很多算法可以帮助人们进行事件追踪和发现.提出一种CDW算法，帮助读者对于一件具有多个版本描述的新闻事件进行多个不同版本的发现.这个算法将文档集映射到话题层，通过提取每个话题的流行词，以得到文档集中具有高区分度的特征.然后根据这些特征对文档集进行聚类，最后得到事件的多个版本.通过在2个实际数据集上进行实验，实验结果表明，该算法与以往的相关算法相比是十分有效的.
- Abstract:
- The development of information technology brings numerous news and events to our daily life. Although previous researches have provided various algorithms to detect and track events, few of them focus on uncovering the diversified versions of an event. In this paper, a novel algorithm CDW which is capable of discovering different versions of one event according to the news reports was proposed. First, documents were mapped to the topic layer to get the information of each topic. Then the highlydifferentiated words of each topic were extracted to cluster the documents. At last, various versions of one event were got. Experiments conducted on two data sets show that the algorithm given in this paper is effective and outperforms various related algorithms, including classical methods such as Kmeans and linear discriminant analysis (LDA).
- 参考文献/References:
-
［1］ALLAN J. Topic detection and tracking: eventbased information organization［M］. Boston: Kluwer Academic Publishers, 2002: 12411253.
［2］HE T T, QU G Z, LI S W, et al. Semiautomatic hot event detection［C］//Lecture Notes in Computer Science.Hongkong, China, 2006： 1008.
［3］YU M Q, LUO W H, XU H B, et al. Research on hierarchical topic detection in topic detection and tracking［J］. Journal of Computer Research and Development, 2006, 43(3): 489495.
［4］邱立坤，龙志祎，钟华，等. 层次化话题发现与跟踪方法及系统实现［J］. 广西师范大学学报：自然科学版, 2007, 25(2): 157160.
QIU Likun, LONG Zhiyi, ZHONG Hua, et al. Hierarchical topic detection and tracking and implementation of system［J］. Journal of Guangxi Normal University: Natural Science Edition, 2007, 25(2): 157160.
［5］CARTHY J. Lexical chains versus keywords for topic tracking［C］//Proceedings of the 5th International Conference on Intelligent Text Processing and Computational Linguistics. Seoul, Korea, 2004: 507510.
［6］ALLAN J, CARBONELL J, DODDINGTON G, et al. Topic detection and tracking pilot study final report［C］//Proceedings of the DARPA Broadcasting News Transcript and Understanding Workshop. ［S.l.］, 1998: 194218.
［7］YANG Y, PIERCE T, CARBONELL J. A study of retrospective and online event detection［C］//Special Interest Group on Information Retrieval’98. Melbourne, Australia, 1998: 2836.
［8］ALLAN J, PAPKA R, LAVRENKO V. Online new event detection and tracking［C］//Special Interest Group on Information Retrieval’98. Melbourne, Australia, 1998: 3745.
［9］BRANTS T, CHEN F, FARAHAT A. A system for new event detection［C］//Special Interest Group on Information Retrieval’03. Toronto, Canada, 2003: 330337.
［10］NALLAPATI R, FENG A, PENG F, et al. Event threading within news topics［C］// International Conference on Information and Knowledge Management. Washington, DC, USA, 2004: 446453.
［11］STEINBACH M, KARYPIS G, KUMAR V. A comparison of document clustering techniques［EB/OL］. ［20110514］.http://www.cs.cmu.edu/~dunja/KDDpapers/Steinbach_IR.pdf.
［12］PAUL S B, USAMA M F. Refining initial points for Kmeans clustering［C］//Proceedings of the Fifteenth International Conference on Machine Learning. San Francisco, USA, 1998: 9199.
［13］JAIN A K, MURTY M N, FLYNN P J. Data clustering: a review［J］. ACM Computing Surveys, 1999, 31(3): 264333.
［14］RYMOND T, HAN J W. Efficient and effective clustering methods for spatial data mining［C］//Proceedings of the 20th International Conference on Very Large Data Bases. Hong Kong, China, 1994: 144155.
［15］KONG L, YAN R, HE Y J, et al. DVD: a model for event diversified versions discovery［C］//AsiaPacific Web Conference’11. Beijing, China, 2011: 1820.
［16］FLAKE G W, LAWRENCE S, GILES C L. Efficient identification of Web communities［C］//International Conference on Knowledge Discovery and Data Mining’00. Boston, USA, 2000: 160169.
［17］ROCCHIO J. Relevance feedback in information retrieval［C］//The SMART Retrieval System: Experiments in Automatic Document Processing. Englewood Cliffs, USA, 1971: 313323.
［18］DASGUPTA S, NG V. Towards subjectifying text clustering［C］// Special Inspector General for Iraq Reconstruction’10. Geneva, Switzerland, 2010: 483490.
［19］DUMAIS S T, PLATT J, HECKERMAN D, et al. Inductive learning algorithms and representations for text categorization［C］// Proceedings of the Seventh Internat ional Conference on Information and Knowledge Management. New York, USA, 1998: 148155.
［20］FRANZ M, WARD T, MCCARLEY J S, et al. Unsupervised and supervised clustering for topic tracking［C］//Special Inspector General for Iraq Reconstruction’01. New Orlean, USA, 2001: 310317.
［21］BLEI D M, ANDREW Y NG, MICHAEL I J. Latent Dirichlet allocation［J］. The Journal of Machine Learning Research, 2003(3): 9931022.
［22］WEI X, CROFT W B. LDAbased document models for adhoc retrieval［C］//Proceedings of the 29th Special Inspector General for Iraq Reconstruction Conference. New York, USA, 2006: 178185.
［23］BHATTACHARYA I, GETOOR I. A latent Dirichlet model for unsupervised entity resolution［C］//SIAM International Conference on Data Mining. Bethesda, USA, 2006: 4758.
［24］JEROME R B. A novel word clustering algorithm based on latent semantic analysis［C］//Acoustics, Speech, and Signal Processing 1996. ［S.l.］, 1996: 172175.
［25］SUN B, SHI L, KONG L, et al. Describing web topics meticulously through word graph analysis［C］//The IEEE Conference on Instructional Technologies ’09. Xiamen, China, 2009: 1114.
［26］PAGE L, BRIN S, MOTWANI R, et al. The pagerank citation ranking: bringing order to the web［C］//Proceedings of the 7th International World Wide Web Conference. Brisbane, Australia, 1998: 161172.
［27］KAREN J S. A statistical interpretation of term specificity and its application in retrieval［J］. Journal of Documentation, 1972, 28(1): 1121.
［28］HARTIGANJ A, WONG M A. A Kmeans clustering algorithm［J］. Journal of the Royal Statistical Society, Series C: Applied Statistics, 1979, 28(1): 100108.
［29］PELLEG D, MOORE A W. Xmeans: extending Kmeans with efficient estimation of the number of clusters［C］//Proceedings of the Seventeenth International Conference on Machine Learning. Stanford, USA, 2000: 727734.
［30］MACQUEEN J B. Some methods for classification and analysis of multivariate observations［C］//Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: University of California Press, 1967: 281297.
备注/Memo
收稿日期： 2011-11-24.
基金项目：国家自然科学基金资助项目(61703081).
通信作者：肖融.
E-mail:xrsmile@gmail.com.
作者简介：
肖融，女，1989年生，硕士研究生，主要研究方向为数据挖掘、信息检索等.
孔亮，男，1985年生，硕士研究生，主要研究方向为数据挖掘、信息检索等.
张岩，男，1970年生，副教授，博士，主要研究方向为web信息处理、智能搜索技术、文本分析与数据挖掘、数据库性能等，发表学术论文多篇.
更新日期/Last Update:
2012-09-26