[1]YANG Xiao,MA Jun,YANG Tong-feng,et al.Automatic multidocument summarization based onthe latent Dirichlet topic allocation model[J].CAAI Transactions on Intelligent Systems,2010,5(2):169-176.
Copy

Automatic multidocument summarization based onthe latent Dirichlet topic allocation model

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 5 Number of periods: 2010 2 Page number: 169-176 Column: 学术论文—自然语言处理与理解 Public date: 2010-04-25
Title:
Automatic multidocument summarization based onthe latent Dirichlet topic allocation model
Author(s):
YANG Xiao1;  MA Jun2;  YANG Tong-feng2;  DU Yan-qi2;  SHAO Hai-min2
1. School of Information Management， Shandong Economic University， Ji’nan 250014， China;
2. School of Computer Science and Technology， Shandong University， Ji’nan 250101, China
Keywords:
multidocument summarization;  sentence scoring;  topic model;  latent dirichlet allocation;  number of topics
CLC:
TP391
DOI:
-
Abstract:
The representative problem of multidocument summarization using probabilistic topic models has begun receiving considerable attention. A multidocument summarization method was proposed based on the latent dirichlet allocation (LDA) model, itself a model representative of probabilistic generative topic models. In this method, the number of topics in the LDA model was determined by model perplexity, and the probabilistic sentence distribution on topics and the probabilistic topic distribution on words were obtained by the Gibbs sampling method. The importance of topics was determined by the sum of topic weights on all sentences. Two sentencescoring methods were proposed, one based on sentence distribution and the other on topic distribution. Evaluated by the recalloriented understudy for gisting evaluation (ROUGE) metrics, results of the both proposed methods surpassed the stateoftheart SumBasic system and the other two LDA based summarization systems for all the ROUGE scores on the DUC2002 generic multidocument summarization test set. 
References:
［1］RADEV D R, HOVY E, MCKEOWN K. Introduction to the special issue on text summarization［J］. Computational Linguistics,2002,28(4):399408.
［2］LEE J H, SUN P, AHN C M, et al. Automatic generic document summarization based on nonnegative matrix factorization［J］. Information Processing and Management, 2009,45(1):2034.
［3］徐永东,徐志明,王晓龙. 基于信息融合的多文档自动文摘技术［J］.计算机学报, 2007,30(11):20482054.
?XU Yongdong, XU Zhiming, WANG Xiaolong. Multidocument automatic summarization technique based on information fusion［J］. Chinese Journal of Computers, 2007, 30(11):20482054.
［4］HIRAO T, ISOZAKI H, MAEDA E, et al. Extracting important sentences with support vector machines［C］//Proc of the 19th International Conference on Computational Linguistics. Taipei, China, 2002: 17.
［5］NENKOVA A, VANDERWENDE L. The impact of frequency on summarization:MSRTR2005101［R］. Redmond, USA: Microsoft Research,2005.
［6］LINC Y, HOVY E. The automated acquisition of topic signatures FOR text summarization［C］//Proc of the 18th International Conference on Computational Linguistics. Sarbrflcken, Germany,2000:271278．
［7］ANTIQUEIRA L, Jr OLIVEIRA O N. A complex network approach to text summarization［J］. Information Science, 2009 (179):584599.
［8］WAN X J, YANG J W. Multidocument summarization using clusterbased link analysis［C］//Proc of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Sheffield, UK, 2008:299306.
［9］HARABAGIU S, HICKL A, LACATUSU F. Satisfying information needs with multidocument summaries［J］. Information Processing and Management, 2007,43(6):16191642.
［10］BLEI D M, NG A Y, JORDAN M I. Latent Dirichlet allocation［J］. Journal of Machine Learning Research, 2003 (3):9931022.
［11］HAGHIGHI A, VANDERWENDE L. Exploring content models for multidocument summarization［C］//Human Language Technologies: the Annual Conference of the North American Chapter of the ACL Boulder. Colorado, 2009:362370.
［12］ARORA R, RAVINDRAN B. Latent Dirichlet allocation based multidocument summarization［C］//Proc of the Second Workshop on Analytics for Noisy Unstructured Text data. Singapore, 2008:9197.
［13］ARORA R, RAVINDRAN B. Latent Dirichlet allocation and singular value decomposition based multidocument summarization［C］//Proc of Eighth IEEE International Conference on Data Mining. Pisa, Italy, 2008：713718.
［14］CHEN Y T, CHEN B, WANG H M. A probabilistic generative framework for extractive broadcast news speech summarization［J］. IEEE Trans on Audio, Speech, and Language Processing,2009,17(1): 95106.
［15］SHAFIEI M M, MILIOS E E. Latent Dirichlet coclustering［C］//Proceedings of the Sixth International Conference on Data Mining (ICDM). Hong Kong, China, 2006: 542551.
［16］CHANG Y L, CHIEN J T. Latent Dirichlet learning for document summarization［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Taipei, China, 2009:16891692.
［17］LIN C Y. ROUGE: a package for automatic evaluation of summaries［C］//Workshop on Text Summarization Branches Out.［S.l.］，Spain, 2004: 7481.
［18］STEYVERS M, GRIFFITHS T. Probabilistic topic models［C］//Handbook of Latent Semantic Analysis. Laurence Erlbaum, 2007：115.
［19］TEH Y W, JORDAN M I, BEAL M J, et al. Hierarchical Dirichlet processes［J］. Journal of the American Statistical Association, 2006, 101(476):15661581.
?［20］秦 兵, 刘 挺, 李 生. 基于局部主题判定与抽取的多文档文摘技术［J］. 自动化学报,2004,30(6):905910. QIN Bing, LIU Ting, LI Sheng. Multidocument summarization based on local topics identification and extraction［J］. Acta Automatica Sinica, 2004，30(6): 905910.
［21］石 晶, 胡 明, 石 鑫, 等. 基于LDA模型的文本分割［J］.计算机学报, 2008.31(10):18651873. SHI Jing, HU Ming, SHI Xin, et al. Text segmentation based on model LDA［J］. Chinese Journal of Computers, 2008，31(10):18651873.
Similar References:

Memo

-

Last Update: 2010-05-24

Copyright © CAAI Transactions on Intelligent Systems