[1]SUN Qingmei,JIN Cong.Image annotation method based on visual attention mechanism and conditional random field[J].CAAI Transactions on Intelligent Systems,2016,11(4):442-448.[doi:10.11992/tis.201606004]
Copy

Image annotation method based on visual attention mechanism and conditional random field

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 11 Number of periods: 2016 4 Page number: 442-448 Column: 学术论文—机器学习 Public date: 2016-07-25
Title:
Image annotation method based on visual attention mechanism and conditional random field
Author(s):
SUN Qingmei;  JIN Cong
School of Computer, Central China Normal University, Wuhan 430079, China
Keywords:
automatic image annotation; visual attention mechanism; inter-word correlation; conditional random fields
CLC:
TP391
DOI:
10.11992/tis.201606004
Abstract:
Traditional image annotation methods interpret all image regions equally, neglecting any understanding of the image. Therefore, an image annotation method based on the visual attention mechanism and conditional random field, called VAMCRF, is proposed. Firstly, people pay more attention to image salient regions during the process of image recognition; this can be achieved through the visual attention mechanism and the support vector machine is then used to assign semantic labels. It then labels the non-salient regions using a k-NN clustering algorithm. Finally, as the annotations of salient and non-salient regions are logically related, the ultimate label vector of the image can be corrected and determined by a conditional random field (CRF) model and inter-word correlation. From the values of average precision, average recall, and F1, the experimental results on Corel5k, IAPR TC-12, and ESP Game confirm that the proposed method is efficient compared with traditional annotation methods.
References:
[1] WANG Meng, NI Bingbing, HUA Xiansheng, et al. Assistive tagging:a survey of multimedia tagging with human-computer joint exploration[J]. ACM computing surveys, 2012, 44(4):25.
[2] JIN Cong, JIN Shuwei. Image distance metric learning based on neighborhood sets for automatic image annotation[J]. Journal of visual communication and image representation, 2016, 34:167-175.
[3] DUYGULU P, BARNARD K, DE FREITAS J F G, et al. Object recognition as machine translation:learning a lexicon for a fixed image vocabulary[C]//Proceedings of the 7th European Conference on Computer Vision. Berlin Heidelberg:Springer-Verlag, 2002:97-112.
[4] JEON J, LAVRENKO V, MANMATHA R. Automatic image annotation and retrieval using cross-media relevance models[C]//Proceedings of the 26th annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY, USA:ACM, 2003:119-126.
[5] LOOG M. Semi-supervised linear discriminant analysis through moment-constraint parameter estimation[J]. Pattern recognition letters, 2014, 37:24-31.
[6] FU Hong, CHI Zheru, FENG Dagan. Recognition of attentive objects with a concept association network for image annotation[J]. Pattern recognition, 2010, 43(10):3539-3547.
[7] FAREED M M S, AHMED G, CHUN Qi. Salient region detection through sparse reconstruction and graph-based ranking[J]. Journal of visual communication and image representation, 2015, 32:144-155.
[8] JIA Cong, QI Jinqing, LI Xiaohui, et al. Saliency detection via a unified generative and discriminative model[J]. Neurocomputing, 2016, 173:406-417.
[9] KHANDOKER A H, PALANISWAMI M, KARMAKAR C K. Support vector machines for automated recognition of obstructive sleep apnea syndrome from ECG recordings[J]. IEEE transactions on information technology in biomedicine, 2009, 13(1):37-48.
[10] PRUTEANU-MALINICI I, MAJOROS W H, OHLER U. Automated annotation of gene expression image sequences via non-parametric factor analysis and conditional random fields[J]. Bioinformatics, 2013, 29(13):i27-i35.
[11] VERMA Y, JAWAHAR C V. Image annotation using metric learning in semantic neighbourhoods[C]//Proceedings of the 12th European Conference on Computer Vision. Berlin Heidelberg:Springer, 2012:836-849.
[12] NAKAYAMA H. Linear distance metric learning for large-scale generic image recognition[D]. Tokyo, Japan:The University of Tokyo, 2011.
[13] FENG S L, MANMATHA R, LAVRENKO V. Multiple Bernoulli relevance models for image and video annotation[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA:IEEE, 2004, 2:Ⅱ-1002-Ⅱ-1009.
[14] MAKADIA A, PAVLOVIC V, KUMAR S. A new baseline for image annotation[C]//Proceedings of the European Conference on Computer Vision. Berlin Heidelberg:Springer-Verlag, 2008:316-329.
[15] GUILLAUMIN M, MENSINK T, VERBEEK J, et al. TagProp:discriminative metric learning in nearest neighbor models for image auto-annotation[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Kyoto:IEEE, 2009:309-316.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems