基于视觉注意机制和条件随机场的图像标注

PDF下载HTML

Title:

Image annotation method based on visual attention mechanism and conditional random field

作者:

孙庆美, 金聪

华中师范大学 计算机学院, 湖北 武汉 430079

Author(s):

SUN Qingmei, JIN Cong

School of Computer, Central China Normal University, Wuhan 430079, China

关键词:

自动图像标注; 视觉注意; 词相关性; 条件随机场

Keywords:

automatic image annotation; visual attention mechanism; inter-word correlation; conditional random fields

分类号:

TP391

DOI:

10.11992/tis.201606004

摘要:

传统的图像标注方法对图像各个区域同等标注，忽视了人们对图像的理解方式。为此提出了基于视觉注意机制和条件随机场的图像标注方法。首先，由于人们在对图像认识的过程中，对显著区域会有较多的关注，因此通过视觉注意机制来取得图像的显著区域，用支持向量机对显著区域赋予语义标签；再利用k-NN聚类算法对非显著区域进行标注；最后，又由于显著区域的标注词与非显著区域的标注词在逻辑上存在一定的关联性，因此条件随机场模型可以根据标注词的关联性校正并确定图像的最终标注向量。在Corel5k、IAPR TC-12和ESP Game图像库上进行实验并且和其他方法进行比较，从平均查准率、平均查全率和F₁的实验结果验证了本文方法的有效性。

Abstract:

Traditional image annotation methods interpret all image regions equally, neglecting any understanding of the image. Therefore, an image annotation method based on the visual attention mechanism and conditional random field, called VAMCRF, is proposed. Firstly, people pay more attention to image salient regions during the process of image recognition; this can be achieved through the visual attention mechanism and the support vector machine is then used to assign semantic labels. It then labels the non-salient regions using a k-NN clustering algorithm. Finally, as the annotations of salient and non-salient regions are logically related, the ultimate label vector of the image can be corrected and determined by a conditional random field (CRF) model and inter-word correlation. From the values of average precision, average recall, and F1, the experimental results on Corel5k, IAPR TC-12, and ESP Game confirm that the proposed method is efficient compared with traditional annotation methods.