[1]闫涵,张旭秀,张净丹.多感知兴趣区域特征融合的图像识别方法[J].智能系统学报,2021,16(2):263-270.[doi:10.11992/tis.201906032]
YAN Han,ZHANG Xuxiu,ZHANG Jingdan.Image recognition method based on multi-perceptual interest region feature fusion[J].CAAI Transactions on Intelligent Systems,2021,16(2):263-270.[doi:10.11992/tis.201906032]
点击复制
YAN Han,ZHANG Xuxiu,ZHANG Jingdan.Image recognition method based on multi-perceptual interest region feature fusion[J].CAAI Transactions on Intelligent Systems,2021,16(2):263-270.[doi:10.11992/tis.201906032]
多感知兴趣区域特征融合的图像识别方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第2期
页码:
263-270
栏目:
学术论文—机器感知与模式识别
出版日期:
2021-03-05
- Title:
- Image recognition method based on multi-perceptual interest region feature fusion
- Keywords:
- deep learning; image recognition; migration learning; feature fusion; integrated learning; feature extraction; CAM visualization; VGGNet; ResNet
- 分类号:
- TP311
- 摘要:
- 针对自然图像识别过程中不同深度学习模型关注兴趣区域不同的现象，本文引入深度卷积神经网络融合机制，结合深度迁移学习方法，给出了一种基于多感知兴趣区域特征融合的图像识别方法。本文将迁移学习方法引入牛津大学视觉组网络模型(visual geometry group network，VGGNet)和残差网络模型(residual network，ResNet)，通过对单个分类模型进行热力图可视化及特征可视化，得到了不同网络模型关联的特征区域不一样的结论。然后在此基础上分别设计特征拼接、特征融合加特征拼接及融合投票方法将不同模型特征进行融合，得到3种新的融合模型。实验结果表明，本文方法在Kaggle数据集上的识别准确率高于VGG-16、VGG-19、ResNet-50、DenseNet-201模型。
- Abstract:
- This paper presents the deep convolution neural network fusion mechanism and proposes an image recognition method based on multi-perceptual interest region feature fusion in combination with the deep-migration learning method. This is to solve the problem of different deep-learning models used on different interest regions when they recognize a natural image. The migration learning method is applied to the convolution neural net architectures, namely VGG and ResNet networks. Then, through the visualization of the heat map and the features of single classification model, a conclusion is drawn that the characteristic regions associated with different network models are different. Based on this, the methods of feature splicing, feature fusion and splicing, and fusion voting systems are designed to fuse different model features, obtaining three new fusion models. The experimental results show that the recognition accuracy of this method on Kaggle dataset is higher than that of VGG-16, VGG-19, ResNet-50, and DenseNet-201 models.
- 参考文献/References:
-
[1] SPENCER JR B F, HOSKERE V, NARAZAKI Y. Advances in computer vision-based civil infrastructure inspection and monitoring[J]. Engineering, 2019, 5(2):199-222.
[2] CHELLAPPA R. The changing fortunes of pattern recognition and computer vision[J]. Image and vision computing, 2016, 55:3-5.
[3] 雷明. 机器学习与应用[M]. 北京:清华大学出版社, 2019:26-33.
[4] MARY N A B, DHARMA D. Coral reef image classification employing improved LDP for feature extraction[J]. Journal of visual communication and image representation, 2017, 49:225-242.
[5] YU Hua, YANG Jie. A direct LDA algorithm for high-dimensional data-with application to face recognition[J]. Pattern recognition, 2001, 34(10):2067-2070.
[6] 刘丽, 匡纲要. 图像纹理特征提取方法综述[J]. 中国图象图形学报, 2009, 14(4):622-635
LIU Li, KUANG Gangyao. Overview of image textural feature extraction methods[J]. Journal of image and graphics, 2009, 14(4):622-635
[7] 李磊, 董卓丽. 利用改进图割的彩色图像分割算法[J]. 武汉大学学报·信息科学版, 2014, 39(12):1504-1508
LI Lei, DONG Zhuoli. Color image segmentation using improved graph cuts[J]. Geomatics and Information Science of Wuhan University, 2014, 39(12):1504-1508
[8] 弓晓虹, 郑音飞, 秦佳乐, 等. 基于乘性梯度的医学超声图像边缘检测算法[J]. 浙江大学学报(工学版), 2014, 48(10):1871-1878
GONG Xiaohong, ZHENG Yinfei, QIN Jiale, et al. Multiplicative gradient based edge detection method for medical ultrasound image[J]. Journal of Zhejiang University (Engineering Science), 2014, 48(10):1871-1878
[9] 张桂梅, 张松, 储珺. 一种新的基于局部轮廓特征的目标检测方法[J]. 自动化学报, 2014, 40(10):2346-2355
ZHANG Guimei, ZHANG Song, CHU Jun. A new object detection algorithm using local contour features[J]. Acta automatica sinica, 2014, 40(10):2346-2355
[10] YE Qihong, XIANG Ming, CUI Zhendong. Fingerprint image enhancement algorithm based on two dimension EMD and Gabor filter[J]. Procedia engineering, 2012, 29:1840-1844.
[11] 张丽琼, 王炳和. 基于小波变换的脉象信号特征提取方法[J]. 数据采集与处理, 2004, 19(3):323-328
ZHANG Liqiong, WANG Binghe. Feature extraction methods for pulse signal based on wavelet transform[J]. Journal of data acquisition & processing, 2004, 19(3):323-328
[12] YAGHOOBI H, MANSOURI H, FARSANGI M A E, et al. Determining the fragmented rock size distribution using textural feature extraction of images[J]. Powder technology, 2019, 342:630-641.
[13] MESNIL G, BORDES A, WESTON J, et al. Learning semantic representations of objects and their parts[J]. Machine learning, 2014, 94(2):281-301.
[14] 胡越, 罗东阳, 花奎, 等. 关于深度学习的综述与讨论[J]. 智能系统学报, 2019, 14(1):1-19
HU Yue, LUO Dongyang, HUA Kui, et al. Overview on deep learning[J]. CAAI transactions on intelligent systems, 2019, 14(1):1-19
[15] SERMANET P, EIGEN D, ZHANG Xiang, et al. Overfeat:integrated recognition, localization and detection using convolutional networks[J]. Computer science, 2013.
[16] ZEILER M D, KRISHNAN D, TAYLOR G W, et al. Deconvolutional networks[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010:2528-2535.
[17] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[18] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https://arxiv.org/abs/1409.1556, 2014.
[19] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:770-778.
[20] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:2261-2269.
[21] 郑远攀, 李广阳, 李晔. 深度学习在图像识别中的应用研究综述[J]. 计算机工程与应用, 2019, 55(12):20-36
ZHENG Yuanpan, LI Guangyang, LI Ye. Survey of application of deep learning in image recognition[J]. Computer engineering and applications, 2019, 55(12):20-36
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]张瑞成,胡绪磊.基于小世界网络的Hopfield联想记忆模型[J].智能系统学报,2014,9(2):214.[doi:10.3969/j.issn.1673-4785.201211050]
ZHANG Ruicheng,HU Xulei.Research on the hopfield associative memory model based on the small-world network[J].CAAI Transactions on Intelligent Systems,2014,9():214.[doi:10.3969/j.issn.1673-4785.201211050]
[5]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[6]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[7]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[8]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[9]姜英,王延江.REM记忆模型在图像分类识别中的应用[J].智能系统学报,2017,12(3):310.[doi:10.11992/tis.201605010]
JIANG Ying,WANG Yanjiang.Application of REM memory model in image recognition and classification[J].CAAI Transactions on Intelligent Systems,2017,12():310.[doi:10.11992/tis.201605010]
[10]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[11]刘彪,黄蓉蓉,林和,等.基于卷积神经网络的盲文音乐识别研究[J].智能系统学报,2019,14(1):186.[doi:10.11992/tis.201805002]
LIU Biao,HUANG Rongrong,LIN He,et al.Research on braille music recognition based on convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2019,14():186.[doi:10.11992/tis.201805002]
[12]王德文,魏波涛.基于孪生变分自编码器的小样本图像分类方法[J].智能系统学报,2021,16(2):254.[doi:10.11992/tis.201906022]
WANG Dewen,WEI Botao.A small-sample image classification method based on a Siamese variational auto-encoder[J].CAAI Transactions on Intelligent Systems,2021,16():254.[doi:10.11992/tis.201906022]
[13]骆彦龙,毕晓君,吴立成,等.基于改进残差学习的东巴象形文字识别[J].智能系统学报,2022,17(1):79.[doi:10.11992/tis.202112009]
LUO Yanlong,BI Xiaojun,WU Licheng,et al.Dongba pictographs recognition based on improved residual learning[J].CAAI Transactions on Intelligent Systems,2022,17():79.[doi:10.11992/tis.202112009]
[14]翟永杰,张智柏,王亚茹.基于改进TransGAN的零样本图像识别方法[J].智能系统学报,2023,18(2):352.[doi:10.11992/tis.202111002]
ZHAI Yongjie,ZHANG Zhibai,WANG Yaru.An image recognition method of zero -shot learning based on an improved TransGAN[J].CAAI Transactions on Intelligent Systems,2023,18():352.[doi:10.11992/tis.202111002]
备注/Memo
收稿日期:2019-06-18。
基金项目:国家自然科学基金项目（61471080/F010408）；国家支撑计划（2015BAF20B02）；国家留学基金委资助计划（201608210308）；辽宁省自然科学基金指导计划（2019-ZD-0108）
作者简介:闫涵，硕士研究生，主要研究方向为图像处理与模式识别;张旭秀，教授，博士后，主要研究方向为图像处理与模式识别、智能控制、信号处理。主持和参与完成国家自然科学基金项目、辽宁省自然科学基金项目9项。发表学术论文40余篇;张净丹，硕士研究生，主要研究方向为图像处理与模式识别
通讯作者:闫涵.E-mail:1346917459@qq.com
更新日期/Last Update:
2021-04-25