[1]孙必慎,石武祯,姜峰.计算视觉核心问题：自然图像先验建模研究综述[J].智能系统学报,2019,14(1):71-81.[doi:10.11992/tis.201804019]
SUN Bishen,SHI Wuzhen,JIANG Feng.Core problem in computer vision: survey of natural image prior models[J].CAAI Transactions on Intelligent Systems,2019,14(1):71-81.[doi:10.11992/tis.201804019]
点击复制
SUN Bishen,SHI Wuzhen,JIANG Feng.Core problem in computer vision: survey of natural image prior models[J].CAAI Transactions on Intelligent Systems,2019,14(1):71-81.[doi:10.11992/tis.201804019]
计算视觉核心问题：自然图像先验建模研究综述
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
14
期数:
2019年第1期
页码:
71-81
栏目:
综述
出版日期:
2019-01-05
- Title:
- Core problem in computer vision: survey of natural image prior models
- Keywords:
- computer vision; image prior; sparse representation; local smoothness; non-local self-similarity; compressed sensing; deep learning; convolutional neural network
- 分类号:
- TP391.4
- 摘要:
- 视觉先验是计算机视觉的核心问题之一，是认知心理层面、系统神经层面与计算视觉层面研究的交合点，涉及各个层面研究的理解与综合。视觉先验功能模拟方面，以自然图像信息为对象，挖掘自然图像一般性规律并将其数学形式化为可计算的图像模型，为众多图像处理与计算机视觉智能应用提供算法和支撑。本文对自然图像先验建模研究各流派工作进行了全面的剖析，并展示了自然图像先验建模工作在视觉信息增强和编码等方向的前瞻性应用。
- Abstract:
- One of the core problems in computer vision is that the visual prior is the point of intersection of the cognitive psychological level, systematic neural level, and computer vision level, and requires an understanding and synthesis of the three. Simulations of the visual prior function are performed to explore and formalize the general rules for natural images that support various applications in image processing and computer science. In this paper, we comprehensively analyze the work of various schools of natural image priori modeling and discuss the prospective application of natural image prior modeling in visual information enhancement and coding.
- 参考文献/References:
-
[1] PARK S, KONKLE T, OLIVA A. Parametric coding of the size and clutter of natural scenes in the human brain[J]. Cerebral cortex, 2015, 25(7):1792-1805.
[2] F?LDIáK P. Neural control:closed-loop human brain reading[J]. Current biology, 2011, 21(2):80-81.
[3] KAY K N, GALLANT J L. I can see what you see[J]. Nature neuroscience, 2009, 12(3):245.
[4] KIM M K, KIM M, OH E, et al. A review on the computational methods for emotional state estimation from the human EEG[J]. Computational and mathematical methods in medicine, 2013, 22(12):3127-3142.
[5] JIAN Muwei, QI Qiang, DONG Junyu, et al. Saliency detection using quaternionic distance based weber local descriptor and level priors[J]. Multimedia tools and applications, 2018, 77(11):14343-14360.
[6] JIAN Muwei, LAM K M, DONG Junyu, et al. Visual-patch-attention-aware saliency detection[J]. IEEE transactions on cybernetics, 2015, 45(8):1575-1586.
[7] JIAN Muwei, LAM K M, DONG Junyu. Facial-feature detection and localization based on a hierarchical scheme[J]. Information sciences, 2014, 262(3):1-14.
[8] JIAN Muwei, QI Qiang, DONG Junyu, et al. Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection[J]. Journal of visual communication and image representation, 2018, 53:31-41.
[9] TREVES A, TASHIRO A, WITTER M P, et al. What is the mammalian dentate gyrus good for?[J]. Neuroscience, 2008, 154(4):1155-1172.
[10] CHANTAS G, GALATSANOS N P, MOLINA R, et al. Variational bayesian image restoration with a product of spatially weighted total variation image priors[J]. IEEE transactions on image processing, 2010, 19(2):351-362.
[11] CHAN T F, OSHER S, SHEN J. The digital TV filter and nonlinear denoising[J]. IEEE transactions on image processing, 2001, 10(2):231-241.
[12] ZHU Songchun, WU Yingnian, MUMFORD D. Filters, random fields and maximum entropy (FRAME):towards a unified theory for texture modeling[J]. International journal of computer vision, 1998, 27(2):107-126.
[13] FREEMAN W T, PASZTOR E C, CARMICHAEL O T. Learning low-level vision[J]. International journal of computer vision, 2000, 40(1):25-47.
[14] WELLING M, OSINDERO S, HINTON G E. Learning sparse topographic representations with products of student-t distributions[C]//Advances in Neural Information Processing Systems. Vancouver, Canada, 2003:1383-1390.
[15] ROTH S, BLACK M J. Fields of experts[J]. International journal of computer vision, 2009, 82:205.
[16] SUN Deqing, ROTH S, BLACK M J. A quantitative analysis of current practices in optical flow estimation and the principles behind them[J]. International journal of computer vision, 2014, 106(2):115-137.
[17] HUBEL D H, WIESEL T N. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex[J]. The journal of physiology, 1962, 160(1):106-154.
[18] XU Mai, LI Shengxi, LU Jianhua, et al. Compressibility constrained sparse representation with learnt dictionary for low bit-rate image compression[J]. IEEE transactions on circuits and systems for video technology, 2014, 24(10):1743-1757.
[19] SUN Yipeng, TAO Xiaoming, LI Yang, et al. Dictionary learning for image coding based on multisample sparse representation[J]. IEEE transactions on circuits and systems for video technology, 2014, 24(11):2004-2010.
[20] SUN Fuming, TANG Jinhui, LI Haojie, et al. Multi-label image categorization with sparse factor representation[J]. IEEE transactions on image processing, 2014, 23(3):1028-1037.
[21] RABBANI H, GAZOR S. Local probability distribution of natural signals in sparse domains[J]. International journal of adaptive control and signal processing, 2014, 28(1):52-62.
[22] ELHAMIFAR E, VIDAL R. Sparse subspace clustering:Algorithm, theory, and applications[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(11):2765-2781.
[23] 肖迪, 马青青, 王兰, 等. 基于稀疏表示的云协助安全数字水印技术[J]. 信息网络安全, 2017, 37(1):1-7 XIAO Di, MA Qingqing, WANG Lan, et al. Cloud-assisted secure digital watermarking based on sparse representation[J]. Netinfo security, 2017, 37(1):1-7
[24] 熊金波, 马蓉, 张媛媛, 等. 面向社交网络的图片信息隐藏方法与实现[J]. 信息网络安全, 2017, 37(3):6-13 XIONG Jinbo, MA Rong, ZHANG Yuanyuan, et al. Image information hiding method and implementation for social network[J]. Netinfo security, 2017, 37(3):6-13
[25] 徐杰, 贺敏, 包秀国. 基于压缩感知的视频台标识别研究[J]. 信息网络安全, 2015(5):77-81 XU Jie, HE Min, BAO Xiuguo. Research on TV logo detection based on compressive sensing[J]. Netinfo security, 2015(5):77-81
[26] 任栋, 宋伟, 于京, 等. 特殊视频内容检测算法研究综述[J]. 信息网络安全, 2016(9):184-191 REN Dong, SONG Wei, YU Jing, et al. A survey on special video content detection algorithms[J]. Netinfo security, 2016(9):184-191
[27] PROTTER M, ELAD M, TAKEDA H, et al. Generalizing the nonlocal-means to super-resolution reconstruction[J]. IEEE transactions on image processing, 2009, 18(1):36-51.
[28] TALEBI H, MILANFAR P. Global image denoising[J]. IEEE transactions on image processing, 2014, 23(2):755-768.
[29] RAM I, ELAD M, COHEN I. Image processing using smooth ordering of its patches[J]. IEEE transactions on image processing, 2013, 22(7):2764-2774.
[30] GILBOA G, OSHER S. Nonlocal operators with applications to image processing[J]. Multiscale modeling and simulation, 2008, 7(3):1005-1028.
[31] ZHANG Lei, DONG Weisheng, ZHANG D, et al. Two-stage image denoising by principal component analysis with local pixel grouping[J]. Pattern recognition, 2010, 43(4):1531-1549.
[32] MAIRAL J, BACH F, PONCE J, et al. Non-local sparse models for image restoration[C]//Proceedings of the 12th International Conference on Computer Vision. Kyoto, Japan, 2009:2272-2279.
[33] JUNG M, BRESSON X, CHAN T F, et al. Nonlocal Mumford-Shah regularizers for color image restoration[J]. IEEE transactions on image processing, 2011, 20(6):1583-1598.
[34] SUN Jian, TAPPEN M F. Learning non-local range Markov random field for image restoration[C]//CVPR 2011. Colorado Springs, CO, USA, 2011:2745-2752.
[35] GAO Xinwei, ZHANG Jian, JIANG Feng, et al. Image interpolation via local autoregressive and nonlocal 3-D sparse regularization[C]//Visual Communications and Image Processing. San Diego, USA, 2012.
[36] DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference On Computer Vision. Cham, Germany, 2014:184-199.
[37] DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(2):295-307.
[38] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:391-407.
[39] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016:1646-1654.
[40] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, United States, 2016:770-778.
[41] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Delving deep into rectifiers:surpassing human-level performance on imagenet classification[C]//Proceedings of IEEE International Conference on Computer Vision. Santiago, Chile, 2015:1026-1034.
[42] WANG Zhaowen, LIU Ding, YANG Jianchao, et al. Deep networks for image super-resolution with sparse prior[C]//Proceedings of IEEE International Conference on Computer Vision. Santiago, Chile, 2015:370-378.
[43] GU S, ZUO W, XIE Q, et al. Convolutional sparse coding for image super-resolution[C]//IEEE International Conference on Computer Vision. IEEE Computer Society, 2015:1823-1831.
[44] LIANG Yudong, WANG Jinjun, ZHOU Sanping, et al. Incorporating image priors with deep convolutional neural networks for image super-resolution[J]. Neurocomputing, 2016, 194:340-347.
[45] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, MA, USA, 2014:2672-2680.
[46] LEDIG C, THEIS L, HUSZáR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, 2017:105-114
[47] JOHNSON J, ALAHI A, FEI-FEI L. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:694-711.
[48] STANKOVI? V, STANKOVI? L, CHENG S. Compressive video sampling[C]//Processing of 16th European Signal Processing Conference. Lausanne, Switzerland, 2008:1–5.
[49] PARK J Y, WAKIN M B. A multiscale framework for compressive sensing of video[C]//Processing of Picture Coding Symposium. Chicago, USA, 2009:1-4.
[50] PRADES-NEBOT J, MA Yi, HUANG T. Distributed video coding using compressive sampling[C]//Processing of Picture Coding Symposium. Chicago, USA, 2009:1-4.
[51] CHEN H W, KANG Liwei, LU C S. Dictionary learning-based distributed compressive video sensing[C]//Processing of Picture Coding Symposium. Nagoya, Japan, 2010:210-213.
[52] WRIGHT S J, NOWAK R D, FIGUEIREDO M A T. Sparse reconstruction by separable approximation[J]. IEEE transactions on signal processing, 2009, 57(7):2479-2493.
[53] PONG K H, LAM K M. Multi-resolution feature fusion for face recognition[J]. Pattern recognition, 2014, 47(2):556-567.
[54] PYATYKH S, HESSER J. Salt and pepper noise removal in binary images using image block prior probabilities[J]. Journal of visual communication and image representation, 2014, 25(5):748-754.
[55] YIN Zhouping, ZHANG Hongmei. Research on saliency prior based image processing algorithm[J]. Journal of multimedia, 2014, 9(2):294-301.
[56] JIANG Feng, TAO Wen, LIU Shaohui, et al. An end-to-end compression framework based on convolutional neural networks[J]. IEEE transactions on circuits and systems for video technology, 2018, 28(10):3007-3018.
[57] ZHAO Lijun, BAI Huihui, WANG Aahong, et al. Learning a virtual codec based on deep convolutional neural network to compress image[EB/OL]. arXiv preprint arXiv:1712.05969, 2017. https://arxiv.org/abs/1712.05969
[58] SULLIVAN G J, OHM J R, HAN W J, et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE transactions on circuits and systems for video technology, 2012, 22(12):1649-1668.
[59] LI Jiahao, LI Bin, XU Jizheng, et al. Intra prediction using fully connected network for video coding[C]//IEEE International Conference on Image Processing. Beijing, China, 2017:1-5.
[60] YAN Ning, LIU Dong, LI Houqiang, et al. A convolutional neural network approach for half-pel interpolation in video coding[C]//Proceedings of 2017 International Symposium on Circuits and Systems. Baltimore, MD, USA, 2017:1-4.
[61] LIU Zhenyu, YU Xianyu, CHEN Shaolin, et al. CNN oriented fast HEVC intra CU mode decision[C]//IEEE International Symposium on Circuits and Systems. Montreal, QC, Canada, 2016:2270-2273.
[62] LIU Dong, MA Haichuan, XIONG Zhiwei, et al. CNN-Based DCT-like transform for image compression[C]//Proceedings of the 24th International Conference on Multimedia Modeling. Bangkok, Thailand, 2018:61-72.
[63] SONG Rui, LIU Dong, LI Houqiang, et al. Neural network-based arithmetic coding of intra prediction modes in HEVC[C]//IEEE Visual Communications and Image Processing Conference. Petersburg, FL, USA, 2017:1–4.
[64] DAI Yuanying, LIU Dong, WU Feng. A convolutional neural network approach for post-processing in HEVC intra coding[C]//Proceedings of the 23rd International Conference on Multimedia Modeling. Reykjavik, Iceland, 2017:28-39.
[65] TODERICI G, O’ MALLEY S M, HWANG S J, et al. Variable rate image compression with recurrent neural networks[EB/OL]. arXiv preprint arXiv:1511.06085, 2016. https://arxiv.org/abs/1511.06085
[66] TODERICI G, VINCENT D, JOHNSTON N, et al. Full resolution image compression with recurrent neural networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii, USA, 2017:5435-5443.
[67] JOHNSTON N, VINCENT D, MINNEN D, et al. Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks[EB/OL]. arXiv preprint arXiv:1703.10114, 2017. https://arxiv.org/abs/1703.10114
[68] THEIS L, SHI Wenzhe, CUNNINGHAM A, et al. Lossy image compression with compressive autoencoders[EB/OL]. arXiv preprint arXiv:1703.00395, 2017. https://arxiv.org/abs/1703.00395
[69] BALLé J, LAPARRA V, SIMONCELLI E P. End-to-end optimization of nonlinear transform codes for perceptual quality[C]//Proceedings of 2016 Picture Coding Symposium. Nuremberg, Germany, 2016:1-5.
[70] AGUSTSSON E, MENTZER F, TSCHANNEN M, et al. Soft-to-hard vector quantization for end-to-end learning compressible representations[C]//Advances in Neural Information Processing Systems. Long Beach, CA, USA, 2017:1141-1151.
[71] LI Mu, ZUO Wangmeng, GU Shuhang, et al. Learning convolutional networks for content-weighted image compression[EB/OL]. arXiv preprint arXiv:1703.10553, 2017. https://arxiv.org/abs/1703.10553
[72] RIPPEL O, BOURDEV L. Real-time adaptive image compression[EB/OL]. arXiv preprint arXiv:1705.05823, 2017. https://arxiv.org/abs/1705.05823
[73] BAIG M H, KOLTUN V, TORRESANI L. Learning to Inpaint for Image Compression[C]//Advances in Neural Information Processing Systems. Long Beach, CA, USA, 2017:1246-1255.
[74] TIMOFTE R, DE SMET V, VAN GOOL L. A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Asian Conference on Computer Vision. Springer, Cham, 2014:111?126.
[75] SCHULTER S, LEISTNER C, BISCHOF H. Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR2015. Boston, Massachusetts, USA, 2015:3791–3799.
[76] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(2):295?307.
[77] WANG Z, LIU D, YANG J, et al. Deeply improved sparse coding for image super-resolution. arXiv preprint[EB/OL]. arXiv preprint arXiv:1507.08905, 2015. https://arxiv.org/abs/1507.08905v1
[78] DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Springer, Cham, 2016:391?407.
[79] SHI W, JIANG F, ZHANG F, et al. Single image super-resolution with dilated convolution based multi-scale information learning inception module[C]//IEEE International Conference on in Image Processing (ICIP2017). Beijing, China, 2017:1?5.
[80] MUN S, FOWLER J E. Block compressed sensing of images using directional transforms[C]//200916th IEEE International Conference on Image Processing IEEE, 2009:3021?3024.
[81] LI, C, YIN W, ZHANG Y. TVAL3:TV minimization by augmented lagrangian and alternating direction algorithm[EB/OL]. https://www.caam.rice.edu/~optimization/L1/TVAL3/
[82] CHEN C, TRAMEL E W, FOWLER J E. Compressed-sensing recovery of images and video using multihypothesis predictions[C]//2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers. IEEE, 2011:1193?1198.
[83] ZHANG J, ZHAO D, ZHAO C, et al. Compressed sensing recovery via collaborative sparsity[C]//Processing of 2012 Data Compression Conference. Snowbird, Utah, USA, 2012:287–296.
[84] ZHANG J, ZHAO D, GAO W. Group-based sparse representation for image restoration[J]. IEEE transactions on image processing, 2014, 23(8):3336?3351.
[85] SHI W, JIANG F, ZHANG S, et al. Deep networks for compressed image sensing[C]//Processing of 2017 IEEE International Conference on Multimedia and Expo. Hongkong, China, 2017:877–882.
- 相似文献/References:
-
[1]夏 凡,王 宏.基于局部异常行为检测的欺骗识别研究[J].智能系统学报,2007,2(5):12.
XIA Fan,WANG Hong.Methodologies for deception detection based on abnormal b ehavior[J].CAAI Transactions on Intelligent Systems,2007,2():12.
[2]杨 戈,刘 宏.视觉跟踪算法综述[J].智能系统学报,2010,5(2):95.
YANG Ge,LIU Hong.Survey of visual tracking algorithms[J].CAAI Transactions on Intelligent Systems,2010,5():95.
[3]刘宏,李哲媛,许超.视错觉现象的分类和研究进展[J].智能系统学报,2011,6(1):1.
LIU Hong,LI Zheyuan,XU Chao.The categories and research advances of visual illusions[J].CAAI Transactions on Intelligent Systems,2011,6():1.
[4]叶果,程洪,赵洋.电影中吸烟活动识别[J].智能系统学报,2011,6(5):440.
YE Guo,CHENG Hong,ZHAO Yang.moking recognition in movies[J].CAAI Transactions on Intelligent Systems,2011,6():440.
[5]史晓鹏,何为,韩力群.采用Hough变换的道路边界检测算法[J].智能系统学报,2012,7(1):81.
SHI Xiaopeng,HE Wei,HAN Liqun.A road edge detection algorithm based on the Hough transform[J].CAAI Transactions on Intelligent Systems,2012,7():81.
[6]顾照鹏,刘宏.单目视觉同步定位与地图创建方法综述[J].智能系统学报,2015,10(4):499.[doi:10.3969/j.issn.1673-4785.201503003]
GU Zhaopeng,LIU Hong.A survey of monocular simultaneous localization and mapping[J].CAAI Transactions on Intelligent Systems,2015,10():499.[doi:10.3969/j.issn.1673-4785.201503003]
[7]赵军,於俊,汪增福.基于改进逆向运动学的人体运动跟踪[J].智能系统学报,2015,10(4):548.[doi:10.3969/j.issn.1673-4785.201403032]
ZHAO Jun,YU Jun,WANG Zengfu.Human motion tracking based on an improved inverse kinematics[J].CAAI Transactions on Intelligent Systems,2015,10():548.[doi:10.3969/j.issn.1673-4785.201403032]
[8]姬晓飞,王昌汇,王扬扬.分层结构的双人交互行为识别方法[J].智能系统学报,2015,10(6):893.[doi:10.11992/tis.201505006]
JI Xiaofei,WANG Changhui,WANG Yangyang.Human interaction behavior-recognition method based on hierarchical structure[J].CAAI Transactions on Intelligent Systems,2015,10():893.[doi:10.11992/tis.201505006]
[9]方鹏,李贤,汪增福.运用核聚类和偏最小二乘回归的歌唱声音转换[J].智能系统学报,2016,11(1):55.[doi:10.11992/tis.201506022]
FANG Peng,LI Xian,WANG Zengfu.Conversion of singing voice based on kernel clustering and partial least squares regression[J].CAAI Transactions on Intelligent Systems,2016,11():55.[doi:10.11992/tis.201506022]
[10]李雪,蒋树强.智能交互的物体识别增量学习技术综述[J].智能系统学报,2017,12(2):140.[doi:10.11992/tis.201701006]
LI Xue,JIANG Shuqiang.Incremental learning and object recognition system based on intelligent HCI: a survey[J].CAAI Transactions on Intelligent Systems,2017,12():140.[doi:10.11992/tis.201701006]
备注/Memo
收稿日期:2018-04-15。
基金项目:国家自然科学基金项目（61572155，61672188，61272386）；国家重点基础研究发展计划项目（2015CB351804）.
作者简介:孙必慎,男,1976年生,高级工程师,主要研究方向为光电火控、侦察、预警及测量技术研究。主持多个国家重点型号装备研制,多次获得国家、部级技术进步奖。发表学术论文50余篇;石武祯,男,1989年生,博士研究生,主要研究方向为图像处理、计算机视觉以及图像编解码。发表学术论文10余篇;姜峰,男,1978年生,教授,博士生导师,主要研究方向为图像处理、计算机视觉以及人工智能。主持项目20余项,多次获得省部级技术进步奖。发表学术论文100余篇。
通讯作者:姜峰.E-mail:fjiang@hit.edu.cn
更新日期/Last Update:
1900-01-01