[1]SUN Bishen,SHI Wuzhen,JIANG Feng.Core problem in computer vision: survey of natural image prior models[J].CAAI Transactions on Intelligent Systems,2019,14(1):71-81.[doi:10.11992/tis.201804019]
Copy

Core problem in computer vision: survey of natural image prior models

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 14 Number of periods: 2019 1 Page number: 71-81 Column: 综述 Public date: 2019-01-05
Title:
Core problem in computer vision: survey of natural image prior models
Author(s):
SUN Bishen1;  SHI Wuzhen2;  JIANG Feng2
1. No.27 Institute, China Electronic Technology Corporation (CETC), Zhengzhou 450005, China;
2. School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
Keywords:
computer vision; image prior; sparse representation; local smoothness; non-local self-similarity; compressed sensing; deep learning; convolutional neural network
CLC:
TP391.4
DOI:
10.11992/tis.201804019
Abstract:
One of the core problems in computer vision is that the visual prior is the point of intersection of the cognitive psychological level, systematic neural level, and computer vision level, and requires an understanding and synthesis of the three. Simulations of the visual prior function are performed to explore and formalize the general rules for natural images that support various applications in image processing and computer science. In this paper, we comprehensively analyze the work of various schools of natural image priori modeling and discuss the prospective application of natural image prior modeling in visual information enhancement and coding.
References:
[1] PARK S, KONKLE T, OLIVA A. Parametric coding of the size and clutter of natural scenes in the human brain[J]. Cerebral cortex, 2015, 25(7):1792-1805.
[2] F?LDIáK P. Neural control:closed-loop human brain reading[J]. Current biology, 2011, 21(2):80-81.
[3] KAY K N, GALLANT J L. I can see what you see[J]. Nature neuroscience, 2009, 12(3):245.
[4] KIM M K, KIM M, OH E, et al. A review on the computational methods for emotional state estimation from the human EEG[J]. Computational and mathematical methods in medicine, 2013, 22(12):3127-3142.
[5] JIAN Muwei, QI Qiang, DONG Junyu, et al. Saliency detection using quaternionic distance based weber local descriptor and level priors[J]. Multimedia tools and applications, 2018, 77(11):14343-14360.
[6] JIAN Muwei, LAM K M, DONG Junyu, et al. Visual-patch-attention-aware saliency detection[J]. IEEE transactions on cybernetics, 2015, 45(8):1575-1586.
[7] JIAN Muwei, LAM K M, DONG Junyu. Facial-feature detection and localization based on a hierarchical scheme[J]. Information sciences, 2014, 262(3):1-14.
[8] JIAN Muwei, QI Qiang, DONG Junyu, et al. Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection[J]. Journal of visual communication and image representation, 2018, 53:31-41.
[9] TREVES A, TASHIRO A, WITTER M P, et al. What is the mammalian dentate gyrus good for?[J]. Neuroscience, 2008, 154(4):1155-1172.
[10] CHANTAS G, GALATSANOS N P, MOLINA R, et al. Variational bayesian image restoration with a product of spatially weighted total variation image priors[J]. IEEE transactions on image processing, 2010, 19(2):351-362.
[11] CHAN T F, OSHER S, SHEN J. The digital TV filter and nonlinear denoising[J]. IEEE transactions on image processing, 2001, 10(2):231-241.
[12] ZHU Songchun, WU Yingnian, MUMFORD D. Filters, random fields and maximum entropy (FRAME):towards a unified theory for texture modeling[J]. International journal of computer vision, 1998, 27(2):107-126.
[13] FREEMAN W T, PASZTOR E C, CARMICHAEL O T. Learning low-level vision[J]. International journal of computer vision, 2000, 40(1):25-47.
[14] WELLING M, OSINDERO S, HINTON G E. Learning sparse topographic representations with products of student-t distributions[C]//Advances in Neural Information Processing Systems. Vancouver, Canada, 2003:1383-1390.
[15] ROTH S, BLACK M J. Fields of experts[J]. International journal of computer vision, 2009, 82:205.
[16] SUN Deqing, ROTH S, BLACK M J. A quantitative analysis of current practices in optical flow estimation and the principles behind them[J]. International journal of computer vision, 2014, 106(2):115-137.
[17] HUBEL D H, WIESEL T N. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex[J]. The journal of physiology, 1962, 160(1):106-154.
[18] XU Mai, LI Shengxi, LU Jianhua, et al. Compressibility constrained sparse representation with learnt dictionary for low bit-rate image compression[J]. IEEE transactions on circuits and systems for video technology, 2014, 24(10):1743-1757.
[19] SUN Yipeng, TAO Xiaoming, LI Yang, et al. Dictionary learning for image coding based on multisample sparse representation[J]. IEEE transactions on circuits and systems for video technology, 2014, 24(11):2004-2010.
[20] SUN Fuming, TANG Jinhui, LI Haojie, et al. Multi-label image categorization with sparse factor representation[J]. IEEE transactions on image processing, 2014, 23(3):1028-1037.
[21] RABBANI H, GAZOR S. Local probability distribution of natural signals in sparse domains[J]. International journal of adaptive control and signal processing, 2014, 28(1):52-62.
[22] ELHAMIFAR E, VIDAL R. Sparse subspace clustering:Algorithm, theory, and applications[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(11):2765-2781.
[23] 肖迪, 马青青, 王兰, 等. 基于稀疏表示的云协助安全数字水印技术[J]. 信息网络安全, 2017, 37(1):1-7 XIAO Di, MA Qingqing, WANG Lan, et al. Cloud-assisted secure digital watermarking based on sparse representation[J]. Netinfo security, 2017, 37(1):1-7
[24] 熊金波, 马蓉, 张媛媛, 等. 面向社交网络的图片信息隐藏方法与实现[J]. 信息网络安全, 2017, 37(3):6-13 XIONG Jinbo, MA Rong, ZHANG Yuanyuan, et al. Image information hiding method and implementation for social network[J]. Netinfo security, 2017, 37(3):6-13
[25] 徐杰, 贺敏, 包秀国. 基于压缩感知的视频台标识别研究[J]. 信息网络安全, 2015(5):77-81 XU Jie, HE Min, BAO Xiuguo. Research on TV logo detection based on compressive sensing[J]. Netinfo security, 2015(5):77-81
[26] 任栋, 宋伟, 于京, 等. 特殊视频内容检测算法研究综述[J]. 信息网络安全, 2016(9):184-191 REN Dong, SONG Wei, YU Jing, et al. A survey on special video content detection algorithms[J]. Netinfo security, 2016(9):184-191
[27] PROTTER M, ELAD M, TAKEDA H, et al. Generalizing the nonlocal-means to super-resolution reconstruction[J]. IEEE transactions on image processing, 2009, 18(1):36-51.
[28] TALEBI H, MILANFAR P. Global image denoising[J]. IEEE transactions on image processing, 2014, 23(2):755-768.
[29] RAM I, ELAD M, COHEN I. Image processing using smooth ordering of its patches[J]. IEEE transactions on image processing, 2013, 22(7):2764-2774.
[30] GILBOA G, OSHER S. Nonlocal operators with applications to image processing[J]. Multiscale modeling and simulation, 2008, 7(3):1005-1028.
[31] ZHANG Lei, DONG Weisheng, ZHANG D, et al. Two-stage image denoising by principal component analysis with local pixel grouping[J]. Pattern recognition, 2010, 43(4):1531-1549.
[32] MAIRAL J, BACH F, PONCE J, et al. Non-local sparse models for image restoration[C]//Proceedings of the 12th International Conference on Computer Vision. Kyoto, Japan, 2009:2272-2279.
[33] JUNG M, BRESSON X, CHAN T F, et al. Nonlocal Mumford-Shah regularizers for color image restoration[J]. IEEE transactions on image processing, 2011, 20(6):1583-1598.
[34] SUN Jian, TAPPEN M F. Learning non-local range Markov random field for image restoration[C]//CVPR 2011. Colorado Springs, CO, USA, 2011:2745-2752.
[35] GAO Xinwei, ZHANG Jian, JIANG Feng, et al. Image interpolation via local autoregressive and nonlocal 3-D sparse regularization[C]//Visual Communications and Image Processing. San Diego, USA, 2012.
[36] DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference On Computer Vision. Cham, Germany, 2014:184-199.
[37] DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(2):295-307.
[38] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:391-407.
[39] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016:1646-1654.
[40] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, United States, 2016:770-778.
[41] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Delving deep into rectifiers:surpassing human-level performance on imagenet classification[C]//Proceedings of IEEE International Conference on Computer Vision. Santiago, Chile, 2015:1026-1034.
[42] WANG Zhaowen, LIU Ding, YANG Jianchao, et al. Deep networks for image super-resolution with sparse prior[C]//Proceedings of IEEE International Conference on Computer Vision. Santiago, Chile, 2015:370-378.
[43] GU S, ZUO W, XIE Q, et al. Convolutional sparse coding for image super-resolution[C]//IEEE International Conference on Computer Vision. IEEE Computer Society, 2015:1823-1831.
[44] LIANG Yudong, WANG Jinjun, ZHOU Sanping, et al. Incorporating image priors with deep convolutional neural networks for image super-resolution[J]. Neurocomputing, 2016, 194:340-347.
[45] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, MA, USA, 2014:2672-2680.
[46] LEDIG C, THEIS L, HUSZáR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, 2017:105-114
[47] JOHNSON J, ALAHI A, FEI-FEI L. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:694-711.
[48] STANKOVI? V, STANKOVI? L, CHENG S. Compressive video sampling[C]//Processing of 16th European Signal Processing Conference. Lausanne, Switzerland, 2008:1–5.
[49] PARK J Y, WAKIN M B. A multiscale framework for compressive sensing of video[C]//Processing of Picture Coding Symposium. Chicago, USA, 2009:1-4.
[50] PRADES-NEBOT J, MA Yi, HUANG T. Distributed video coding using compressive sampling[C]//Processing of Picture Coding Symposium. Chicago, USA, 2009:1-4.
[51] CHEN H W, KANG Liwei, LU C S. Dictionary learning-based distributed compressive video sensing[C]//Processing of Picture Coding Symposium. Nagoya, Japan, 2010:210-213.
[52] WRIGHT S J, NOWAK R D, FIGUEIREDO M A T. Sparse reconstruction by separable approximation[J]. IEEE transactions on signal processing, 2009, 57(7):2479-2493.
[53] PONG K H, LAM K M. Multi-resolution feature fusion for face recognition[J]. Pattern recognition, 2014, 47(2):556-567.
[54] PYATYKH S, HESSER J. Salt and pepper noise removal in binary images using image block prior probabilities[J]. Journal of visual communication and image representation, 2014, 25(5):748-754.
[55] YIN Zhouping, ZHANG Hongmei. Research on saliency prior based image processing algorithm[J]. Journal of multimedia, 2014, 9(2):294-301.
[56] JIANG Feng, TAO Wen, LIU Shaohui, et al. An end-to-end compression framework based on convolutional neural networks[J]. IEEE transactions on circuits and systems for video technology, 2018, 28(10):3007-3018.
[57] ZHAO Lijun, BAI Huihui, WANG Aahong, et al. Learning a virtual codec based on deep convolutional neural network to compress image[EB/OL]. arXiv preprint arXiv:1712.05969, 2017. https://arxiv.org/abs/1712.05969
[58] SULLIVAN G J, OHM J R, HAN W J, et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE transactions on circuits and systems for video technology, 2012, 22(12):1649-1668.
[59] LI Jiahao, LI Bin, XU Jizheng, et al. Intra prediction using fully connected network for video coding[C]//IEEE International Conference on Image Processing. Beijing, China, 2017:1-5.
[60] YAN Ning, LIU Dong, LI Houqiang, et al. A convolutional neural network approach for half-pel interpolation in video coding[C]//Proceedings of 2017 International Symposium on Circuits and Systems. Baltimore, MD, USA, 2017:1-4.
[61] LIU Zhenyu, YU Xianyu, CHEN Shaolin, et al. CNN oriented fast HEVC intra CU mode decision[C]//IEEE International Symposium on Circuits and Systems. Montreal, QC, Canada, 2016:2270-2273.
[62] LIU Dong, MA Haichuan, XIONG Zhiwei, et al. CNN-Based DCT-like transform for image compression[C]//Proceedings of the 24th International Conference on Multimedia Modeling. Bangkok, Thailand, 2018:61-72.
[63] SONG Rui, LIU Dong, LI Houqiang, et al. Neural network-based arithmetic coding of intra prediction modes in HEVC[C]//IEEE Visual Communications and Image Processing Conference. Petersburg, FL, USA, 2017:1–4.
[64] DAI Yuanying, LIU Dong, WU Feng. A convolutional neural network approach for post-processing in HEVC intra coding[C]//Proceedings of the 23rd International Conference on Multimedia Modeling. Reykjavik, Iceland, 2017:28-39.
[65] TODERICI G, O’ MALLEY S M, HWANG S J, et al. Variable rate image compression with recurrent neural networks[EB/OL]. arXiv preprint arXiv:1511.06085, 2016. https://arxiv.org/abs/1511.06085
[66] TODERICI G, VINCENT D, JOHNSTON N, et al. Full resolution image compression with recurrent neural networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii, USA, 2017:5435-5443.
[67] JOHNSTON N, VINCENT D, MINNEN D, et al. Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks[EB/OL]. arXiv preprint arXiv:1703.10114, 2017. https://arxiv.org/abs/1703.10114
[68] THEIS L, SHI Wenzhe, CUNNINGHAM A, et al. Lossy image compression with compressive autoencoders[EB/OL]. arXiv preprint arXiv:1703.00395, 2017. https://arxiv.org/abs/1703.00395
[69] BALLé J, LAPARRA V, SIMONCELLI E P. End-to-end optimization of nonlinear transform codes for perceptual quality[C]//Proceedings of 2016 Picture Coding Symposium. Nuremberg, Germany, 2016:1-5.
[70] AGUSTSSON E, MENTZER F, TSCHANNEN M, et al. Soft-to-hard vector quantization for end-to-end learning compressible representations[C]//Advances in Neural Information Processing Systems. Long Beach, CA, USA, 2017:1141-1151.
[71] LI Mu, ZUO Wangmeng, GU Shuhang, et al. Learning convolutional networks for content-weighted image compression[EB/OL]. arXiv preprint arXiv:1703.10553, 2017. https://arxiv.org/abs/1703.10553
[72] RIPPEL O, BOURDEV L. Real-time adaptive image compression[EB/OL]. arXiv preprint arXiv:1705.05823, 2017. https://arxiv.org/abs/1705.05823
[73] BAIG M H, KOLTUN V, TORRESANI L. Learning to Inpaint for Image Compression[C]//Advances in Neural Information Processing Systems. Long Beach, CA, USA, 2017:1246-1255.
[74] TIMOFTE R, DE SMET V, VAN GOOL L. A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Asian Conference on Computer Vision. Springer, Cham, 2014:111?126.
[75] SCHULTER S, LEISTNER C, BISCHOF H. Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR2015. Boston, Massachusetts, USA, 2015:3791–3799.
[76] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(2):295?307.
[77] WANG Z, LIU D, YANG J, et al. Deeply improved sparse coding for image super-resolution. arXiv preprint[EB/OL]. arXiv preprint arXiv:1507.08905, 2015. https://arxiv.org/abs/1507.08905v1
[78] DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Springer, Cham, 2016:391?407.
[79] SHI W, JIANG F, ZHANG F, et al. Single image super-resolution with dilated convolution based multi-scale information learning inception module[C]//IEEE International Conference on in Image Processing (ICIP2017). Beijing, China, 2017:1?5.
[80] MUN S, FOWLER J E. Block compressed sensing of images using directional transforms[C]//200916th IEEE International Conference on Image Processing IEEE, 2009:3021?3024.
[81] LI, C, YIN W, ZHANG Y. TVAL3:TV minimization by augmented lagrangian and alternating direction algorithm[EB/OL]. https://www.caam.rice.edu/~optimization/L1/TVAL3/
[82] CHEN C, TRAMEL E W, FOWLER J E. Compressed-sensing recovery of images and video using multihypothesis predictions[C]//2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers. IEEE, 2011:1193?1198.
[83] ZHANG J, ZHAO D, ZHAO C, et al. Compressed sensing recovery via collaborative sparsity[C]//Processing of 2012 Data Compression Conference. Snowbird, Utah, USA, 2012:287–296.
[84] ZHANG J, ZHAO D, GAO W. Group-based sparse representation for image restoration[J]. IEEE transactions on image processing, 2014, 23(8):3336?3351.
[85] SHI W, JIANG F, ZHANG S, et al. Deep networks for compressed image sensing[C]//Processing of 2017 IEEE International Conference on Multimedia and Expo. Hongkong, China, 2017:877–882.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems