[1]YIN Rui,SU Songzhi,LI Shaozi.Convolutional neural network’s image moment regularizing strategy[J].CAAI Transactions on Intelligent Systems,2016,11(1):43-48.[doi:10.11992/tis.201509018]
Copy
Convolutional neural network’s image moment regularizing strategy
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
11
Number of periods:
2016 1
Page number:
43-48
Column:
学术论文—智能系统
Public date:
2016-02-25
- Title:
- Convolutional neural network’s image moment regularizing strategy
- CLC:
- TP391.4
- DOI:
- 10.11992/tis.201509018
- Abstract:
- There are two kinds of pooling strategies for convolutional neural network (CNN) as follows: max pooling and average pooling. Max pooling simply chooses the maximum element, which makes this strategy extremely prone to overfitting. Average pooling endows all elements with the same weight, which lowers the weight of the high-frequency components. In this study, we propose moment pooling as a regularization strategy for CNN. First, we introduce the geometric moment to CNN pooling and calculate the central moment of the pooling region. Then, we randomly select the response values based on the probability-like interpolation method from the four neighbors of the moment as per their probability. Experiments on the MNIST, CIFAR10, and CIFAR100 datasets show that moment pooling obtains the fewest training and test errors with training iteration increments. This strategy’s robustness and strong discrimination capability yield better generalization results than those from the max and average pooling methods.
- References:
-
[1] MONTAVON G, ORR G, MÜLLER K R. Neural networks:tricks of the trade[M]. 2nd ed. Berlin Heidelberg:Springer, 2012.
[2] HINTON G E, SRIVASTAVE N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[EB/OL].[2012-07-03]. http://arxiv.org/pdf/1207.0580.pdf.
[3] NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning. Haifa, Israel, 2010.
[4] RANZATO M, BOUREAU Y L, LECUN Y. Sparse feature learning for deep belief networks[C]//Proceedings of Advances in Neural Information Processing Systems (NIPS). Cambridge, MA, 2007.
[5] LECUN Y, BOSER B E, DENKER J S, et al. Handwritten digit Recognition with a back-propagation network[C]//Proceedings of Advances in Neural Information Processing Systems (NIPS). Cambridge, MA, 1989.
[6] HU M K. Visual pattern recognition by moment invariants[J]. IRE Transactions on Information Theory, 1962, 8(2):179-187.
[7] ROSIN P L. Measuring corner properties[J]. Computer vision and image understanding, 1999, 73(2):291-307.
[8] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB:an efficient alternative to SIFT or SURF[C]//Proceedings of IEEE International Conference on Computer Vision (ICCV). Barcelona, 2011:2564-2571.
[9] EVANS O D, KIM Y. Efficientimplementation of image warping on a multimedia processor[J]. Real-time imaging, 1998, 4(6):417-428.
[10] GONZALEZ R C, WOODS R E. Digital image processing[M]. 2nd ed.New Jersey:Prentice-Hall, 2002.
[11] JIA Y, SHEHAMER E, DONAHUE J,et al.Caffe:convolutional architecture for fast feature emibedding[C]//Proceedings of the ACM International conference on Multimedia. ACM, 2014:625-678.
[12] BOTTOU L. Stochastic gradient descent tricks[M]//MONTAVON G, ORR G B, MÜLLER K R. Neural Networks:Tricks of the Trade. 2nd ed. Berlin Heidelberg:Springer, 2012:421-436.
[13] KRIZHEVSKY A. The CIFAR-10, CIFAR-100 database[EB/OL]. http://www.cs.toronto.edu/~kriz/cifar.html. LECUN Y, CORTES C, BURGES C J C. The MNIST database of handwritten digits[EB/OL]. http://yann.lecun.com/exdb/mnist/.
[14] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
- Similar References:
Memo
-
Last Update:
1900-01-01