[1]LIU Wei,LIU Shang,ZHOU Xuan.Subbatch learning method for BP neural networks[J].CAAI Transactions on Intelligent Systems,2016,11(2):226-232.[doi:10.11992/tis.201509015]
Copy

Subbatch learning method for BP neural networks

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 11 Number of periods: 2016 2 Page number: 226-232 Column: 学术论文—机器学习 Public date: 2016-04-25
Title:
Subbatch learning method for BP neural networks
Author(s):
LIU Wei;  LIU Shang;  ZHOU Xuan
College of Science, Liaoning Technical University, Fuxin 123000, China
Keywords:
subbatch learning; neural network; backpropagation algorithms; batch size; training methods and evaluation; classification
CLC:
TP301.6
DOI:
10.11992/tis.201509015
Abstract:
When solving problems in shallow neural networks, the full-batch learning method converges slowly and the single-batch learning method fluctuates easily. By referring to the subbatch training method for deep neural networks, this paper proposes the subbatch learning method and the subbatch learning parameter optimization and allocation method for shallow neural networks. Experimental comparisons indicate that subbatch learning in shallow neural networks converges quickly and stably. The batch size and learning rate have significant impacts on the net convergence, convergence time, and generation ability. Selecting the optimal parameters can dramatically shorten the iteration time for convergence and the training time as well as improve the classification accuracy.
References:
[1] JACOBS R A. Increased rates of convergence through learning rate adaptation[J]. Neural networks, 1988, 1(4): 295-307.
[2] 刘幺和, 陈睿, 彭伟, 等. 一种BP神经网络学习率的优化设计[J]. 湖北工业大学学报, 2007, 22(3): 1-3. LIU Yaohe, CHEN Rui, PENG Wei. Optimal design for learning rate of BP neutral network[J]. Journal of Hubei university of technology, 2007, 22(3): 1-3.
[3] 贾立山, 谈至明, 王知. 基于随机参数调整的改进反向传播学习算法[J]. 同济大学学报:自然科学版, 2011, 39(5): 751-757. JIA Lishan, TAN Zhiming, WANG Zhi. Modified BP algorithm based on random adjustment of parameters[J]. Journal of Tongji university (natural science), 2011, 39(5): 751-757.
[4] RIEDMILLER M, BRAUN H. RPROP-A fast adaptive learning algorithm[C]//Proceedings of the International Symposium on Computer and Information Sciences (ISCIS VII).Ankara, Turkey, 1992.
[5] CHARALAMBOUS C. Conjugate gradient algorithm for efficient training of artificial neural networks[J]. Devices and systems, IEE proceedings G-Circuits, 1992, 139(3): 301-310.
[6] VOGL T P, MANGIS J K, RIGLER A K, et al. Accelerating the convergence of the back-propagation method[J]. Biological cybernetics, 1988, 59(4/5): 257-263.
[7] DENNIS J E Jr, SCHNABEL R B. Numerical methods for unconstrained optimization and nonlinear equations[M]. Philadelphia, USA: SIAM, 1996.
[8] MOR J J. The Levenberg-Marquardt algorithm: implementation and theory[M]//WATSON G A. Numerical Analysis. Berlin Heidelberg: Springer, 1978: 105-116.
[9] 侯祥林, 陈长征, 虞和济, 等. 神经网络权值和阈值的优化方法[J]. 东北大学学报:自然科学版, 1999, 20(4): 447-450. HOU Xianglin, CHEN Changzheng, YU Heji, et al. Optimum method about weights and thresholds of neural network[J]. Journal of northeastern university: natural science, 1999, 20(4): 447-450.
[10] 刘奕君, 赵强, 郝文利. 基于遗传算法优化BP神经网络的瓦斯浓度预测研究[J]. 矿业安全与环保, 2015, 42(2): 56-60. LIU Yijun, ZHAO Qiang, HAO Wenli. Study of gas concentration prediction based on genetic algorithm and optimizing BP neural network[J]. Mining safety & environmental protection, 2015, 42(2): 56-60.
[11] HAM F M, KOSTANIC I. Principles of neurocomputing for science and engineering[M]. New York, NY: McGraw-Hill Science, 2000.
[12] http://archive.ics.uci.edu/ml/datasets/seeds.
[13] 史峰, 王小川, 郁磊, 等. MATLAB神经网络30个案例分析[M]. 北京: 北京航空航天大学出版社, 2010: 1-3.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems