[1]刘威,刘尚,周璇.BP神经网络子批量学习方法研究[J].智能系统学报编辑部,2016,11(2):226-232.[doi:10.11992/tis.201509015]
LIU Wei,LIU Shang,ZHOU Xuan.Subbatch learning method for BP neural networks[J].CAAI Transactions on Intelligent Systems,2016,11(2):226-232.[doi:10.11992/tis.201509015]
点击复制
LIU Wei,LIU Shang,ZHOU Xuan.Subbatch learning method for BP neural networks[J].CAAI Transactions on Intelligent Systems,2016,11(2):226-232.[doi:10.11992/tis.201509015]
BP神经网络子批量学习方法研究
《智能系统学报》编辑部[ISSN 1673-4785/CN 23-1538/TP] 卷:
11
期数:
2016年第2期
页码:
226-232
栏目:
学术论文—机器学习
出版日期:
2016-04-25
- Title:
- Subbatch learning method for BP neural networks
- Keywords:
- subbatch learning; neural network; backpropagation algorithms; batch size; training methods and evaluation; classification
- 分类号:
- TP301.6
- 摘要:
- 针对浅层神经网络全批量学习收敛缓慢和单批量学习易受随机扰动的问题,借鉴深度神经网基于子批量的训练方法,提出了针对浅层神经网络的子批量学习方法和子批量学习参数优化配置方法。数值实验结果表明:浅层神经网络子批量学习方法是一种快速稳定的收敛算法,算法中批量和学习率等参数配置对于网络的收敛性、收敛时间和泛化能力有着重要的影响,学习参数经优化后可大幅缩短网络收敛迭代次数和训练时间,并提高网络分类准确率。
- Abstract:
- When solving problems in shallow neural networks, the full-batch learning method converges slowly and the single-batch learning method fluctuates easily. By referring to the subbatch training method for deep neural networks, this paper proposes the subbatch learning method and the subbatch learning parameter optimization and allocation method for shallow neural networks. Experimental comparisons indicate that subbatch learning in shallow neural networks converges quickly and stably. The batch size and learning rate have significant impacts on the net convergence, convergence time, and generation ability. Selecting the optimal parameters can dramatically shorten the iteration time for convergence and the training time as well as improve the classification accuracy.
- 参考文献/References:
-
[1] JACOBS R A. Increased rates of convergence through learning rate adaptation[J]. Neural networks, 1988, 1(4): 295-307.
[2] 刘幺和, 陈睿, 彭伟, 等. 一种BP神经网络学习率的优化设计[J]. 湖北工业大学学报, 2007, 22(3): 1-3. LIU Yaohe, CHEN Rui, PENG Wei. Optimal design for learning rate of BP neutral network[J]. Journal of Hubei university of technology, 2007, 22(3): 1-3.
[3] 贾立山, 谈至明, 王知. 基于随机参数调整的改进反向传播学习算法[J]. 同济大学学报:自然科学版, 2011, 39(5): 751-757. JIA Lishan, TAN Zhiming, WANG Zhi. Modified BP algorithm based on random adjustment of parameters[J]. Journal of Tongji university (natural science), 2011, 39(5): 751-757.
[4] RIEDMILLER M, BRAUN H. RPROP-A fast adaptive learning algorithm[C]//Proceedings of the International Symposium on Computer and Information Sciences (ISCIS VII).Ankara, Turkey, 1992.
[5] CHARALAMBOUS C. Conjugate gradient algorithm for efficient training of artificial neural networks[J]. Devices and systems, IEE proceedings G-Circuits, 1992, 139(3): 301-310.
[6] VOGL T P, MANGIS J K, RIGLER A K, et al. Accelerating the convergence of the back-propagation method[J]. Biological cybernetics, 1988, 59(4/5): 257-263.
[7] DENNIS J E Jr, SCHNABEL R B. Numerical methods for unconstrained optimization and nonlinear equations[M]. Philadelphia, USA: SIAM, 1996.
[8] MOR J J. The Levenberg-Marquardt algorithm: implementation and theory[M]//WATSON G A. Numerical Analysis. Berlin Heidelberg: Springer, 1978: 105-116.
[9] 侯祥林, 陈长征, 虞和济, 等. 神经网络权值和阈值的优化方法[J]. 东北大学学报:自然科学版, 1999, 20(4): 447-450. HOU Xianglin, CHEN Changzheng, YU Heji, et al. Optimum method about weights and thresholds of neural network[J]. Journal of northeastern university: natural science, 1999, 20(4): 447-450.
[10] 刘奕君, 赵强, 郝文利. 基于遗传算法优化BP神经网络的瓦斯浓度预测研究[J]. 矿业安全与环保, 2015, 42(2): 56-60. LIU Yijun, ZHAO Qiang, HAO Wenli. Study of gas concentration prediction based on genetic algorithm and optimizing BP neural network[J]. Mining safety & environmental protection, 2015, 42(2): 56-60.
[11] HAM F M, KOSTANIC I. Principles of neurocomputing for science and engineering[M]. New York, NY: McGraw-Hill Science, 2000.
[12] http://archive.ics.uci.edu/ml/datasets/seeds.
[13] 史峰, 王小川, 郁磊, 等. MATLAB神经网络30个案例分析[M]. 北京: 北京航空航天大学出版社, 2010: 1-3.
- 相似文献/References:
-
[1]丁永生.计算智能的新框架：生物网络结构[J].智能系统学报编辑部,2007,2(2):26.
DING Yong-sheng.A new scheme for computational intelligence: bio-network architecture[J].CAAI Transactions on Intelligent Systems,2007,2():26.
[2]徐 雄.人工情感的进化控制系统实现[J].智能系统学报编辑部,2008,3(2):135.
XU Xiong.Implementation of an evolutionary control system based on artificial emotion[J].CAAI Transactions on Intelligent Systems,2008,3():135.
[3]周孔丹,李 宁,鲁华祥.单电子电路的鲁棒性研究[J].智能系统学报编辑部,2008,3(3):195.
ZHOU Kong-dan,LI Ning,LU Hua-xiang.Researching the robustness of single electron devices[J].CAAI Transactions on Intelligent Systems,2008,3():195.
[4]张米娜,韩红桂,乔俊飞.前馈神经网络结构动态增长-修剪方法[J].智能系统学报编辑部,2011,6(2):101.
ZHANG Mina,HAN Honggui,QIAO Junfei.Research on dynamic feedforward neural network structure based on growing and pruning methods[J].CAAI Transactions on Intelligent Systems,2011,6():101.
[5]张文辉,高九州,马静,等.漂浮基空间机器人的径向基神经网络鲁棒自适应控制[J].智能系统学报编辑部,2011,6(2):114.
ZHANG Wenhui,GAO Jiuzhou,MA Jing,et al.The RBF neural network robust adaptive control of a freefloating space robot[J].CAAI Transactions on Intelligent Systems,2011,6():114.
[6]薄迎春,乔俊飞,杨刚.一种多模块协同参与的神经网络[J].智能系统学报编辑部,2011,6(3):225.
BO Yingchun,QIAO Junfei,YANG Gang.A multimodule cooperative neural network[J].CAAI Transactions on Intelligent Systems,2011,6():225.
[7]蒲兴成,张军,张毅.基于神经网络的改进行为协调控制及其在智能轮椅路径规划中的应用[J].智能系统学报编辑部,2011,6(5):456.
PU Xingcheng,ZHANG Jun,ZHANG Yi.Modified behavior coordination for intelligent wheelchair path planning based on a neural network[J].CAAI Transactions on Intelligent Systems,2011,6():456.
[8]段海庆,朱齐丹.基于反步自适应神经网络的船舶航迹控制[J].智能系统学报编辑部,2012,7(3):259.
DUAN Haiqing,ZHU Qidan.Trajectory tracking control of ships based onan adaptive backstepping neural network[J].CAAI Transactions on Intelligent Systems,2012,7():259.
[9]乔俊飞,逄泽芳,韩红桂.基于改进粒子群算法的污水处理过程神经网络优化控制[J].智能系统学报编辑部,2012,7(5):429.
QIAO Junfei,PANG Zefang,HAN Honggui.Neural network optimal control for wastewater treatment processbased on APSO[J].CAAI Transactions on Intelligent Systems,2012,7():429.
[10]郭一,刘金琨.带执行器饱和的柔性关节机器人位置反馈动态面控制[J].智能系统学报编辑部,2013,8(1):21.[doi:10.3969/j.issn.1673-4785.201204012]
GUO Yi,LIU Jinkun.Position feedback dynamic surface control for flexible joint robots with actuator saturation[J].CAAI Transactions on Intelligent Systems,2013,8():21.[doi:10.3969/j.issn.1673-4785.201204012]
备注/Memo
收稿日期:2015-9-7;改回日期:。
基金项目:国家自然科学基金项目(51304114,71371091).
作者简介:刘威,男,1977年生,副教授,博士,中国计算机学会会员,主要研究方向为模式识别、时间序列数据挖掘、矿业系统工程;刘尚,男,1988年生,硕士研究生,主要研究方向为模式识别、人工智能、计算机视觉;周璇,女,1992年生,硕士研究生,主要研究方向为模式识别、矿业系统工程。
通讯作者:刘尚.E-mail:whiteinblue@126.com.
更新日期/Last Update:
1900-01-01