[1]张昭昭,乔俊飞,杨刚.自适应前馈神经网络结构优化设计[J].智能系统学报,2011,6(4):312-317.
　ZHANG Zhaozhao,QIAO Junfei,YANG Gang.An adaptive algorithm for designingoptimal feedforward neural network architecture[J].CAAI Transactions on Intelligent Systems,2011,6(4):312-317.
点击复制

自适应前馈神经网络结构优化设计

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 6 期数: 2011年第4期 页码: 312-317 栏目: 学术论文—机器学习 出版日期: 2011-08-25
Title:
An adaptive algorithm for designingoptimal feedforward neural network architecture
文章编号:
1673-4785(2011)04-0312-06
作者:
张昭昭1，2，乔俊飞1，杨刚1
1.北京工业大学 电子信息与控制工程学院,北京100124;
2.辽宁工程技术大学 电子与信息工程学院, 辽宁 葫芦岛 125105
Author(s):
ZHANG Zhaozhao1，2, QIAO Junfei1, YANG Gang1
1. College of Electronic and Control Engineering, Beijing University of Technology, Beijing 100124, China;
2. Institute of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China
关键词:
前馈神经网络; 结构设计; 自适应搜索策略; 互信息
Keywords:
feedforward neural network;  architecture design;  adaptive search strategy;  mutual information
分类号:
TP273
文献标志码:
A
摘要:
针对多数前馈神经网络结构设计算法采取贪婪搜索策略而易陷入局部最优结构的问题，提出一种自适应前馈神经网络结构设计算法.该算法在网络训练过程中采取自适应寻优策略合并和分裂隐节点，达到设计最优神经网络结构的目的.在合并操作中，以互信息为准则对输出线性相关的隐节点进行合并；在分裂操作中，引入变异系数，有助于跳出局部最优网络结构.算法将合并和分裂操作之后的权值调整与网络对样本的学习过程结合，减少了网络对样本的学习次数，提高了网络的学习速度，增强了网络的泛化性能.非线性函数逼近结果表明， 所提算法能得到更小的检测误差，最终网络结构紧凑.
Abstract:
Due to the fact that most algorithms use a greedy strategy in designing artificial neural networks which are susceptible to becoming trapped at the architectural local optimal point, an adaptive algorithm for designing an optimal feedforward neural network was proposed. During the training process of the neural network, the adaptive optimization strategy was adopted to merge and split the hidden unit to design optimal neural network architecture. In the merge operation, the hidden units were merged based on mutual information criterion. In the split operation, a mutation coefficient was introduced to help jump out of locally optimal network. The process of adjusting the connection weight after merge and split operations was combined with the process of training the neural network. Therefore, the number of training samples was reduced, the training speed was increased, and the generalization performance was improved. The results of approximating nonlinear functions show that the proposed algorithm can limit testing errors and a compact neural network structure.
参考文献/References:
［1］邱健斌,王劭伯.进化神经网络PID控制器的研究与应用［J］.智能系统学报, 2008, 3(3): 245249.
QIU Jianbin, WANG Shaobo. An improved PID controller based on an evolutionary neural network［J］. CAAI Transactions on Intelligent Systems, 2008, 3(3): 245249. 
?［2］张昭昭,乔俊飞,韩红桂. 一种基于神经网络复杂度的修剪算法［J］.控制与决策, 2010, 25(6): 178182.
?ZHANG Zhaozhao, QIAO Junfei, HAN Honggui. A pruning algorithm based on neural complexity［J］. Control and Decision, 2010, 25(6): 178182.
［3］乔俊飞,张颖.一种多层前馈神经网络的快速修剪算法［J］.智能系统学报, 2008, 3(2): 173176.
QIAO Junfei, ZHANG Ying. Fast unit pruning algorithm for multilayer feedforward network design［J］. CAAI Transactions on Intelligent Systems, 2008, 3(2): 173176.
［4］ MA L，KHORASANI K. Constructive feedforward neural networks using Hermite poly nomial activation function［J］. IEEE Transactions on Neural Network, 2005, 16(4): 821833.
［5］ISLAM Monirual, SATTAR A, AMIN F, YAO Xin, MURASE K. A new adaptive merging and growing algorithm for designing artificial neural networks［J］. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, 2009, 39(3): 705722.
［6］吴晓刚， 王旭东， 余腾伟.发动机输出转矩的改进BP神经网络估计［J］. 电机与控制学报, 2010, 14(3): 104108.
?WU Xiaogang， WANG Xudong， YU Tengwei. Estimation of engine output torque based on improved BP neural network［J］. Electric Machines and Control, 2010, 14(3): 104108.
［7］宋勇，李贻斌，李彩虹. 递归神经网络的进化机器人路径规划方法［J］. 哈尔滨工程大学学报, 2009, 30(8): 898902.
?SONG Yong， LI Yibin, LI Caihong. Path planning based on a recurrent neural network for an evolutionary robot［J］. Journal of Harbin Engineering University, 2009, 30(8): 898902.
［8］陆瑶，张杰，冯英浚. 非线性动态系统的模糊神经网络自适应H∞鲁棒控制［J］. 哈尔滨工程大学学报, 2009, 30(9): 10821086.
LU Yao, ZHANG Jie, FENG Yingjun. H∞ robust adaptive control of a fuzzy neural network based nonlinear dynamic system［J］. Journal of Harbin Engineering University, 2009, 30(9): 10821086.
［9］罗耀华, 从静. 基于BP神经网络的三相逆变器故障诊断研究［J］. 应用科技, 2010, 37(6): 5660.
LUO Yaohua, CONG Jing. Fault diagnosis of threephase inverter using BP neural network［J］. Applied Science and Technology, 2010, 37(6): 5660.
?［10］KRASKOV A, STOGBAUER H, GRASSBERGER P. Estimating mutual information［J］. Phys Rev E, Sta Plasmas Fluids Relat Interdiscip Top， 2004, 69( 0661138): 116. 
［11］HONG Jie， HU Baogang. Twophase construction of multilayer perceptions using information theory［J］. IEEE Transactions on Neural Network, 2009, 20(4): 542550.
?［12］LIU Yinyin, STARZYK J A, ZHU Zhen. Optimized approximation algorithm in neural networks without overfitting［J］. IEEE Transactions on Neural Network, 2008, 19(6): 983995.
［13］HASSIBI B, STORK D, WOLFF G, WATANABE T. Optimal brain surgeon: extensions and performance comparisons［C］//Adavances in Neural Informati on Processing Systems 6. San Mateo, USA: Morgan Kaufman, 1994: 263270.
?［14］FAHLMAN S E, LEBIERE C. The cascade correlation learning architecture［C］//Advances in Neural Information Processing Systems 2. San Mateo, USA: Morgan Kaufman, 1990: 524532. 
相似文献/References:
[1]乔俊飞,李凡军,杨翠丽.随机权神经网络研究现状与展望[J].智能系统学报,2016,11(6):758.[doi:10.11992/tis.201612015]
　QIAO Junfei,LI Fanjun,YANG Cuili.Review and prospect on neural networks with random weights[J].CAAI Transactions on Intelligent Systems,2016,11():758.[doi:10.11992/tis.201612015]
[2]乔俊飞,安茹,韩红桂.基于相对贡献指标的自组织RBF神经网络的设计[J].智能系统学报,2018,13(2):159.[doi:10.11992/tis.201608009]
　QIAO Junfei,AN Ru,HAN Honggui.Design of self-organizing RBF neural network based on relative contribution index[J].CAAI Transactions on Intelligent Systems,2018,13():159.[doi:10.11992/tis.201608009]
[3]蒙西,乔俊飞,李文静.基于快速密度聚类的RBF神经网络设计[J].智能系统学报,2018,13(3):331.[doi:10.11992/tis.201702014]
　MENG Xi,QIAO Junfei,LI Wenjing.Construction of RBF neural networks via fast density clustering[J].CAAI Transactions on Intelligent Systems,2018,13():331.[doi:10.11992/tis.201702014]

备注/Memo

收稿日期：
基金项目：国家自然科学基金资助项目（60873043）；国家“863”计划资助项目（2009AA04Z155）；北京市自然科学基金资助项目（4092010）；教育部博士点基金资助项目（200800050004）.
通信作者：张昭昭.E-mail:zzzhao123@126.com.
作者简介：
张昭昭，男，1973年生，博士研究生，主要研究方向为智能系统与智能信息处理、神经网络结构设计与优化.
乔俊飞，男，1968年生，教授，博士生导师，主要研究方向为复杂过程建模与控制、计算智能与智能优化控制，发表学术论文100余篇，其中被SCI、EI检索60余篇. 杨刚，男，1983年生，博士研究生，主要研究方向为神经计算与智能优化控制.

更新日期/Last Update: 2011-09-30
Copyright © 《 智能系统学报》 编辑部
地址：（150001）黑龙江省哈尔滨市南岗区南通大街145-1号楼 电话：0451- 82534001、82518134　邮箱：tis@vip.sina.com