[1]ZHANG Zhaozhao,QIAO Junfei,YANG Gang.An adaptive algorithm for designingoptimal feedforward neural network architecture[J].CAAI Transactions on Intelligent Systems,2011,6(4):312-317.
Copy

An adaptive algorithm for designingoptimal feedforward neural network architecture

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 6 Number of periods: 2011 4 Page number: 312-317 Column: 学术论文—机器学习 Public date: 2011-08-25
Title:
An adaptive algorithm for designingoptimal feedforward neural network architecture
Author(s):
ZHANG Zhaozhao1; 2;  QIAO Junfei1;  YANG Gang1
1. College of Electronic and Control Engineering, Beijing University of Technology, Beijing 100124, China;
2. Institute of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China
Keywords:
feedforward neural network;  architecture design;  adaptive search strategy;  mutual information
CLC:
TP273
DOI:
-
Abstract:
Due to the fact that most algorithms use a greedy strategy in designing artificial neural networks which are susceptible to becoming trapped at the architectural local optimal point, an adaptive algorithm for designing an optimal feedforward neural network was proposed. During the training process of the neural network, the adaptive optimization strategy was adopted to merge and split the hidden unit to design optimal neural network architecture. In the merge operation, the hidden units were merged based on mutual information criterion. In the split operation, a mutation coefficient was introduced to help jump out of locally optimal network. The process of adjusting the connection weight after merge and split operations was combined with the process of training the neural network. Therefore, the number of training samples was reduced, the training speed was increased, and the generalization performance was improved. The results of approximating nonlinear functions show that the proposed algorithm can limit testing errors and a compact neural network structure.
References:
［1］邱健斌,王劭伯.进化神经网络PID控制器的研究与应用［J］.智能系统学报, 2008, 3(3): 245249.
QIU Jianbin, WANG Shaobo. An improved PID controller based on an evolutionary neural network［J］. CAAI Transactions on Intelligent Systems, 2008, 3(3): 245249. 
?［2］张昭昭,乔俊飞,韩红桂. 一种基于神经网络复杂度的修剪算法［J］.控制与决策, 2010, 25(6): 178182.
?ZHANG Zhaozhao, QIAO Junfei, HAN Honggui. A pruning algorithm based on neural complexity［J］. Control and Decision, 2010, 25(6): 178182.
［3］乔俊飞,张颖.一种多层前馈神经网络的快速修剪算法［J］.智能系统学报, 2008, 3(2): 173176.
QIAO Junfei, ZHANG Ying. Fast unit pruning algorithm for multilayer feedforward network design［J］. CAAI Transactions on Intelligent Systems, 2008, 3(2): 173176.
［4］ MA L，KHORASANI K. Constructive feedforward neural networks using Hermite poly nomial activation function［J］. IEEE Transactions on Neural Network, 2005, 16(4): 821833.
［5］ISLAM Monirual, SATTAR A, AMIN F, YAO Xin, MURASE K. A new adaptive merging and growing algorithm for designing artificial neural networks［J］. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, 2009, 39(3): 705722.
［6］吴晓刚， 王旭东， 余腾伟.发动机输出转矩的改进BP神经网络估计［J］. 电机与控制学报, 2010, 14(3): 104108.
?WU Xiaogang， WANG Xudong， YU Tengwei. Estimation of engine output torque based on improved BP neural network［J］. Electric Machines and Control, 2010, 14(3): 104108.
［7］宋勇，李贻斌，李彩虹. 递归神经网络的进化机器人路径规划方法［J］. 哈尔滨工程大学学报, 2009, 30(8): 898902.
?SONG Yong， LI Yibin, LI Caihong. Path planning based on a recurrent neural network for an evolutionary robot［J］. Journal of Harbin Engineering University, 2009, 30(8): 898902.
［8］陆瑶，张杰，冯英浚. 非线性动态系统的模糊神经网络自适应H∞鲁棒控制［J］. 哈尔滨工程大学学报, 2009, 30(9): 10821086.
LU Yao, ZHANG Jie, FENG Yingjun. H∞ robust adaptive control of a fuzzy neural network based nonlinear dynamic system［J］. Journal of Harbin Engineering University, 2009, 30(9): 10821086.
［9］罗耀华, 从静. 基于BP神经网络的三相逆变器故障诊断研究［J］. 应用科技, 2010, 37(6): 5660.
LUO Yaohua, CONG Jing. Fault diagnosis of threephase inverter using BP neural network［J］. Applied Science and Technology, 2010, 37(6): 5660.
?［10］KRASKOV A, STOGBAUER H, GRASSBERGER P. Estimating mutual information［J］. Phys Rev E, Sta Plasmas Fluids Relat Interdiscip Top， 2004, 69( 0661138): 116. 
［11］HONG Jie， HU Baogang. Twophase construction of multilayer perceptions using information theory［J］. IEEE Transactions on Neural Network, 2009, 20(4): 542550.
?［12］LIU Yinyin, STARZYK J A, ZHU Zhen. Optimized approximation algorithm in neural networks without overfitting［J］. IEEE Transactions on Neural Network, 2008, 19(6): 983995.
［13］HASSIBI B, STORK D, WOLFF G, WATANABE T. Optimal brain surgeon: extensions and performance comparisons［C］//Adavances in Neural Informati on Processing Systems 6. San Mateo, USA: Morgan Kaufman, 1994: 263270.
?［14］FAHLMAN S E, LEBIERE C. The cascade correlation learning architecture［C］//Advances in Neural Information Processing Systems 2. San Mateo, USA: Morgan Kaufman, 1990: 524532. 
Similar References:

Memo

-

Last Update: 2011-09-30

Copyright © CAAI Transactions on Intelligent Systems