[1]汤礼颖,贺利乐,何林,等.一种卷积神经网络集成的多样性度量方法[J].智能系统学报,2021,16(6):1030-1038.[doi:10.11992/tis.202011023]
TANG Liying,HE Lile,HE Lin,et al.Diversity measuring method of a convolutional neural network ensemble[J].CAAI Transactions on Intelligent Systems,2021,16(6):1030-1038.[doi:10.11992/tis.202011023]
点击复制
TANG Liying,HE Lile,HE Lin,et al.Diversity measuring method of a convolutional neural network ensemble[J].CAAI Transactions on Intelligent Systems,2021,16(6):1030-1038.[doi:10.11992/tis.202011023]
一种卷积神经网络集成的多样性度量方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第6期
页码:
1030-1038
栏目:
学术论文—机器感知与模式识别
出版日期:
2021-11-05
- Title:
- Diversity measuring method of a convolutional neural network ensemble
- Keywords:
- CNN; ensemble learning; diversity measures; machine learning; multiple classifier ensembles; probability vector outputs; Oracle outputs; basic model
- 分类号:
- TP181;TP391
- 摘要:
- 分类器模型之间的多样性是分类器集成的一个重要性能指标。目前大多数多样性度量方法都是基于基分类器模型的0/1输出结果（即Oracle 输出）进行计算，针对卷积神经网络的概率向量输出结果，仍需要将其转化为Oracle输出方式进行度量，这种方式未能充分利用卷积神经网络输出的概率向量所包含的丰富信息。针对此问题，利用多分类卷积神经网络模型的输出特性，提出了一种基于卷积神经网络的概率向量输出方式的集成多样性度量方法，建立多个不同结构的卷积神经网络基模型并在CIFAR-10和CIFAR-100数据集上进行实验。实验结果表明，与双错度量、不一致性度量和Q统计多样性度量方法相比，所提出的方法能够更好地体现模型之间的多样性，为模型选择集成提供更好的指导。
- Abstract:
- Diversity among classifier models has been recognized as a significant performance index of a classifier ensemble. Currently, most diversity measuring methods are defined based on the 0/1 outputs (namely Oracle outputs) of the base model. The probability vector outputs of a convolutional neural network (CNN) still need to be converted into Oracle outputs for measurement, which fails to fully use the rich information contained in the CNN probability vector outputs. To solve this problem, a new diversity measuring method for probabilistic vector outputs based on CNNs is proposed. Several base models of CNN models with various structures are established and tested on the CIFAR-10 and CIFAR-100 datasets. Compared with double-fault measure, disagreement measure, and Q-Statistic, the proposed method can better reflect the differences between the models and provide better guidance for a selective ensemble of CNN models.
- 参考文献/References:
-
[1] OPITZ D, MACLIN R. Popular ensemble methods: an empirical study[J]. Journal of artificial intelligence research, 1999, 11: 169-198.
[2] ZHOU Zhuhui. Ensemble methods: foundations and algorithms[M]. New York: CRC Press, 2012: 236.
[3] YULE G U. On the association of attributes in statistics: with illustrations from the material of the childhood society, &c[J]. Philosophical transactions of the royal society of London. Series A, 1900, 1900, 194: 257-319.
[4] SKALAK D B. The sources of increased accuracy for two proposed boosting algorithms[C]//Proceedings of American Association for Artificial Intelligence, AAAI-96, Integrating Multiple Learned Models Workshop. Portland, USA, 1996: 1133.
[5] GIACINTO G, ROLI F. Design of effective neural network ensembles for image classification purposes[J]. Image and vision computing, 2001, 19(9/10): 699-707.
[6] KUNCHEVA L I, WHITAKER C J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy[J]. Machine learning, 2003, 51(2): 181-207.
[7] KOHAVI R, WOLPERT D H. Bias plus variance decomposition for zero-one loss functions[C]//Proceedings of the 13th International Conference on Machine Learning. San Francisco, USA, 1996: 275-283.
[8] CONOVER W J. Statistical methods for rates and proportions[J]. Technometrics, 1974, 16(2): 326-327.
[9] SHIPP C A, KUNCHEVA L I. Relationships between combination methods and measures of diversity in combining classifiers[J]. Information fusion, 2002, 3(2): 135-148.
[10] HANSEN L K, SALAMON P. Neural network ensembles[J]. IEEE transactions on pattern analysis and machine intelligence, 2002, 12(10): 993-1001.
[11] CUNNINGHAM P, CARNEY J. Diversity versus quality in classification ensembles based on feature selection[C]//Proceedings of the 11th European Conference on Machine Learning. Catalonia, Spain, 2000: 109-116.
[12] PARTRIDGE D, KRZANOWSKI W. Software diversity: practical statistics for its measurement and exploitation[J]. Information and software technology, 1997, 39(10): 707-717.
[13] 邢红杰, 魏勇乐. 基于相关熵和距离方差的支持向量数据描述选择性集成[J]. 计算机科学, 2016, 43(5): 252-256, 264
XING Hongjie, WEI Yongle. Selective ensemble of SVDDs based on correntropy and distance variance[J]. Computer science, 2016, 43(5): 252-256, 264
[14] 李莉. 基于差异性度量的分类器集成优化方法研究与应用[D]. 大连: 大连海事大学, 2017.
LI Li. Optimization method research and application of multiple classifiers ensemble based on diversity measure[D]. Dalian: Dalian Maritime University, 2017.
[15] 赵军阳, 韩崇昭, 韩德强, 等. 采用互补信息熵的分类器集成差异性度量方法[J]. 西安交通大学学报, 2016, 50(2): 13-19
ZHAO Junyang, Han Chongzhao, Han Deqiang, et al. A novel measure method for diversity of classifier integrations using complement informationentropy[J]. Journal of Xi’an Jiaotong University, 2016, 50(2): 13-19
[16] 周钢, 郭福亮. 基于信息熵的集成学习过程多样性度量研究[J]. 计算机工程与科学, 2019, 41(9): 1700-1707
ZHOU Gang, GUO Fuliang. Process diversity measurement of ensemble learning based on information entropy[J]. Computer engineering and science, 2019, 41(9): 1700-1707
[17] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251
ZHOU Feiyan, JIN Linpeng, DONG Jun. DONG Jun. Review of convolutional neural network[J]. Chinese journal of computers, 2017, 40(6): 1229-1251
[18] FAN Tiegang, ZHU Ying, CHEN Junmin. A new measure of classifier diversity in multiple classifier system[C]//Proceedings of 2008 International Conference on Machine Learning and Cybernetics. Kunming, China, 2008.
[19] 常亮, 邓小明, 周明全, 等. 图像理解中的卷积神经网络[J]. 自动化学报, 2016, 42(9): 1300-1312
CHANG Liang, DENG Xiaoming, ZHOU Mingquan, et al. Convolutional neural networks in image understanding[J]. Acta Automatica Sinica, 2016, 42(9): 1300-1312
[20] KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images[J]. Handbook of systemic autoimmune diseases, 2009, 1(4): 7.
[21] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA, 2012.
[22] SINHA N K, GRISCIK M P. A stochastic approximation method[J]. IEEE transactions on systems, man, and cybernetics, 2007, SMC-1(4): 338-344.
[23] GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[J]. Journal of machine learning research, 2011, 15: 315-323.
- 相似文献/References:
-
[1]殷瑞,苏松志,李绍滋.一种卷积神经网络的图像矩正则化策略[J].智能系统学报,2016,11(1):43.[doi:10.11992/tis.201509018]
YIN Rui,SU Songzhi,LI Shaozi.Convolutional neural network’s image moment regularizing strategy[J].CAAI Transactions on Intelligent Systems,2016,11():43.[doi:10.11992/tis.201509018]
[2]胡小生,温菊屏,钟勇.动态平衡采样的不平衡数据集成分类方法[J].智能系统学报,2016,11(2):257.[doi:10.11992/tis.201507015]
HU Xiaosheng,WEN Juping,ZHONG Yong.Imbalanced data ensemble classification using dynamic balance sampling[J].CAAI Transactions on Intelligent Systems,2016,11():257.[doi:10.11992/tis.201507015]
[3]龚震霆,陈光喜,任夏荔,等.基于卷积神经网络和哈希编码的图像检索方法[J].智能系统学报,2016,11(3):391.[doi:10.11992/tis.201603028]
GONG Zhenting,CHEN Guangxi,REN Xiali,et al.An image retrieval method based on a convolutional neural network and hash coding[J].CAAI Transactions on Intelligent Systems,2016,11():391.[doi:10.11992/tis.201603028]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]师亚亭,李卫军,宁欣,等.基于嘴巴状态约束的人脸特征点定位算法[J].智能系统学报,2016,11(5):578.[doi:10.11992/tis.201602006]
SHI Yating,LI Weijun,NING Xin,et al.A facial feature point locating algorithmbased on mouth-state constraints[J].CAAI Transactions on Intelligent Systems,2016,11():578.[doi:10.11992/tis.201602006]
[6]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[7]杨晓兰,强彦,赵涓涓,等.基于医学征象和卷积神经网络的肺结节CT图像哈希检索[J].智能系统学报,2017,12(6):857.[doi:10.11992/tis.201706035]
YANG Xiaolan,QIANG Yan,ZHAO Juanjuan,et al.Hashing retrieval for CT images of pulmonary nodules based on medical signs and convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2017,12():857.[doi:10.11992/tis.201706035]
[8]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[9]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[10]王成济,罗志明,钟准,等.一种多层特征融合的人脸检测方法[J].智能系统学报,2018,13(1):138.[doi:10.11992/tis.201707018]
WANG Chengji,LUO Zhiming,ZHONG Zhun,et al.Face detection method fusing multi-layer features[J].CAAI Transactions on Intelligent Systems,2018,13():138.[doi:10.11992/tis.201707018]
备注/Memo
收稿日期:2020-11-20。
基金项目:国家自然科学基金项目（61903291）
作者简介:汤礼颖，硕士研究生，主要研究方向为图像识别与目标检测;贺利乐，教授，博士生导师，主要研究方向为机器人智能化技术、机器学习。2015年获陕西省高等学校科学技术奖二等奖，2016年获陕西省科学技术奖三等奖。获发明专利授权5件，出版专著1部，教材4部，发表学术论文86篇;何林，讲师，主要研究方向为深度学习
通讯作者:贺利乐.E-mail:hllnh2013@163.com
更新日期/Last Update:
2021-12-25