[1]TANG Liying,HE Lile,HE Lin,et al.Diversity measuring method of a convolutional neural network ensemble[J].CAAI Transactions on Intelligent Systems,2021,16(6):1030-1038.[doi:10.11992/tis.202011023]
Copy
Diversity measuring method of a convolutional neural network ensemble
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
16
Number of periods:
2021 6
Page number:
1030-1038
Column:
学术论文—机器感知与模式识别
Public date:
2021-11-05
- Title:
- Diversity measuring method of a convolutional neural network ensemble
- Keywords:
- CNN; ensemble learning; diversity measures; machine learning; multiple classifier ensembles; probability vector outputs; Oracle outputs; basic model
- CLC:
- TP181;TP391
- DOI:
- 10.11992/tis.202011023
- Abstract:
- Diversity among classifier models has been recognized as a significant performance index of a classifier ensemble. Currently, most diversity measuring methods are defined based on the 0/1 outputs (namely Oracle outputs) of the base model. The probability vector outputs of a convolutional neural network (CNN) still need to be converted into Oracle outputs for measurement, which fails to fully use the rich information contained in the CNN probability vector outputs. To solve this problem, a new diversity measuring method for probabilistic vector outputs based on CNNs is proposed. Several base models of CNN models with various structures are established and tested on the CIFAR-10 and CIFAR-100 datasets. Compared with double-fault measure, disagreement measure, and Q-Statistic, the proposed method can better reflect the differences between the models and provide better guidance for a selective ensemble of CNN models.
- References:
-
[1] OPITZ D, MACLIN R. Popular ensemble methods: an empirical study[J]. Journal of artificial intelligence research, 1999, 11: 169-198.
[2] ZHOU Zhuhui. Ensemble methods: foundations and algorithms[M]. New York: CRC Press, 2012: 236.
[3] YULE G U. On the association of attributes in statistics: with illustrations from the material of the childhood society, &c[J]. Philosophical transactions of the royal society of London. Series A, 1900, 1900, 194: 257-319.
[4] SKALAK D B. The sources of increased accuracy for two proposed boosting algorithms[C]//Proceedings of American Association for Artificial Intelligence, AAAI-96, Integrating Multiple Learned Models Workshop. Portland, USA, 1996: 1133.
[5] GIACINTO G, ROLI F. Design of effective neural network ensembles for image classification purposes[J]. Image and vision computing, 2001, 19(9/10): 699-707.
[6] KUNCHEVA L I, WHITAKER C J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy[J]. Machine learning, 2003, 51(2): 181-207.
[7] KOHAVI R, WOLPERT D H. Bias plus variance decomposition for zero-one loss functions[C]//Proceedings of the 13th International Conference on Machine Learning. San Francisco, USA, 1996: 275-283.
[8] CONOVER W J. Statistical methods for rates and proportions[J]. Technometrics, 1974, 16(2): 326-327.
[9] SHIPP C A, KUNCHEVA L I. Relationships between combination methods and measures of diversity in combining classifiers[J]. Information fusion, 2002, 3(2): 135-148.
[10] HANSEN L K, SALAMON P. Neural network ensembles[J]. IEEE transactions on pattern analysis and machine intelligence, 2002, 12(10): 993-1001.
[11] CUNNINGHAM P, CARNEY J. Diversity versus quality in classification ensembles based on feature selection[C]//Proceedings of the 11th European Conference on Machine Learning. Catalonia, Spain, 2000: 109-116.
[12] PARTRIDGE D, KRZANOWSKI W. Software diversity: practical statistics for its measurement and exploitation[J]. Information and software technology, 1997, 39(10): 707-717.
[13] 邢红杰, 魏勇乐. 基于相关熵和距离方差的支持向量数据描述选择性集成[J]. 计算机科学, 2016, 43(5): 252-256, 264
XING Hongjie, WEI Yongle. Selective ensemble of SVDDs based on correntropy and distance variance[J]. Computer science, 2016, 43(5): 252-256, 264
[14] 李莉. 基于差异性度量的分类器集成优化方法研究与应用[D]. 大连: 大连海事大学, 2017.
LI Li. Optimization method research and application of multiple classifiers ensemble based on diversity measure[D]. Dalian: Dalian Maritime University, 2017.
[15] 赵军阳, 韩崇昭, 韩德强, 等. 采用互补信息熵的分类器集成差异性度量方法[J]. 西安交通大学学报, 2016, 50(2): 13-19
ZHAO Junyang, Han Chongzhao, Han Deqiang, et al. A novel measure method for diversity of classifier integrations using complement informationentropy[J]. Journal of Xi’an Jiaotong University, 2016, 50(2): 13-19
[16] 周钢, 郭福亮. 基于信息熵的集成学习过程多样性度量研究[J]. 计算机工程与科学, 2019, 41(9): 1700-1707
ZHOU Gang, GUO Fuliang. Process diversity measurement of ensemble learning based on information entropy[J]. Computer engineering and science, 2019, 41(9): 1700-1707
[17] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251
ZHOU Feiyan, JIN Linpeng, DONG Jun. DONG Jun. Review of convolutional neural network[J]. Chinese journal of computers, 2017, 40(6): 1229-1251
[18] FAN Tiegang, ZHU Ying, CHEN Junmin. A new measure of classifier diversity in multiple classifier system[C]//Proceedings of 2008 International Conference on Machine Learning and Cybernetics. Kunming, China, 2008.
[19] 常亮, 邓小明, 周明全, 等. 图像理解中的卷积神经网络[J]. 自动化学报, 2016, 42(9): 1300-1312
CHANG Liang, DENG Xiaoming, ZHOU Mingquan, et al. Convolutional neural networks in image understanding[J]. Acta Automatica Sinica, 2016, 42(9): 1300-1312
[20] KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images[J]. Handbook of systemic autoimmune diseases, 2009, 1(4): 7.
[21] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA, 2012.
[22] SINHA N K, GRISCIK M P. A stochastic approximation method[J]. IEEE transactions on systems, man, and cybernetics, 2007, SMC-1(4): 338-344.
[23] GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[J]. Journal of machine learning research, 2011, 15: 315-323.
- Similar References:
Memo
-
Last Update:
2021-12-25