[1]YAN Aijun,HAO Chen.Stochastic configuration networks based on mixed distribution weighted M-estimation and adaptive regularization[J].CAAI Transactions on Intelligent Systems,2025,20(6):1392-1403.[doi:10.11992/tis.202501023]
Copy

Stochastic configuration networks based on mixed distribution weighted M-estimation and adaptive regularization

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 6 Page number: 1392-1403 Column: 学术论文—机器学习 Public date: 2025-11-05
Title:
Stochastic configuration networks based on mixed distribution weighted M-estimation and adaptive regularization
Author(s):
YAN Aijun1; 2; 3;  HAO Chen1; 2
1. School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China;
2. Engineering Research Center of Digital Community, Ministry of Education, Beijing 100124, China;
3. Beijing Laboratory for Urban Mass Transit
Keywords:
neural network; stochastic configuration networks; parameter prediction; M-estimation; mixture distribution; L2 regularization; model robustness; model generalization; municipal solid waste incineration
CLC:
TP183
DOI:
10.11992/tis.202501023
Abstract:
To enhance the robustness and generalization capability of stochastic configuration networks (SCNs), this paper proposes a novel SCN modeling method based on mixed-distribution weighted M-estimation and adaptive regularization. First, a Gaussian–Cauchy mixed-distribution weighted M-estimation is employed to determine the penalty weights of training samples, and the output weights of the model are evaluated according to the contribution of each training instance, thereby improving model robustness. Second, an adaptive L2 regularization parameter is assigned based on the variation of modeling residuals and the number of hidden nodes, ensuring that the model maintains good generalization. Finally, the effectiveness of the proposed method is empirically validated using four benchmark datasets and historical data from municipal solid waste incineration processes. Experimental results demonstrate that the parameter models developed using the proposed method outperform comparative methods in both robustness and generalization, thereby broadening the application scope of SCNs.
References:
[1] MAO Chengsheng, YAO Liang, LUO Yuan. ImageGCN: multi-relational image graph convolutional networks for disease identification with chest X-rays[J]. IEEE transactions on medical imaging, 2022, 41(8): 1990-2003.
[2] QIAO Junfei, SUN Zijian, MENG Xi. A comprehensively improved interval type-2 fuzzy neural network for NOx emissions prediction in MSWI process[J]. IEEE transactions on industrial informatics, 2023, 19(11): 11286-11297.
[3] SCARDAPANE S, WANG Dianhui. Randomness in neural networks: an overview[J]. Wiley interdisciplinary reviews: data mining and knowledge discovery, 2017, 7(2): e1200.
[4] 乔俊飞, 李凡军, 杨翠丽. 随机权神经网络研究现状与展望[J]. 智能系统学报, 2016, 11(6): 758-767.
QIAO Junfei, LI Fanjun, YANG Cuili. Review and prospect on neural networks with random weights[J]. CAAI transactions on intelligent systems, 2016, 11(6): 758-767.
[5] PAO Y H, TAKEFUJI Y. Functional-link net computing: theory, system architecture, and functionalities[J]. Computer, 1992, 25(5): 76-79.
[6] LI Ming, WANG Dianhui. Insights into randomized algorithms for neural networks: practical issues and common pitfalls[J]. Information sciences, 2017, 382: 170-178.
[7] WANG Dianhui, LI Ming. Stochastic configuration networks: fundamentals and algorithms[J]. IEEE transactions on cybernetics, 2017, 47(10): 3466-3479.
[8] DAI Wei, ZHOU Xinyu, LI Depeng, et al. Hybrid parallel stochastic configuration networks for industrial data analytics[J]. IEEE transactions on industrial informatics, 2021, 18(4): 2331-2341.
[9] LI Kang, YANG Cuili, WANG Wei, et al. An improved stochastic configuration network for concentration prediction in wastewater treatment process[J]. Information sciences, 2023, 622: 148-160.
[10] EL-MELEGY M T. Model-wise and point-wise random sample consensus for robust regression and outlier detection[J]. Neural networks, 2014, 59: 23-35.
[11] DAI Wei, CHEN Qixin, CHU Fei, et al. Robust regularized random vector functional link network and its industrial application[J]. IEEE access, 2017, 5: 16162-16172.
[12] WANG Dianhui, LI Ming. Robust stochastic configuration networks with kernel density estimation for uncertain data regression[J]. Information sciences, 2017, 412: 210-222.
[13] 李温鹏, 周平. 高炉铁水质量鲁棒正则化随机权神经网络建模[J]. 自动化学报, 2020, 46(4): 721-733.
LI Wenpeng, ZHOU Ping. Robust regularized RVFLNs modeling of molten iron quality in blast furnace ironmaking[J]. Acta automatica sinica, 2020, 46(4): 721-733.
[14] LI Ming, HUANG Changqin, WANG Dianhui. Robust stochastic configuration networks with maximum correntropy criterion for uncertain data regression[J]. Information sciences, 2019, 473: 73-86.
[15] DAI Wei, LI Depeng, CHEN Qixin, et al. Data driven particle size estimation of hematite grinding process using stochastic configuration network with robust technique[J]. Journal of Central South University, 2019, 26(1): 43-62.
[16] ZHOU Ping, LYU Youbin, WANG Hong, et al. Data-driven robust RVFLNs modeling of a blast furnace iron-making process using cauchy distribution weighted M-estimation[J]. IEEE transactions on industrial electronics, 2017, 64(9): 7141-7151.
[17] ZOU Hui, HASTIE T. Regularization and variable selection via the elastic net[J]. Journal of the royal statistical society: series B (statistical methodology), 2005, 67(2): 301-320.
[18] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of machine learning research, 2014, 15: 1929-1958.
[19] 王前进, 代伟, 陆群, 等. 一种随机配置网络软测量模型的稀疏学习方法[J]. 控制与决策, 2022, 37(12): 3171-3182.
WANG Qianjin, DAI Wei, LU Qun, et al. A sparse learning method for SCN soft measurement model[J]. Control and decision, 2022, 37(12): 3171-3182.
[20] ZHAO Lijie, ZOU Shida, HUANG Mingzhong, et al. Distributed regularized stochastic configuration networks via the elastic net[J]. Neural computing and applications, 2021, 33(8): 3281-3297.
[21] 赵立杰, 邹世达, 郭烁, 等. 基于正则化随机配置网络的球磨机工况识别[J]. 控制工程, 2020, 27(1): 1-7.
ZHAO Lijie, ZOU Shida, GUO Shuo, et al. Ball mill load condition recognition model based on regularized stochastic configuration networks[J]. Control engineering of China, 2020, 27(1): 1-7.
[22] FAN Jun, YAN Ailing, XIU Naihua. Asymptotic properties for M-estimators in linear models with dependent random errors[J]. Journal of statistical planning and inference, 2014, 148: 49-66.
[23] ROUSSEEUW P J, CROUX C. Alternatives to the Median absolute deviation[J]. Journal of the American statistical association, 1993, 88(424): 1273-1283.
[24] L?PEZ-RUBIO E, PALOMO E J, DOM?NGUEZ E. Robust self-organization with M-estimators[J]. Neurocomputing, 2015, 151: 408-423.
[25] AGLIARI E, ALEMANNO F, AQUARO M, et al. Regularization, early-stopping and dreaming: a hopfield-like setup to address generalization and overfitting[J]. Neural networks, 2024, 177: 106389.
[26] 张成龙, 丁世飞, 郭丽丽, 等. 随机配置网络研究进展[J]. 软件学报, 2024, 35(5): 2379-2399.
ZHANG Chenglong, DING Shifei, GUO Lili, et al. Research progress on stochastic configuration network[J]. Journal of software, 2024, 35(5): 2379-2399.
[27] WANG Tianzheng, TANG Jian, XIA Heng, et al. Data-driven multi-objective intelligent optimal control of municipal solid waste incineration process[J]. Engineering applications of artificial intelligence, 2024, 137: 109157.
[28] MENG Xi, TANG Jian, QIAO Junfei. NOx emissions prediction with a brain-inspired modular neural network in municipal solid waste incineration processes[J]. IEEE transactions on industrial informatics, 2022, 18(7): 4622-4631.
[29] 汤健, 夏恒, 余文, 等. 城市固废焚烧过程智能优化控制研究现状与展望[J]. 自动化学报, 2023, 49(10): 2019-2059.
TANG Jian, XIA Heng, YU Wen, et al. Research status and prospects of intelligent optimization control for municipal solid waste incineration process[J]. Acta automatica sinica, 2023, 49(10): 2019-2059.
[30] 孙剑, 蒙西, 乔俊飞. 数据驱动的城市固废焚烧过程烟气含氧量预测控制[J]. 控制理论与应用, 2024, 41(3): 484-495.
SUN Jian, MENG Xi, QIAO Junfei. Data-driven predictive control of oxygen content in flue gas for municipal solid waste incineration process[J]. Control theory & applications, 2024, 41(3): 484-495.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems