[1]GUO Yingchen,SUI Lin,XIONG Weili.Sparse ONLSTM and non-negative constrained industrial soft sensing modeling[J].CAAI Transactions on Intelligent Systems,2025,20(6):1366-1378.[doi:10.11992/tis.202502004]
Copy

Sparse ONLSTM and non-negative constrained industrial soft sensing modeling

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 6 Page number: 1366-1378 Column: 学术论文—机器学习 Public date: 2025-11-05
Title:
Sparse ONLSTM and non-negative constrained industrial soft sensing modeling
Author(s):
GUO Yingchen1;  SUI Lin1;  XIONG Weili1; 2
1. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China;
2. Key Laboratory of Advanced Process Control for Industry (Ministry of Education), Jiangnan University, Wuxi 214122, China
Keywords:
soft sensor; long short-term memory networks; ordered neurons; non-negative strangulation; redundant information; variable selection; sparse optimization; deep learning
CLC:
TP274
DOI:
10.11992/tis.202502004
Abstract:
Industrial processes often exhibit characteristics such as multivariable interactions, nonlinear behaviors, and time-varying changes. Thus, the resulting modeling data contain excessive redundant information and complex time-dependent patterns, which increase modeling complexity and degrade the model performance. To address these challenges, an ordered neurons long short-term memory (ONLSTM) network integrated with non-negative garrote-based regularization is proposed herein for industrial soft sensor modeling. First, the shrinkage coefficients of the non-negative garrote are embedded into the weight matrix of the ONLSTM input layer to eliminate redundant input nodes and enable variable selection. Second, these coefficients are integrated into the weight matrix of the ONLSTM hidden layer to assign weights based on the importance of hidden neurons. Consequently, redundant nodes and their corresponding information pathways are pruned, achieving sparse optimization of the network structure. The proposed method is validated via numerical simulations and subsequently employed to predict the SO2 concentration in flue gas emissions from a desulfurization process in a thermal power plant. Experimental results demonstrate that the algorithm effectively selects variables and sparsely optimizes the model structure while maintaining high predictive performance, offering promising prospects for broader industrial applications.
References:
[1] DAI Wei, ZHOU Xinyu, LI Depeng, et al. Hybrid parallel stochastic configuration networks for industrial data analytics[J]. IEEE transactions on industrial informatics, 2022, 18(4): 2331-2341.
[2] YUAN Xiaofeng, WANG Yalin, YANG Chunhua, et al. Weighted linear dynamic system for feature representation and soft sensor application in nonlinear dynamic industrial processes[J]. IEEE transactions on industrial electronics, 2018, 65(2): 1508-1517.
[3] 杜康萍, 隋璘, 熊伟丽. 基于自适应稀疏宽度学习系统的软测量建模[J]. 系统仿真学报, 2025, 37(6): 1449-1461.
DU Kangping, SUI Lin, XIONG Weili. Soft sensor modeling based on adaptive sparse broad learning system[J]. Journal of system simulation, 2025, 37(6): 1449-1461.
[4] CHEN Hongtian, HUANG Biao. Fault-tolerant soft sensors for dynamic systems[J]. IEEE transactions on control systems technology, 2023, 31(6): 2805-2818.
[5] YAO Le, GE Zhiqiang. Industrial big data modeling and monitoring framework for plant-wide processes[J]. IEEE transactions on industrial informatics, 2020, 17(9): 6399-6408.
[6] YUAN Xiaofeng, OU Chen, WANG Yalin, et al. A layer-wise data augmentation strategy for deep learning networks and its soft sensor application in an industrial hydrocracking process[J]. IEEE transactions on neural networks and learning systems, 2019, 32(8): 3296-3305.
[7] YAN Aijun, GUO Jingcheng, WANG Dianhui. Robust stochastic configuration networks for industrial data modelling with Student’s-t mixture distribution[J]. Information sciences, 2022, 607: 493-505.
[8] GAO Yunlong, LIN Tingting, PAN Jinyan, et al. Fuzzy sparse deviation regularized robust principal component analysis[J]. IEEE transactions on image processing, 2022, 31: 5645-5660.
[9] GENG Zhiqiang, CHEN Zhiwei, MENG Qingchao, et al. Novel Transformer based on gated convolutional neural network for dynamic soft sensor modeling of industrial processes[J]. IEEE transactions on industrial informatics, 2022, 18(3): 1521-1529.
[10] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[11] YUAN Xiaofeng, LI Lin, WANG Yalin. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE transactions on industrial informatics, 2019, 16(5): 3168-3176.
[12] SUI Lin, SUN Wenxin, LIU Wentao, et al. A sparse regularized soft sensor based on GRU and self-interpretation double nonnegative garrote: from variable selection to structure optimization[J]. Control engineering practice, 2024, 153: 106074.
[13] 张凯, 王亚礼, 张晓雯, 等. 基于特征融合的粗轧出口温度建模方法与应用[J/OL]. 控制理论与应用, 2024, DOI: 10.7641/CTA.2024.40252.
ZHANG Kai, WANG Yali, ZHANG Xiaowen, et al. A feature fusion-based rough rolling exit temperature modeling method and its applications[J/OL]. Control theory & applications, 2024, DOI: 10.7641/CTA.2024.402 52.
[14] LIU Qingqiang, SHANG Zhiqing, LU Shixiang, et al. Physics-guided TL-LSTM network for early-stage degradation trajectory prediction of lithium-ion batteries[J]. Journal of energy storage, 2025, 106: 114736.
[15] HOGEA E, ONCHI? D M, YAN Ruqiang, et al. LogicLSTM: logically-driven long short-term memory model for fault diagnosis in gearboxes[J]. Journal of manufacturing systems, 2024, 77: 892-902.
[16] YAN Haoran, QIN Yi, XIANG Sheng, et al. Long-term gear life prediction based on ordered neurons LSTM neural networks[J]. Measurement, 2020, 165: 108205.
[17] SHEN Yikang, TAN Shawn, SORDONI A, et al. Ordered neurons: integrating tree structures into recurrent neural networks[C]//Proceedings of the 7th International Conference for Learning Representations. New Orleans: ICLR, 2019.
[18] 李祥宇, 隋璘, 熊伟丽. 基于自注意力机制与卷积ONLSTM网络的软测量算法[J]. 智能系统学报, 2023, 18(5): 957-965.
LI Xiangyu, SUI Lin, XIONG Weili. Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network[J]. CAAI transactions on intelligent systems, 2023, 18(5): 957-965.
[19] XIANG Sheng, QIN Yi, ZHU Caichao, et al. LSTM networks based on attention ordered neurons for gear remaining life prediction[J]. ISA transactions, 2020, 106: 343-354.
[20] 李祥宇, 隋璘, 马君霞, 等. 基于时序迁移与双流加权的ONLSTM软测量建模[J]. 化工学报, 2023, 74(11): 4622-4633.
LI Xiangyu, SUI Lin, MA Junxia, et al. ONLSTM soft sensor modeling based on time series transfer and dual stream weighting[J]. CIESC journal, 2023, 74(11): 4622-4633.
[21] ZHANG Haoran, ZHAO Chunhui. Stable transfer learning-based control: an off-dynamics adaptive approach for unknown nonlinear systems[J]. Neurocomputing, 2025, 616: 128951.
[22] CHIPLUNKAR R, HUANG Biao. Siamese neural network-based supervised slow feature extraction for soft sensor application[J]. IEEE transactions on industrial electronics, 2020, 68(9): 8953-8962.
[23] 刘建伟, 崔立鹏, 刘泽宇, 等. 正则化稀疏模型[J]. 计算机学报, 2015, 38(7): 1307-1325.
LIU jianwei, CUI lipeng, LIU zeyu, et al. Survey on the regularized sparse models[J]. Chinese journal of computers, 2015, 38(7): 1307-1325.
[24] YUAN Ming, LIN Yi. On the non-negative garrote estimator[J]. Journal of the royal statistical society series B: statistical methodology, 2007, 69(2): 143-161.
[25] SUN Kai, LIU Jialin, KANG Jialin, et al. Development of a variable selection method for soft sensor using artificial neural network and nonnegative garrote[J]. Journal of process control, 2014, 24(7): 1068-1075.
[26] WANG Jianguo, JANG Shishang, WONG D S H, et al. Soft-sensor development with adaptive variable selection using nonnegative garrote[J]. Control engineering practice, 2013, 21(9): 1157-1164.
[27] 隋璘, 马君霞, 熊伟丽. 基于注意力绞杀的门控循环单元网络及其工业软测量应用[J/OL]. 控制理论与应用, 2024, DOI: 10.7641/CTA.2024.305 65.
SUI Lin, MA Junxia, XIONG Weili. Gated recurrent unit network based on attention garrote and its application for industrial soft sensors[J/OL]. Control theory & applications, 2024, DOI: 10.7641/CTA.2024.305 65.
[28] SUI Lin, SUN Kai, MA Junxia, et al. Input variable selection and structure optimization for LSTM-based soft sensor with a dual nonnegative garrote approach[J]. IEEE transactions on instrumentation and measurement, 2023, 72: 1-11.
[29] KINGMA D P, BA J . Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference for Learning Representations. San Diego: ICLR, 2015.
[30] FRIEDMAN J H. Multivariate adaptive regression splines[J]. The annals of statistics, 1991, 19(1): 1-67.
[31] YUAN Xiaofeng, LI Lin, SHARDT Y A W, et al. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development[J]. IEEE transactions on industrial electronics, 2020, 68(5): 4404-4414.
[32] SUN Kai, WU Xiuliang, XUE Jingyu, et al. Development of a new multi-layer perceptron based soft sensor for SO2 emissions in power plant[J]. Journal of process control, 2019, 84: 182-191.
[33] 周祖飞, 金新荣. 影响湿法烟气脱硫效率的因素分析[J]. 浙江电力, 2001, 20(3): 42-45.
ZHOU Zufei, JIN Xinrong. Analysis of factors affecting the efficiency of wet flue gas desulfurization[J]. Zhejiang electric power, 2001, 20(3): 42-45.
[34] 钟毅, 高翔, 骆仲泱等. 湿法烟气脱硫系统脱硫效率的影响因素[J]. 浙江大学学报: 工学版, 2008, 42(5): 890-894.
ZHONG Yi, GAO Xiang, LUO Zhongyang, et al. Factors influencing desulfurization efficiency of wet flue gas desulfurization system[J]. Journal of Zhejiang University(engineering science), 2008, 42(5): 890-894.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems