[1]LI Xiangyu,SUI Lin,XIONG Weili.Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network[J].CAAI Transactions on Intelligent Systems,2023,18(5):957-965.[doi:10.11992/tis.202211037]
Copy

Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 957-965 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network
Author(s):
LI Xiangyu1;  SUI Lin1;  XIONG Weili1; 2
1. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China;
2. Key Laboratory of Advanced Process Control for Industry(Ministry of Education), Jiangnan University, Wuxi 214122, China
Keywords:
self-attention mechanism; ordered neurons long short-term memory; soft sensor; penicillin fermentation; feature extraction; convolution; redundant information; deep learning
CLC:
TP274
DOI:
10.11992/tis.202211037
Abstract:
According to the nonlinear and dynamic characteristics of actual industrial processes and considering the redundant information in process variables, this paper presents a multilayer time-series prediction model of convolutional ordered neurons long short-term memory network (ONLSTM) with a self-attention mechanism. First, the convolutional neural network is used to reduce the dimensions of local features, extract the specific local features of the input variables, and rank the neurons in the LSTM hidden layer specifically by constructing the hierarchical importance index to identify the hierarchical structure information and improve the ability of networks to judge important information of the network model. Second, the self-attention mechanism is introduced into the ONLSTM network. This mechanism dynamically assigns different attention weights to the input variables according to their internal correlation to improve the prediction performance of the model. Finally, the model is applied to predict product concentration in the penicillin fermentation process, following which it is compared with other advanced network models to verify the effectiveness of the proposed model.
References:
[1] KADLEC P, GABRYS B, STRANDT S. Data-driven soft sensors in the process industry[J]. Computers & chemical engineering, 2009, 33(4): 795-814.
[2] YUAN Xiaofeng, WANG Yalin, YANG Chunhua, et al. Weighted linear dynamic system for feature representation and soft sensor application in nonlinear dynamic industrial processes[J]. IEEE transactions on industrial electronics, 2018, 65(2): 1508-1517.
[3] WANG Jie, ZHAO Chunhui. Mode-cloud data analytics based transfer learning for soft sensor of manufacturing industry with incremental learning ability[J]. Control engineering practice, 2020, 98: 104392.
[4] 祁成, 史旭东, 熊伟丽. 基于二阶相似度的即时学习软测量建模方法[J]. 智能系统学报, 2020, 15(5): 910-918
QI Cheng, SHI Xudong, XIONG Weili. A just-in-time learning soft sensor modeling method based on the second-order similarity[J]. CAAI transactions on intelligent systems, 2020, 15(5): 910-918
[5] ZHOU Ping. Autoencoder and PCA based RVFLNs modeling for multivariate molten iron quality in blast furnace ironmaking[J]. Zidonghua Xuebao/acta automatica sinica, 2018, 44(10): 1799-1811.
[6] WANG Zi xiu, HE Q P, WANG Jin. Comparison of variable selection methods for PLS-based soft sensor modeling[J]. Journal of process control, 2015, 26: 56-72.
[7] SHENG Xiaochen, MA Junxia, XIONG Weili. Smart soft sensor design with hierarchical sampling strategy of ensemble Gaussian process regression for fermentation processes[J]. Sensors, 2020, 20(7): 1957.
[8] 赵超, 李俊, 戴坤成, 等. 基于自适应加权最小二乘支持向量机的青霉素发酵过程软测量建模[J]. 南京理工大学学报, 2017, 41(1): 100-107
ZHAO Chao, LI Jun, DAI Kuncheng, et al. Soft sensor modeling for penicillin fermentation process based on adaptive weighted least squares support vector machine[J]. Journal of Nanjing University of Science and Technology, 2017, 41(1): 100-107
[9] SUN Kai, HUANG S H, WONG D S H, et al. Design and application of a variable selection method for multilayer perceptron neural network with LASSO[J]. IEEE transactions on neural networks and learning systems, 2017, 28(6): 1386-1396.
[10] SUN Qingqiang, GE Zhiqiang. A survey on deep learning for data-driven soft sensors[J]. IEEE transactions on industrial informatics, 2021, 17(9): 5853-5866.
[11] SHANG Chao, YANG Fan, HUANG Dexian, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of process control, 2014, 24(3): 223-233.
[12] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[13] YUAN Xiaofeng, LI Lin, WANG Yalin. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE transactions on industrial informatics, 2020, 16(5): 3168-3176.
[14] ZHENG Jiaqi, MA Lianwei, WU Yi, et al. Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes[J]. ACS omega, 2022, 7(19): 16653-16664.
[15] 常树超, 赵春晖. 一种时空协同的图卷积长短期记忆网络及其工业软测量应用[J]. 控制与决策, 2022, 37(1): 77-86
CHANG Shuchao, ZHAO Chunhui. A spatio-temporal synergistic graph convolution long short-term memory network and its application for industrial soft sensors[J]. Control and decision, 2022, 37(1): 77-86
[16] 孙凯, 隋璘, 张芳芳, 等. 基于非负绞杀与长短期记忆神经网络的动态软测量算法[J]. 控制理论与应用, 2023, 40(1): 83-93
SUN Kai, SUI Lin, ZHANG Fangfang, et al. Dynamic soft sensor algorithm based on nonnegative garrote and long short-term memory neural network[J]. Control theory & applications, 2023, 40(1): 83-93
[17] LUI C F, LIU Yiqi, XIE Min. A supervised bidirectional long short-term memory network for data-driven dynamic soft sensor modeling[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 1-13.
[18] XIE Ruimin, HAO Kuangrong, HUANG Biao, et al. Data-driven modeling based on two-stream λ gated recurrent unit network with soft sensor application[J]. IEEE transactions on industrial electronics, 2020, 67(8): 7034-7043.
[19] SHEN YIKANG, TAN S, SORDONI A, et al. Ordered neurons: integrating tree structures into recurrent neural networks[EB/OL]. (2019?05?08)[2022?01?01]. https://arxiv.org/abs/1810.09536.
[20] SHI Fei, CAO Hongrui, WANG Yuke, et al. Chatter detection in high-speed milling processes based on ON-LSTM and PBT[J]. The international journal of advanced manufacturing technology, 2020, 111(11): 3361-3378.
[21] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. (2016?05?19)[2022?01?01]. https://arxiv.org/abs/1409.0473.
[22] YUAN Xiaofeng, LI Lin, SHARDT Y A W, et al. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development[J]. IEEE transactions on industrial electronics, 2021, 68(5): 4404-4414.
[23] LI Lin, WANG Yalin, YUAN Xiaofeng, et al. Quality prediction model for process sequential data of irregular measurements with sampling-interval-attention LSTM[C]//2020 Chinese Automation Congress. Shanghai: IEEE, 2021: 7186?7191.
[24] ZHU Xiuli, HAO Kuangrong, XIE Ruimin, et al. Soft sensor based on eXtreme gradient boosting and bidirectional converted gates long short-term memory self-attention network[J]. Neurocomputing, 2021, 434: 126-136.
[25] GOPAKUMAR V, TIWARI S, RAHMAN I. A deep learning based data driven soft sensor for bioprocesses[J]. Biochemical engineering journal, 2018, 136: 28-39.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems