[1]李祥宇,隋璘,熊伟丽.基于自注意力机制与卷积ONLSTM网络的软测量算法[J].智能系统学报,2023,18(5):957-965.[doi:10.11992/tis.202211037]
LI Xiangyu,SUI Lin,XIONG Weili.Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network[J].CAAI Transactions on Intelligent Systems,2023,18(5):957-965.[doi:10.11992/tis.202211037]
点击复制
LI Xiangyu,SUI Lin,XIONG Weili.Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network[J].CAAI Transactions on Intelligent Systems,2023,18(5):957-965.[doi:10.11992/tis.202211037]
基于自注意力机制与卷积ONLSTM网络的软测量算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
957-965
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network
- Keywords:
- self-attention mechanism; ordered neurons long short-term memory; soft sensor; penicillin fermentation; feature extraction; convolution; redundant information; deep learning
- 分类号:
- TP274
- 摘要:
- 针对实际工业过程的非线性和动态性特点，并考虑过程变量中存在的冗余信息，提出一种带自注意力机制的卷积有序神经元长短时记忆网络(ordered neurons long short-term memory, ONLSTM)多层时序预测模型。首先利用卷积神经网络降低局部特征维度，对输入变量进行局部特征提取，并通过构建层级重要性指标对长短时记忆网络(long short-term memory, LSTM)隐藏层神经元进行特定排序，以辨识层级结构信息，提高网络模型的重要信息判断能力；其次将自注意力机制引入ONLSTM网络，根据各输入变量之间内部相关性，自适应地为其分配不同的注意力权重，以提高模型预测性能；最后将模型应用于青霉素发酵过程的产物浓度预测，并与其他先进网络模型进行对比，验证了模型的有效性。
- Abstract:
- According to the nonlinear and dynamic characteristics of actual industrial processes and considering the redundant information in process variables, this paper presents a multilayer time-series prediction model of convolutional ordered neurons long short-term memory network (ONLSTM) with a self-attention mechanism. First, the convolutional neural network is used to reduce the dimensions of local features, extract the specific local features of the input variables, and rank the neurons in the LSTM hidden layer specifically by constructing the hierarchical importance index to identify the hierarchical structure information and improve the ability of networks to judge important information of the network model. Second, the self-attention mechanism is introduced into the ONLSTM network. This mechanism dynamically assigns different attention weights to the input variables according to their internal correlation to improve the prediction performance of the model. Finally, the model is applied to predict product concentration in the penicillin fermentation process, following which it is compared with other advanced network models to verify the effectiveness of the proposed model.
- 参考文献/References:
-
[1] KADLEC P, GABRYS B, STRANDT S. Data-driven soft sensors in the process industry[J]. Computers & chemical engineering, 2009, 33(4): 795-814.
[2] YUAN Xiaofeng, WANG Yalin, YANG Chunhua, et al. Weighted linear dynamic system for feature representation and soft sensor application in nonlinear dynamic industrial processes[J]. IEEE transactions on industrial electronics, 2018, 65(2): 1508-1517.
[3] WANG Jie, ZHAO Chunhui. Mode-cloud data analytics based transfer learning for soft sensor of manufacturing industry with incremental learning ability[J]. Control engineering practice, 2020, 98: 104392.
[4] 祁成, 史旭东, 熊伟丽. 基于二阶相似度的即时学习软测量建模方法[J]. 智能系统学报, 2020, 15(5): 910-918
QI Cheng, SHI Xudong, XIONG Weili. A just-in-time learning soft sensor modeling method based on the second-order similarity[J]. CAAI transactions on intelligent systems, 2020, 15(5): 910-918
[5] ZHOU Ping. Autoencoder and PCA based RVFLNs modeling for multivariate molten iron quality in blast furnace ironmaking[J]. Zidonghua Xuebao/acta automatica sinica, 2018, 44(10): 1799-1811.
[6] WANG Zi xiu, HE Q P, WANG Jin. Comparison of variable selection methods for PLS-based soft sensor modeling[J]. Journal of process control, 2015, 26: 56-72.
[7] SHENG Xiaochen, MA Junxia, XIONG Weili. Smart soft sensor design with hierarchical sampling strategy of ensemble Gaussian process regression for fermentation processes[J]. Sensors, 2020, 20(7): 1957.
[8] 赵超, 李俊, 戴坤成, 等. 基于自适应加权最小二乘支持向量机的青霉素发酵过程软测量建模[J]. 南京理工大学学报, 2017, 41(1): 100-107
ZHAO Chao, LI Jun, DAI Kuncheng, et al. Soft sensor modeling for penicillin fermentation process based on adaptive weighted least squares support vector machine[J]. Journal of Nanjing University of Science and Technology, 2017, 41(1): 100-107
[9] SUN Kai, HUANG S H, WONG D S H, et al. Design and application of a variable selection method for multilayer perceptron neural network with LASSO[J]. IEEE transactions on neural networks and learning systems, 2017, 28(6): 1386-1396.
[10] SUN Qingqiang, GE Zhiqiang. A survey on deep learning for data-driven soft sensors[J]. IEEE transactions on industrial informatics, 2021, 17(9): 5853-5866.
[11] SHANG Chao, YANG Fan, HUANG Dexian, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of process control, 2014, 24(3): 223-233.
[12] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[13] YUAN Xiaofeng, LI Lin, WANG Yalin. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE transactions on industrial informatics, 2020, 16(5): 3168-3176.
[14] ZHENG Jiaqi, MA Lianwei, WU Yi, et al. Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes[J]. ACS omega, 2022, 7(19): 16653-16664.
[15] 常树超, 赵春晖. 一种时空协同的图卷积长短期记忆网络及其工业软测量应用[J]. 控制与决策, 2022, 37(1): 77-86
CHANG Shuchao, ZHAO Chunhui. A spatio-temporal synergistic graph convolution long short-term memory network and its application for industrial soft sensors[J]. Control and decision, 2022, 37(1): 77-86
[16] 孙凯, 隋璘, 张芳芳, 等. 基于非负绞杀与长短期记忆神经网络的动态软测量算法[J]. 控制理论与应用, 2023, 40(1): 83-93
SUN Kai, SUI Lin, ZHANG Fangfang, et al. Dynamic soft sensor algorithm based on nonnegative garrote and long short-term memory neural network[J]. Control theory & applications, 2023, 40(1): 83-93
[17] LUI C F, LIU Yiqi, XIE Min. A supervised bidirectional long short-term memory network for data-driven dynamic soft sensor modeling[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 1-13.
[18] XIE Ruimin, HAO Kuangrong, HUANG Biao, et al. Data-driven modeling based on two-stream λ gated recurrent unit network with soft sensor application[J]. IEEE transactions on industrial electronics, 2020, 67(8): 7034-7043.
[19] SHEN YIKANG, TAN S, SORDONI A, et al. Ordered neurons: integrating tree structures into recurrent neural networks[EB/OL]. (2019?05?08)[2022?01?01]. https://arxiv.org/abs/1810.09536.
[20] SHI Fei, CAO Hongrui, WANG Yuke, et al. Chatter detection in high-speed milling processes based on ON-LSTM and PBT[J]. The international journal of advanced manufacturing technology, 2020, 111(11): 3361-3378.
[21] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. (2016?05?19)[2022?01?01]. https://arxiv.org/abs/1409.0473.
[22] YUAN Xiaofeng, LI Lin, SHARDT Y A W, et al. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development[J]. IEEE transactions on industrial electronics, 2021, 68(5): 4404-4414.
[23] LI Lin, WANG Yalin, YUAN Xiaofeng, et al. Quality prediction model for process sequential data of irregular measurements with sampling-interval-attention LSTM[C]//2020 Chinese Automation Congress. Shanghai: IEEE, 2021: 7186?7191.
[24] ZHU Xiuli, HAO Kuangrong, XIE Ruimin, et al. Soft sensor based on eXtreme gradient boosting and bidirectional converted gates long short-term memory self-attention network[J]. Neurocomputing, 2021, 434: 126-136.
[25] GOPAKUMAR V, TIWARI S, RAHMAN I. A deep learning based data driven soft sensor for bioprocesses[J]. Biochemical engineering journal, 2018, 136: 28-39.
- 相似文献/References:
-
[1]汪权彬,谭营.基于数据增广和复制的中文语法错误纠正方法[J].智能系统学报,2020,15(1):99.[doi:10.11992/tis.202001014]
WANG Quanbin,TAN Ying.Chinese grammatical error correction method based on data augmentation and copy mechanism[J].CAAI Transactions on Intelligent Systems,2020,15():99.[doi:10.11992/tis.202001014]
[2]毛明毅,吴晨,钟义信,等.加入自注意力机制的BERT命名实体识别模型[J].智能系统学报,2020,15(4):772.[doi:10.11992/tis.202003003]
MAO Mingyi,WU Chen,ZHONG Yixin,et al.BERT named entity recognition model with self-attention mechanism[J].CAAI Transactions on Intelligent Systems,2020,15():772.[doi:10.11992/tis.202003003]
[3]鲍维克,袁春.面向推荐系统的分期序列自注意力网络[J].智能系统学报,2021,16(2):353.[doi:10.11992/tis.202005028]
BAO Weike,YUAN Chun.Recommendation system with long-term and short-term sequential self-attention network[J].CAAI Transactions on Intelligent Systems,2021,16():353.[doi:10.11992/tis.202005028]
[4]石拓,张齐,石磊.多尺度视角特征动态融合的盗窃犯罪预测模型[J].智能系统学报,2022,17(6):1104.[doi:10.11992/tis.202203016]
SHI Tuo,ZHANG Qi,SHI Lei.Prediction model of theft crime based on the dynamic fusion of multiscale perspective characteristics[J].CAAI Transactions on Intelligent Systems,2022,17():1104.[doi:10.11992/tis.202203016]
[5]梁艳,温兴,潘家辉.融合全局与局部特征的跨数据集表情识别方法[J].智能系统学报,2023,18(6):1205.[doi:10.11992/tis.202212030]
LIANG Yan,WEN Xing,PAN Jiahui.Cross-dataset facial expression recognition method fusing global and local features[J].CAAI Transactions on Intelligent Systems,2023,18():1205.[doi:10.11992/tis.202212030]
[6]闫河,刘灵坤,黄俊滨,等.结合多尺度注意力机制和双向门控循环网络的视频摘要模型[J].智能系统学报,2024,19(2):446.[doi:10.11992/tis.202209048]
YAN He,LIU Lingkun,HUANG Junbin,et al.Video summarization model based on the multiscale attention mechanism and bidirectional gated recurrent network[J].CAAI Transactions on Intelligent Systems,2024,19():446.[doi:10.11992/tis.202209048]
备注/Memo
收稿日期:2022-11-15。
基金项目:国家自然科学基金项目(61773182)；国家重点研发计划子课题(2018YFC1603705-03).
作者简介:李祥宇，硕士研究生，主要研究方向为复杂工业过程建模;隋璘，博士研究生, 主要研究方向为机器学习、软测量建模;熊伟丽，教授，博士生导师，主要研究方向为基于数据挖掘、机器学习和大数据解析的复杂工业过程建模、控制及优化，智能软测量技术与应用，以及面向酿造过程、污水处理过程的智能自动化系统设计与开发。主持国家自然科学基金面上项目、国家自然科学基金青年项目、中国博士后基金项目、江苏省产学研前瞻性研究项目等省部级以上项目近10项。作为骨干人员参与完成国家重点研发计划课题、国家863计划等7项。获得江苏省科学技术二等奖1项、中国商业联合会科技进步一等奖、中石化自动化应用协会科技进步一等奖共3项。已授权发明专利27项，其中国际发明专利4项，以第一/责任作者发表学术论文近百篇
通讯作者:熊伟丽.E-mail:greenpre@163.com
更新日期/Last Update:
1900-01-01