[1]LIU Xiaoyong,YE Zhenhuan.Optimal lower boundary regression model based on double norms l 1-l 1 optimization[J].CAAI Transactions on Intelligent Systems,2020,15(5):934-942.[doi:10.11992/tis.201902006]
Copy

Optimal lower boundary regression model based on double norms l 1-l 1 optimization

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 15 Number of periods: 2020 5 Page number: 934-942 Column: 学术论文—机器感知与模式识别 Public date: 2020-09-05
Title:
Optimal lower boundary regression model based on double norms l 1-l 1 optimization
Author(s):
LIU Xiaoyong;  YE Zhenhuan
College of Engineering, Zunyi Normal University, Zunyi 563006, China
Keywords:
${\ell _1}$-norm-based structural risk minimization; ${\ell _1}$-norm on approximation error; lower boundary regression model; generalization performance; modeling accuracy; optimality; linear programming
CLC:
TP391.1
DOI:
10.11992/tis.201902006
Abstract:
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables. Considering the uncertainties in the structure and parameters of the model derived from sensor measurement data, a new model called optimal lower boundary model is proposed to remove the uncertainties in parameters and characteristics. The proposed method is a combination of structural risk minimization theory (SRM) and some ideas from approximation error minimization. An optimal lower boundary regression model (LBRM) is presented using ${\ell _1} - {\ell _1}$ double norms optimization. First, constraint conditions subjected to LBRM are defined. Then, ${\ell _2}$-norm optimization based on structural risk is converted into simple ${\ell _1}$-norm optimization so that approximation error between the measurements based on ${\ell _1}$-norm is computed and minimized. Next, LBRM is integrated into ${\ell _1}$-norm optimization (based on structural risk). Thus, simpler linear programming can be applied to the constructed double-norms optimization problem to solve parameters of LBRM. Finally, the proposed method is demonstrated by experiments regarding uncertain measurements and parameters of nonlinear system. It has the following prominent features: modeling accuracy of LBRM can be guaranteed by introducing the ${\ell _1}$-norm minimization on approximation error; model’s structural complexity is under control by ${\ell _1}$-norm optimization based on structural risk, thus the performance of the model can be improved further.
References:
[1] HAN Honggui, GUO Yanan, QIAO Junfei. Nonlinear system modeling using a self-organizing recurrent radial basis function neural network[J]. Applied soft computing, 2018, 71: 1105-1116.
[2] FRAVOLINI M L, NAPOLITANO M R, DEL CORE G, et al. Experimental interval models for the robust fault detection of aircraft air data sensors[J]. Control engineering practice, 2018, 78: 196-212.
[3] FANG Shengen, ZHANG Qiuhu, REN Weixin. An interval model updating strategy using interval response surface models[J]. Mechanical systems and signal processing, 2015, 60-61: 909-927.
[4] LEUNG F H F, LAM H K, LING S H, et al. Tuning of the structure and parameters of a neural network using an improved genetic algorithm[J]. IEEE transactions on neural networks, 2003, 14(1): 79-88.
[5] 赵文清, 严海, 王晓辉. BP神经网络和支持向量机相结合的电容器介损角辨识[J]. 智能系统学报, 2019, 14(1): 134-140
ZHAO Wenqing, YAN Hai, WANG Xiaohui. Capacitor dielectric loss angle identification based on a BP neural network and SVM[J]. CAAI transactions on intelligent systems, 2019, 14(1): 134-140
[6] 刘道华, 张礼涛, 曾召霞, 等. 基于正交最小二乘法的径向基神经网络模型[J]. 信阳师范学院学报(自然科学版), 2013, 26(3): 428-431
LIU Daohua, ZHANG Litao, ZENG Zhaoxia, et al. Radial basis function neural network model based on orthogonal least squares[J]. Journal of Xinyang Normal University (natural science edition), 2013, 26(3): 428-431
[7] 刘道华, 张飞, 张言言. 一种改进的RBF神经网络对县级政府编制预测[J]. 信阳师范学院学报(自然科学版), 2016, 29(2): 265-269
LIU Daohua, ZHANG Fei, ZHANG Yanyan. A prediction for the preparation of county government based on improved RBF neural networks[J]. Journal of Xinyang Normal University (natural science edition), 2016, 29(2): 265-269
[8] HAN Honggui, GE Luming, QIAO Junfei. An adaptive second order fuzzy neural network for nonlinear system modeling[J]. Neurocomputing, 2016, 2014: 837-847.
[9] LI Fanjun, QIAO Junfei, HAN Honggui, et al. A self-organizing cascade neural network with random weights for nonlinear system modeling[J]. Applied soft computing, 2016, 42: 184-193.
[10] HAN Honggui, QIAO Junfei. Hierarchical neural network modeling approach to predict sludge volume index of wastewater treatment process[J]. IEEE transactions on control systems technology, 2013, 21(6): 2423-2431.
[11] VAPNIK V N. An overview of statistical learning theory[J]. IEEE transactions on neural networks, 1999, 10(5): 988-999.
[12] 唐波, 彭友仙, 陈彬, 等. 基于BP神经网络的交流输电线路可听噪声预测模型[J]. 信阳师范学院学报(自然科学版), 2015, 28(1): 136-140
TANG Bo, PENG Youxian, CHEN Bin, et al. Audible noise prediction model of ac power lines based on BP neural network[J]. Journal of Xinyang Normal University (natural science edition), 2015, 28(1): 136-140
[13] HAO Peiyi. Interval regression analysis using support vector networks[J]. Fuzzy sets and systems, 2009, 160(17): 2466-2485.
[14] HAO Peiyi. Possibilistic regression analysis by support vector machine[C]//Proceedings of 2011 IEEE International Conference on Fuzzy Systems. Taipei, China, 2011: 889-894.
[15] 石磊, 侯丽萍. 基于改进PSO算法参数优化的模糊支持向量分类机[J]. 信阳师范学院学报(自然科学版), 2013, 26(2): 288-291
SHI Lei, HOU Liping. Parameter optimization of fuzzy support vector classifiers based on the improved PSO[J]. Journal of Xinyang Normal University (natural science edition), 2013, 26(2): 288-291
[16] SOUZA L C, SOUZA R M C R, AMARAL G J A, et al. A parametrized approach for linear regression of interval data[J]. Knowledge-based systems, 2017, 131: 149-159.
[17] DE A LIMA NETO E, DE A T DE CARVALHO F. An exponential-type kernel robust regression model for interval-valued variables[J]. Information sciences, 2018, 454-455: 419-442.
[18] BASAK D, PAL S, PATRANABIS D C. Support vector regression[J]. Neural information processing-letters and reviews, 2007, 11(10): 203-224.
[19] SMOLA A J, SCH?LKOPF B. A tutorial on support vector regression[J]. Statistics and computing, 2004, 14(3): 199-222.
[20] SOARES Y M G, FAGUNDES R A A. Interval quantile regression models based on swarm intelligence[J]. Applied soft computing, 2018, 72: 474-485.
[21] LU Zhao, SUN Jing, BUTTS K. Linear programming SVM-ARMA2K with application in engine system identification[J]. IEEE transactions on automation science and engineering, 2011, 8(4): 846-854.
[22] RIVAS-PEREA P, COTA-RUIZ J. An algorithm for training a large scale support vector machine for regression based on linear programming and decomposition methods[J]. Pattern recognition letters, 2013, 34(4): 439-451.
[23] HSU C W, CHANG C C, LIN C J. A practical guide to support vector classification. Technical report. Department of Computer Science, National Taiwan University (2003)[EB/OL](2010-4-15). http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.
Similar References:

Memo

-

Last Update: 2021-01-15

Copyright © CAAI Transactions on Intelligent Systems