[1]JIANG Yunliang,YIN Zezong,ZHANG Xiongtao,et al.TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification[J].CAAI Transactions on Intelligent Systems,2024,19(6):1419-1427.[doi:10.11992/tis.202307029]
Copy

TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1419-1427 Column: 学术论文—机器感知与模式识别 Public date: 2024-12-05
Title:
TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification
Author(s):
JIANG Yunliang1; 2; 3;  YIN Zezong1; 2;  ZHANG Xiongtao1; 2;  SHEN Qing1; 2;  LI Hua2; 4
1. School of Information Engineering, Huzhou University, Huzhou 313000, China;
2. Zhejiang Province Key Laboratory of Smart Management and Application of Modern Agricultural Resources, Huzhou 313000, China;
3. School of Computer Science and Technolo
Keywords:
TSK fuzzy classifier; knowledge distillation; high-order fuzzy dark knowledge; electro encephalo gram (EEG); least learning machine; epilepsy; motor imagery; fuzzy system
CLC:
TP181
DOI:
10.11992/tis.202307029
Abstract:
In the classification and detection task of electroencephalogram (EEG) signals, the low-order Takagi-Sugeno-Kang (TSK) fuzzy classifier runs faster but performs poorly, while the high-order TSK fuzzy classifier demonstrates strong prediction performance. However, the extremely complex fuzzy rules in the consequent part notably affect the running speed of the model. Therefore, this study proposes a novel TSK fuzzy distillation classifier, STSK-LLM-KD, based on negative Euclidean probability and high-order fuzzy dark knowledge transfer. First, the least learning machine based on knowledge distillation (LLM-KD) is used to quickly solve the consequent parameters of the teacher model and obtain corresponding negative Euclidean probabilities to generate soft labels. Then, the high-order fuzzy dark knowledge of the teacher model is extracted by calculating the Kullback-Leible divergence between soft labels and transferred to the low-order student model. This approach enhances the performance of the model beyond that of the high-order TSK fuzzy classifier while maintaining a faster training speed. Experimental results on the motor imagery EEG dataset and Hauz Khas epilepsy EEG dataset in New Delhi fully verify the advantages of the proposed STSK-LLM-KD. Compared to other fuzzy classifiers, STSK-LLM-KD performs better; compared to deep knowledge distillation models, STSK-LLM-KD more effectively improves the performance of the student model.
References:
[1] ZHANG Xiongtao, CHUNG F L, WANG Shitong. An interpretable fuzzy DBN-based classifier for indoor user movement prediction in ambient assisted living applications[J]. IEEE transactions on industrial informatics, 2020, 16(1): 42-53.
[2] JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE transactions on fuzzy systems, 2017, 25(1): 3-20.
[3] ZHANG Yong, LIU Bo, JI Xiaomin, et al. Classification of EEG signals based on autoregressive model and wavelet packet decomposition[J]. Neural processing letters, 2017, 45(2): 365-378.
[4] WANG Jun, LIN Defu, DENG Zhaohong, et al. Multitask TSK fuzzy system modeling by jointly reducing rules and consequent parameters[J]. IEEE transactions on systems, man, and cybernetics: systems, 2021, 51(7): 4078-4090.
[5] QIN Bin, CHUNG F L, WANG Shitong. Biologically plausible fuzzy-knowledge-out and its induced wide learning of interpretable TSK fuzzy classifiers[J]. IEEE transactions on fuzzy systems, 2020, 28(7): 1276-1290.
[6] JIANG Yizhang, WU Dongrui, DENG Zhaohong, et al. Seizure classification from EEG signals using transfer learning, semi-supervised learning and TSK fuzzy system[J]. IEEE transactions on neural systems and rehabilitation engineering, 2017, 25(12): 2270-2284.
[7] DENG Zhaohong, XU Peng, XIE Lixiao, et al. Transductive joint-knowledge-transfer TSK FS for recognition of epileptic EEG signals[J]. IEEE transactions on neural systems and rehabilitation engineering, 2018, 26(8): 1481-1494.
[8] JIANG Yizhang, ZHANG Yuanpeng, LIN Chuang, et al. EEG-based driver drowsiness estimation using an online multi-view and transfer TSK fuzzy system[J]. IEEE transactions on intelligent transportation systems, 2021, 22(3): 1752-1764.
[9] ZHENG Yuanhang, XU Zeshui, WANG Xinxin. The fusion of deep learning and fuzzy systems: a state-of-the-art survey[J]. IEEE transactions on fuzzy systems, 2022, 30(8): 2783-2799.
[10] XIE Runshan, WANG Shitong. A wide interpretable Gaussian Takagi-Sugeno-Kang fuzzy classifier and its incremental learning[J]. Knowledge-based systems, 2022, 241: 108203.
[11] ZHOU Ta, ISHIBUCHI H, WANG Shitong. Stacked blockwise combination of interpretable TSK fuzzy classifiers by negative correlation learning[J]. IEEE transactions on fuzzy systems, 2018, 26(6): 3327-3341.
[12] TIAN Xiaobin, DENG Zhaohong, YING Wenhao, et al. Deep multi-view feature learning for EEG-based epileptic seizure detection[J]. IEEE transactions on neural systems and rehabilitation engineering, 2019, 27(10): 1962-1972.
[13] 蒋云良, 翁江玮, 申情, 等. 基于增强深度特征和TSK模糊分类器的癫痫脑电信号识别[J]. 控制与决策, 2023, 38(1): 171-180.
JIANG Yunliang, WENG Jiangwei, SHEN Qing, et al. TSK fuzzy classifier based on enhanced deep feature for epilepsy EEG signal recognition[J]. Control and decision, 2023, 38(1): 171-180.
[14] QIN Bin, NOJIMA Y, ISHIBUCHI H, et al. Realizing deep high-order TSK fuzzy classifier by ensembling interpretable zero-order TSK fuzzy subclassifiers[J]. IEEE transactions on fuzzy systems, 2021, 29(11): 3441-3455.
[15] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015-03-09)[2021-01-01]. http://arxiv.org/abs/1503.02531.
[16] ZHAO Wenda, TONG Tingting, WANG Haipeng, et al. Diversity consistency learning for remote-sensing object recognition with limited labels[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 5631510.
[17] WANG Peng, WEN Jun, SI Chenyang, et al. Contrast-reconstruction representation learning for self-supervised skeleton-based action recognition[J]. IEEE transactions on image processing, 2022, 31: 6224-6238.
[18] TU Zhigang, LIU Xiangjian, XIAO Xuan. A general dynamic knowledge distillation method for visual analytics[J]. IEEE transactions on image processing, 2022, 31: 6517-6531.
[19] LIU Zhiwei, LYU Wentao, WANG Chengqun, et al. D-CenterNet: an anchor-free detector with knowledge distillation for industrial defect detection[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 2518412.
[20] SONG Jie, CHEN Ying, YE Jingwen, et al. Spot-adaptive knowledge distillation[J]. IEEE transactions on image processing, 2022, 31: 3359-3370.
[21] BLAKENEY C, LI Xiaomin, YAN Yan, et al. Parallel blockwise knowledge distillation for deep neural network compression[J]. IEEE transactions on parallel and distributed systems, 2021, 32(7): 1765-1776.
[22] ZHANG Libo, DU Dawei, LI Congcong, et al. Iterative knowledge distillation for automatic check-out[J]. IEEE transactions on multimedia, 2020, 23: 4158-4170.
[23] ZHAO Qilu, DONG Junyu, YU Hui, et al. Distilling ordinal relation and dark knowledge for facial age estimation[J]. IEEE transactions on neural networks and learning systems, 2021, 32(7): 3108-3121.
[24] JIANG Yunliang, WENG Jiangwei, ZHANG Xiongtao, et al. A CNN-based born-again TSK fuzzy classifier integrating soft label information and knowledge distillation[J]. IEEE transactions on fuzzy systems, 2023, 31(6): 1843-1854.
[25] GU Xiangming, CHENG Xiang. Distilling a deep neural network into a takagi-sugeno-Kang fuzzy inference system[EB/OL]. (2020-10-10)[2021-01-01]. http://arxiv.org/abs/2010.04974.
[26] ERDEM D, KUMBASAR T. Enhancing the learning of interval type-2 fuzzy classifiers with knowledge distillation[C]//2021 IEEE International Conference on Fuzzy Systems. Luxembourg: IEEE, 2021: 1-6.
[27] ZHOU Ta, CHUNG F L, WANG Shitong. Deep TSK fuzzy classifier with stacked generalization and triplely concise interpretability guarantee for large data[J]. IEEE transactions on fuzzy systems, 2017, 25(5): 1207-1221.
[28] DEMIRLI K, MUTHUKUMARAN P. Higher order fuzzy system identification using subtractive clustering[J]. Journal of intelligent & fuzzy systems, 2000, 9(3/4): 129-158.
[29] BISIADA M. Empirical studies in translation and discourse (Volume 14)[M]. Berlin: Language Science Press, 2021: 165-198.
[30] SCHL?GL A, NEUPER C, PFURTSCHELLER G. Estimating the mutual information of an EEG-based brain-computer interface[J]. Biomedizinische Technik Biomedical engineering, 2002, 47(1/2): 3-8.
[31] SAMEER M, GUPTA B. Detection of epileptical seizures based on alpha band statistical features[J]. Wireless personal communications, 2020, 115(2): 909-925.
[32] WU Dongrui, JIANG Xue, PENG Ruimin. Transfer learning for motor imagery based brain-computer interfaces: a tutorial[J]. Neural networks, 2022, 153: 235-253.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems