[1]ZHANG Jingyu,XU Xinying,XIE Gang,et al.Continuous classification of garbage based on the elastic weightconsolidation and knowledge distillation[J].CAAI Transactions on Intelligent Systems,2023,18(4):878-885.[doi:10.11992/tis.202211023]
Copy
Continuous classification of garbage based on the elastic weightconsolidation and knowledge distillation
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
18
Number of periods:
2023 4
Page number:
878-885
Column:
吴文俊人工智能科学技术奖论坛
Public date:
2023-07-15
- Title:
- Continuous classification of garbage based on the elastic weightconsolidation and knowledge distillation
- Keywords:
- domestic garbage; image classification; continuous learning; deep learning; knowledge distillation; regularization; temperature coefficient; generalization capability
- CLC:
- TP391
- DOI:
- 10.11992/tis.202211023
- Abstract:
- The current garbage classification methods focus on the classification of common domestic garbage of fixed classes, which cannot meet the dynamic and continuous classification requirements brought by the growth of the number of garbage classes. To solve this problem, the paper proposes an elastic weight consolidation and knowledge distillation (EWC-KD) continuous garbage classification method. The method enhances the memory ability of the model through EWC regularization loss function and distillation loss function. EWC regularization loss function limits the update range of important parameters, and the distillation loss function with temperature coefficient enhances the generalization ability of the model by protecting the class information carried in the class label. Experiments on five garbage classification tasks show that the performance of this method is better than that of the comparison method. Our method can maintain high classification accuracy and low backward transfer value on all tasks, and can enhance the continuous classification ability of the garbage classification system.
- References:
-
[1] TONG Yeqing, LIU Jiafa, LIU Sizhe. China is implementing “Garbage Classification” action[J]. Environmental pollution, 2020, 259: 113707.
[2] 李金玉, 陈晓雷, 张爱华, 等. 基于深度学习的垃圾分类方法综述[J]. 计算机工程, 2022, 48(2): 1–9
LI Jinyu, CHEN Xiaolei, ZHANG Aihua, et al. Survey of garbage classification methods based on deep learning[J]. Computer engineering, 2022, 48(2): 1–9
[3] KANG Zhuang, YANG Jie, LI Guilan, et al. An automatic garbage classification system based on deep learning[J]. IEEE access, 2020, 8: 140019–140029.
[4] 张涛. 基于GAICNet的垃圾识别分类检测网络[J]. 智能计算机与应用, 2022, 12(4): 47–53
ZHANG Tao. Garbage identification classification and detection method based on GAICNet[J]. Intelligent computer and applications, 2022, 12(4): 47–53
[5] 许玉蕊, 刘银华, 高鑫. 基于特征融合卷积神经网络的垃圾分类[J]. 自动化与仪表, 2021, 36(9): 11–16
XU Yurui, LIU Yinhua, GAO Xin. Garbage classification based on feature fusion convolutional neural network[J]. Automation & instrumentation, 2021, 36(9): 11–16
[6] 杨旺功, 赵一飞. 注意力机制与双线性网络的垃圾图像分类研究[J]. 计算机仿真, 2021, 38(12): 222–226
YANG Wanggong, ZHAO Yifei. Research on garbage image classification based on attention mechanism and bilinear network[J]. Computer simulation, 2021, 38(12): 222–226
[7] ZENG Ming, LU Xiangzhe, XU Wenkang, et al. PublicGarbageNet: a deep learning framework for public garbage classification[C]// 39th Chinese Control Conference. Piscataway: IEEE, 2020: 7200-7205.
[8] MENG Sha, ZHANG Ning, REN Yunwen. X-DenseNet: deep learning for garbage classification based on visual images[J]. Journal of physics:conference series, 2020, 1575(1): 012139.
[9] SHI Cuiping, XIA Ruiyang, WANG Liguo. A novel multi-branch channel expansion network for garbage image classification[J]. IEEE access, 2020, 8: 154436–154452.
[10] BIRCANO?LU C, ATAY M, BE?ER F, et al. RecycleNet: intelligent waste sorting using deep neural networks[C]//2018 Innovations in Intelligent Systems and Applications. Piscataway: IEEE, 2018: 1?7.
[11] MAO Weilung, CHEN Weichun, WANG Chien, et al. Recycling waste classification using optimized convolutional neural network[J]. Resources, conservation and recycling, 2021, 164: 105132.
[12] YANG Jianfei, ZENG Zhaoyang, WANG Kai, et al. GarbageNet: a unified learning framework for robust garbage classification[J]. IEEE transactions on artificial intelligence, 2021, 2(4): 372–380.
[13] DE LANGE M, ALJUNDI R, MASANA M, et al. A continual learning survey: defying forgetting in classification tasks[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(7): 3366–3385.
[14] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: incremental classifier and representation learning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5533?5542.
[15] SHIN H, LEE J K, KIM J, et al. Continual learning with deep generative replay[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 2994?3003.
[16] LOPEZ-PAZ D, RANZATO M. Gradient episodic memory for continual learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6470?6479.
[17] CHAUDHRY A, RANZATO M, ROHRBACH M, et al. Efficient lifelong learning with A-GEM[EB/OL]. (2018?12?02)[2022?11?16]. https://arxiv.org/abs/1812.00420
[18] MALLYA A, LAZEBNIK S. PackNet: adding multiple tasks to a single network by iterative pruning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7765?7773.
[19] FERNANDO C, BANARSE D, BLUNDELL C, et al. PathNet: evolution channels gradient descent in super neural networks[EB/OL]. (2017?01?30)[2022?11?16].https://arxiv.org/abs/1701.08734
[20] ALJUNDI R, CHAKRAVARTY P, TUYTELAARS T. Expert gate: lifelong learning with a network of experts[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 7120-7129.
[21] KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the national academy of sciences of the United States of America, 2017, 114(13): 3521–3526.
[22] LIU Xialei, MASANA M, HERRANZ L, et al. Rotate your networks: better weight consolidation and less catastrophic forgetting[C]//2018 24th International Conference on Pattern Recognition. Piscataway: IEEE, 2018: 2262-2268.
[23] LEE S W, KIM J H, JUN J, et al. Overcoming catastrophic forgetting by incremental moment matching[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 4655?4665.
[24] LI Zhizhong, HOIEM D. Learning without forgetting[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(12): 2935–2947.
[25] RANNEN A, ALJUNDI R, BLASCHKO M B, et al. Encoder based lifelong learning[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 1329?1337.
[26] CHAUDHRY A, DOKANIA P K, AJANTHAN T, et al. Riemannian walk for incremental learning: understanding forgetting and intransigence[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 556?572.
[27] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015?03?09)[2022?11?16]. https://arxiv.org/abs/1503.02531.
[28] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84–90.
[29] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261?2269.
[30] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770?778.
[31] YAN Xin. Garbagenet: a large-scale garbage dataset for image classification[EB/OL]. (2023?01?20)[2023?03?10]. https://git.openi.org.cn/Garbage sorting/GarbageNet.
- Similar References:
Memo
-
Last Update:
1900-01-01