[1]朱广泽,刘兴波,聂秀山,等.基于双曲乘积量化的无监督持续学习[J].智能系统学报,2026,21(5):1237-1248.[doi:10.11992/tis.202507031]
ZHU Guangze,LIU Xingbo,NIE Xiushan,et al.Unsupervised continual learning based hyperbolic product quantization[J].CAAI transactions on intelligent systems,2026,21(5):1237-1248.[doi:10.11992/tis.202507031]
点击复制
ZHU Guangze,LIU Xingbo,NIE Xiushan,et al.Unsupervised continual learning based hyperbolic product quantization[J].CAAI transactions on intelligent systems,2026,21(5):1237-1248.[doi:10.11992/tis.202507031]
基于双曲乘积量化的无监督持续学习
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第5期
页码:
1237-1248
栏目:
学术论文—机器感知与模式识别
出版日期:
2026-09-05
- Title:
- Unsupervised continual learning based hyperbolic product quantization
- Keywords:
- continual learning; unsupervised learning; product quantization; codebook; hyperbolic space; Lorentzian distances; contrastive learning; catastrophic forgetting
- 分类号:
- TP391.4
- 摘要:
- 无监督持续学习(unsupervised continual learning，UCL)是一种能够在无需人工标注的前提下，从动态变化的任务流中持续获取知识的学习范式，对于构建具备长期持续学习能力的智能系统具有重要意义。现有的无监督持续学习方法普遍存在早期任务性能不佳和灾难性遗忘的问题，主要原因在于特征信息的多样性不足和结构对齐能力薄弱。为了有效捕获数据的层次结构语义信息并增强特征信息的多样性，提出了一种新颖的无监督持续学习方法——基于双曲乘积量化的无监督持续学习方法(unsupervised continual learning based hyperbolic product quantization，HPQUCL)，首次在UCL范式中引入双曲空间中的乘积量化机制。该方法利用双曲几何的层次结构特性增强了特征表示的多样性，有效缓解了模型早期任务性能不佳的问题。此外，设计了一种基于双曲距离的混合重放策略，基于洛伦兹距离选择结构上具有代表性的样本进行重放，有效减缓了灾难性遗忘问题。为了进一步提升模型的判别能力，提出了一种在双曲空间中原始表示与量化表示之间的对比学习损失。在多个基准数据集上实验证明，HPQUCL在知识保持和早期任务性能方面显著优于主流的无监督持续学习方法。结果表明，引入双曲量化与结构感知重放机制为提升无监督持续学习的性能提供了有效途径。
- Abstract:
- Unsupervised continual learning (UCL) is a paradigm for continuously acquiring knowledge from dynamically changing task streams without manual annotations, critical for building intelligent systems with long-term continuous learning capabilities. Existing UCL methods typically suffer from poor early task performance and catastrophic forgetting, stemming from insufficient feature diversity and weak structural alignment. To capture data’s hierarchical semantics and enhance feature diversity, we propose a novel method—hyperbolic product quantization-based unsupervised continual learning (HPQUCL)—the first to introduce hyperbolic space product quantization into UCL. It leverages hyperbolic geometry’s hierarchical properties to boost feature diversity, alleviating poor early task performance. Additionally, a hybrid replay strategy based on hyperbolic distance is designed, selecting structurally representative samples via Lorentz distance for replay to mitigate catastrophic forgetting. A contrastive loss between original and quantized representations in hyperbolic space is also proposed, which further enhances discriminative ability. Experiments on multiple benchmarks show HPQUCL significantly outperforms mainstream UCL methods in knowledge retention and early task performance, demonstrating hyperbolic quantization and structure-aware replay as effective for improving UCL.
- 参考文献/References:
-
[1] MCCLOSKEY M, COHEN N J. Catastrophic interference in connectionist networks: the sequential learning problem[M]//Psychology of Learning and Motivation. Amsterdam: Elsevier, 1989: 109-165.
[2] RING M B. CHILD: a first step towards continual learning[J]. Machine learning, 1997, 28(1): 77-104
[3] ALJUNDI R, BABILONI F, ELHOSEINY M, et al. Memory aware synapses: learning what (not) to forget[C]//Computer Vision – ECCV 2018. Cham: Springer, 2018: 144-161.
[4] CHAUDHRY A, RANZATO M, ROHRBACH M, et al. Efficient lifelong learning with A-GEM[C]//International Conference on Learning Representations. New Orleans: OpenReview.net, 2019: 1-20.
[5] CHEN Cheng, ZHANG Ji, SONG Jingkuan, et al. Class gradient projection for continual learning[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 5575-5583.
[6] LI Zhizhong, HOIEM D. Learning without forgetting[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(12): 2935-2947
[7] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: incremental classifier and representation learning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5533-5542.
[8] RUSU A A, RABINOWITZ N C, DESJARDINS G, et al. Progressive neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc., 2016: 1-5.
[9] SPRECHMANN P, JAYAKUMAR S M, RAE J W, et al. Memory-based parameter adaptation[C]//International Conference on Learning Representations. Vancouver: OpenReview.net, 2018: 1-16.
[10] VENIAT T, DENOYER L, RANZATO M. Efficient continual learning with modular networks and task-driven priors[C]//International Conference on Machine Learning. Vienna: PMLR, 2021: 1-30.
[11] ZENKE F, POOLE B, GANGULI S. Continual learning through synaptic intelligence[C]//Proceedings of the 34th International Conference on Machine Learning. Sydney: PMLR, 2017: 3987–3995.
[12] ACHILLE A, ECCLES T, MATTHEY L, et al. Life-long disentangled representation learning with cross-domain latent homologies[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: ACM, 2018: 9895-9905.
[13] RAMAPURAM J, GREGOROVA M, KALOUSIS A. Lifelong generative modeling[J]. Neurocomputing, 2020, 404: 381-400
[14] RAO D, VISIN F, RUSU A A, et al. Continual unsupervised representation learning[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019: 7645–7655.
[15] WU Chenshen, HERRANZ L, LIU Xialei, et al. Memory replay GANs: learning to generate new categories without forgetting[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal: Curran Associates Inc., 2018: 1-10.
[16] YE Fei, BORS A G. Learning latent representations across multiple data domains using lifelong VAEGAN[C]//Computer Vision–ECCV 2020. Cham: Springer, 2020: 777-795.
[17] GRILL J B, STRUB F, ALTCH? F, et al. Bootstrap your own latent: a new approach to self-supervised learning[C]//Proceedings of the 34th Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2020: 21271–21284.
[18] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of the 37th International Conference on Machine Learning. Virtual: PMLR, 2020: 1597–1607.
[19] CARON M, MISRA I, MAIRAL J, et al. Unsupervised learning of visual features by contrasting cluster assignments[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2020: 9912–9924.
[20] MADAAN D, YOON J, LI Yuanchun, et al. Representational continuity for unsupervised continual learning[C]//10th International Conference on Learning Representations. Baltimore: OpenReview.net, 2022: 1-18.
[21] CHEN Cheng, SONG Jingkuan, ZHU Xiaosu, et al. CUCL: codebook for unsupervised continual learning[C]//Proceedings of the 31st ACM International Conference on Multimedia. New York: ACM, 2023: 1729-1737.
[22] CHEN Xinlei, HE Kaiming. Exploring simple Siamese representation learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 15745-15753.
[23] ZBONTAR J, JING Li, MISRA I, et al. Barlow twins: self-supervised learning via redundancy reduction[C]//International Conference on Machine Learning. Virtual: PMLR, 2021: 12310–12320.
[24] JANG Y K, CHO N I. Self-supervised product quantization for deep unsupervised image retrieval[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 12065-12074.
[25] GE Tiezheng, HE Kaiming, KE Qifa, et al. Optimized product quantization[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(4): 744-755
[26] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90
[27] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[28] YU Xiaofan, ROSING T, GUO Yunhui. Evolve: enhancing unsupervised continual learning with multiple experts[C]//2024 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2024: 2355-2366.
[29] CHEN Jingzhou, XIONG Fengchao, QIAN Yuntao, et al. Hierarchical contrastive learning for multigranularity ship classification with learnable class queries[J]. IEEE transactions on geoscience and remote sensing, 2025, 63: 5622916
[30] LIN Qinghong, CHEN Xiaojun, ZHANG Qin, et al. Deep unsupervised hashing with latent semantic components[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Virtual: AAAI, 2022: 7488-7496.
[31] MA Yueen, SONG Zixing, HU Xuming, et al. Graph component contrastive learning for concept relatedness estimation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washington D. C. : AAAI, 2023: 13362–13370.
[32] MALLYA A, LAZEBNIK S. PackNet: adding multiple tasks to a single network by iterative pruning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7765-7773.
[33] SERR? J, SURIS D, MIRON M, et al. Overcoming catastrophic forgetting with hard attention to the task[C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm: PMLR, 2018: 4548–4557.
[34] WANG Mingyang, ADEL H, LANGE L, et al. Rehearsal-free modular and compositional continual learning for language models[C]//Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Mexico City: Association for Computational Linguistics, 2024: 469-480.
[35] KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the national academy of sciences of the United States of America, 2017, 114(13): 3521-3526
[36] BUZZEGA P, BOSCHINI M, PORRELLO A, et al. Dark experience for general continual learning: a strong, simple baseline[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc. , 2020: 15920-15930.
[37] AYUB A, WAGNER A R. EEC: learning to encode and regenerate images for continual learning[C]//International Conference on Learning Representations. Virtual: OpenReview.net, 2021: 1-12.
[38] ZHAN Jingtao, MAO Jiaxin, LIU Yiqun, et al. Jointly optimizing query encoder and product quantization to improve retrieval performance[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. Virtual: ACM, 2021: 2487-2496.
[39] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego: ICLR, 2015: 1-14.
[40] XIE Saining, GIRSHICK R, DOLL?R P, et al. Aggregated residual transformations for deep neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5987-5995.
[41] CUI Hui, ZHU Lei, LI Jingjing, et al. Scalable deep hashing for large-scale social image retrieval[J]. IEEE transactions on image processing, 2020, 29: 1271-1284
[42] MOROZOV S, BABENKO A. Unsupervised neural quantization for compressed-domain similarity search[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3036-3045.
[43] WANG Jinpeng, ZENG Ziyun, CHEN Bin, et al. Contrastive quantization with code memory for unsupervised image retrieval[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Virtual: AAAI, 2022: 2468-2476.
[44] GAO Lianli, ZHU Xiaosu, SONG Jingkuan, et al. Beyond product quantization: deep progressive quantization for image retrieval[C]//Proceedings of the International Joint Conference on Artificial Intelligence. Macau: IJCAI, 2019: 723-729.
[45] LU Zepu, LIAN Defu, ZHANG Jin, et al. Differentiable optimized product quantization and beyond[C]//Proceedings of the ACM Web Conference. Austin: ACM, 2023: 3353-3363.
[46] GAO Zhi, WU Yuwei, HARANDI M, et al. Curvature-adaptive meta-learning for fast adaptation to manifold data[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(2): 1545-1562
[47] GU A, SALA F, GUNEL B, et al. Learning mixed-curvature representations in products of model spaces[C]//International Conference on Learning Representations. Vancouver: OpenReview.net, 2018: 1-13.
[48] QIU Zexuan, LIU Jiahong, CHEN Yankai, et al. HiHPQ: hierarchical hyperbolic product quantization for unsupervised image retrieval[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washington D. C. : AAAI, 2024: 4614-4622.
[49] 徐蓉, 姜峰, 姚鸿勋. 流形学习概述[J]. 智能系统学报, 2006, 1(1): 44-51 XU Rong, JIANG Feng, YAO Hongxun. Overview of manifold learning[J]. CAAI transactions on intelligent systems, 2006, 1(1): 44-51
[50] LAW M T, LIAO Renjie, SNELL J, et al. Lorentzian distance learning for hyperbolic representations[C]//Proceedings of the International Conference on Machine Learning. Long Beach: PMLR, 2019: 3672-3681.
[51] KRIZHEVSKY A, HINTON G. Learning multiple layers offeatures from tiny images[EB/OL]. (2009-04-08)[2025-07-31]. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
[52] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[53] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc., 2016: 3630–3638.
[54] LIU Hanmo, DI Shimin, LI Haoyang, et al. Effective data selection and replay for unsupervised continual learning[C]//2024 IEEE 40th International Conference on Data Engineering. Utrecht: IEEE, 2024: 1449-1463.
[55] AGHASANLI A, LI Yi, ANGELOV P. Prototype-based continual learning with label-free replay buffer and cluster preservation loss[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2025: 6547-6556.
[56] FAN Yan, WANG Yu, ZHU Pengfei, et al. Dynamic sub-graph distillation for robust semi-supervised continual learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2024: 11927-11935.
[57] WU Zhirong, XIONG Yuanjun, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3733-3742.
- 相似文献/References:
-
[1]申彦,朱玉全.CMP上基于数据集划分的K-means多核优化算法[J].智能系统学报,2015,10(4):607.[doi:10.3969/j.issn.1673-4785.201411036]
SHEN Yan,ZHU Yuquan.An optimized algorithm of K-means based on data set partition on CMP systems[J].CAAI transactions on intelligent systems,2015,10():607.[doi:10.3969/j.issn.1673-4785.201411036]
[2]程旸,王士同.基于局部保留投影的多可选聚类发掘算法[J].智能系统学报,2016,11(5):600.[doi:10.11992/tis.201508022]
CHENG Yang,WANG Shitong.A multiple alternative clusterings mining algorithm using locality preserving projections[J].CAAI transactions on intelligent systems,2016,11():600.[doi:10.11992/tis.201508022]
[3]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI transactions on intelligent systems,2018,13():85.[doi:10.11992/tis.201707032]
[4]冯冰,李绍滋.中医脉诊信号的无监督聚类分析研究[J].智能系统学报,2018,13(4):564.[doi:10.11992/tis.201703030]
FENG Bing,LI Shaozi.Unsupervised clustering analysis of human-pulse signal in traditional Chinese medicine[J].CAAI transactions on intelligent systems,2018,13():564.[doi:10.11992/tis.201703030]
[5]鲍国强,应文豪,蒋亦樟,等.多层递阶融合模糊特征映射的模糊C均值聚类算法[J].智能系统学报,2018,13(4):594.[doi:10.11992/tis.201703047]
BAO Guoqiang,YING Wenhao,JIANG Yizhang,et al.Fuzzy C-means clustering algorithm for multilayered hierarchical fusion fuzzy feature mapping[J].CAAI transactions on intelligent systems,2018,13():594.[doi:10.11992/tis.201703047]
[6]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(5):808.[doi:10.11992/tis.201804051]
YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI transactions on intelligent systems,2018,13():808.[doi:10.11992/tis.201804051]
[7]赵玉新,赵廷.海底声呐图像智能底质分类技术研究综述[J].智能系统学报,2020,15(3):587.[doi:10.11992/tis.202004026]
ZHAO Yuxin,ZHAO Ting.Survey of the intelligent seabed sediment classification technology based on sonar images[J].CAAI transactions on intelligent systems,2020,15():587.[doi:10.11992/tis.202004026]
[8]王倩倩,苗夺谦,张远健.深度自编码与自更新稀疏组合的异常事件检测算法[J].智能系统学报,2020,15(6):1197.[doi:10.11992/tis.202007003]
WANG Qianqian,MIAO Duoqian,ZHANG Yuanjian.Abnormal event detection method based on deep auto-encoder and self-updating sparse combination[J].CAAI transactions on intelligent systems,2020,15():1197.[doi:10.11992/tis.202007003]
[9]许子微,陈秀宏.自步稀疏最优均值主成分分析[J].智能系统学报,2021,16(3):416.[doi:10.11992/tis.201911028]
XU Ziwei,CHEN Xiuhong.Sparse optimal mean principal component analysis based on self-paced learning[J].CAAI transactions on intelligent systems,2021,16():416.[doi:10.11992/tis.201911028]
[10]雷涛,王洁,薛丁华,等.差异特征融合的无监督SAR图像变化检测[J].智能系统学报,2021,16(3):595.[doi:10.11992/tis.202103011]
LEI Tao,WANG Jie,XUE Dinghua,et al.Unsupervised SAR image change detection based on difference feature fusion[J].CAAI transactions on intelligent systems,2021,16():595.[doi:10.11992/tis.202103011]
[11]杨奡飞,续欣莹,谢刚,等.印刷电路板缺陷持续检测与定位方法研究[J].智能系统学报,2025,20(1):219.[doi:10.11992/tis.202310024]
YANG Aofei,XU Xinying,XIE Gang,et al.Research on continual detection and localization method for printed circuit board defect[J].CAAI transactions on intelligent systems,2025,20():219.[doi:10.11992/tis.202310024]
备注/Memo
收稿日期:2025-7-31。
基金项目:国家自然科学青年基金项目(62206160)；山东省自然科学青年基金项目(ZR2022QF082).
作者简介:朱广泽，硕士研究生，主要研究方向为无监督持续学习、联邦学习和跨模态检索。E-mail：228239765@qq.com。;刘兴波，教授，博士生导师，CCF AI专委会执行委员、CCML专委会通讯委员。主要研究方向为多媒体检索、计算机视觉。获山东省科技进步二等奖1项、山东省人工智能自然科学一等奖1项。以第一或通信作者发表学术论文17篇。E-mail：sclxb@mail.sdu.edu.cn。;聂秀山，教授，博士生导师，山东建筑大学计算机和人工智能学院院长，CCF AI专委会秘书、CCML专委会执行委员。主要研究方向为多媒体检索、计算机视觉、机器学习、数据挖掘。获山东省科技进步二等奖2项、山东省人工智能自然科学一等奖1项。以第一或通信作者发表学术论文30余篇。E-mail：niexsh@hotmail.com。
通讯作者:刘兴波. E-mail：sclxb@mail.sdu.edu.cn
更新日期/Last Update:
2026-09-05