[1]谈超,关佶红,周水庚.增量与演化流形学习综述[J].智能系统学报,2012,7(5):377-388.
　TAN Chao,GUAN Jihong,ZHOU Shuigeng.Incremental and evolutionary manifold learning: a survey[J].CAAI Transactions on Intelligent Systems,2012,7(5):377-388.
点击复制

增量与演化流形学习综述

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 7 期数: 2012年第5期 页码: 377-388 栏目: 综述 出版日期: 2012-10-25
Title:
Incremental and evolutionary manifold learning: a survey
文章编号:
1673-4785(2012)05-0377-12
作者:
谈超1，关佶红1，周水庚2,3
1.同济大学 计算机科学与技术系，上海 201804；
2.复旦大学 计算机学院，上海 200433；
3.复旦大学 上海市智能信息处理重点实验室，上海 200433
Author(s):
TAN Chao1, GUAN Jihong1, ZHOU Shuigeng2,3
1．Department of Computer Science and Technology, Tongji University, Shanghai 201804, China;
2．School of Computer Science, Fudan University, Shanghai 200433, China;
3．Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai 200433, China
关键词:
流形学习; 增量流形学习; 演化流形学习
Keywords:
manifold learning;  incremental manifold learning;  evolutionary manifold learning.
分类号:
TP181
文献标志码:
A
摘要:
流形学习的目标是发现观测数据嵌入在高维数据空间中的低维光滑流形.近年来，在线或增量地发现内在低维流形结构成为流形学习的研究热点.从增量学习和演化学习2个方面入手，对该领域已有研究进展进行综述.增量流形学习较之传统的批量流形学习方法具有动态增量的能力，而演化流形学习能够在线地发现海量动态数据的内在规律，有利于进行维数约简和数据分析.文中对主要的增量与演化流形学习算法的基本原理、特点进行了阐述，分析了各自的优点与不足，指出了该领域的开放问题，并对进一步的研究方向进行了展望.
Abstract:
Manifold learning is to find the lowdimensional smooth manifold of observation data embedded in highdimensional data space. In recent years, exploring the intrinsic lowdimensional manifold structure online or incrementally becomes a hot research topic in manifold learning area. This paper surveys the state of the art of incremental and evolutionary manifold learning, including the mechanisms and features of major existing incremental and evolutionary manifold learning methods, their advantages and disadvantages, and highlights the open research issues and future research directions.
参考文献/References:
［1］LAW M, ZHANG Nan, JAIN A K. Nonlinear manifold learning for data stream［C］//Proceedings of the Fourth SIAM International Conference on Data Mining. Lake Buena Vista, USA, 2004: 3344.
［2］徐蓉，姜峰，姚鸿勋，等．流形学习概述［J］．智能系统学报, 2006, 1(1): 4451. 
XU Rong, JIANG Feng, YAO Hongxun, et al．Overview of manifold learning［J］．CAAI Transactions on Intelligent Systems, 2006, 1(1): 4451.
［3］SEUNG H, LEE D. The manifold ways of perception［J］. Science, 2000, 290(5500): 22682269.
［4］PEARSON K. On lines and planes of closest fit to systems of points in space［J］. Philosophical Magazine, 1901, 2(6): 559572.
［5］TENENBAUM J, DE SILVA V, LANGFORD J. A global geometric framework for nonlinear dimensionality reduction［J］. Science, 2000, 290(5500): 23192323.
［6］BELKIN M, NIYOGI P. Laplacian eigenmaps for dimensionality reduction and data representation［J］. Neural Computation, 2003, 15(6): 13731396.
［7］ROWEI S, SAUL L. Nonlinear dimensionality reduction by locally linear embedding［J］. Science, 2000, 290(5500): 23232326.
［8］曾宪华，罗四维．动态增殖流形学习算法［J］．计算机研究与发展, 2007, 44(9): 14621468. 
ZENG Xianhua, LUO Siwei. A dynamically incremental manifold learning algorithm［J］. Journal of Computer Research and Development, 2007, 44(9): 14621468.
［9］DE SILVA V, TENENBAUM J B. Global versus local methods in nonlinear dimensionality reduction［M］//BECKER S, THRUN S, OBERMAYER K. Advances in Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2003: 721728.
［10］BERGER M, GOSTIAUX B. Differential geometry: manifolds, curves and surfaces［M］. ［S.l.］: SpringerVerlag, 1988: 474.
［11］PLESS R, SOUVENIR R. A survey of manifold learning for images［J］. IPSJ Transactions on Computer Vision and Applications, 2009, 1: 8394.
［12］BREGLER C, OMPHUNDRO S M. Nonlinear manifold learning for visual speech recognition［C］//Proceedings of the 5th International Conference on Computer Vision. Washington, DC, USA: IEEE Computer Society, 1995: 494499.
［13］HADID A, KOUROPTEVA O, PIETIKANINEN M. Unsupervised learning using locally linear embedding: experiments in face pose analysis［C］//Proceedings of the 16th International Conference on Pattern Recognition. Quebec City, Canada, 2002: 111114.
［14］JENKINS O C, MATARIC M J. A spatiotemporal extension to Isomap nonlinear dimension reduction［C］//Proceedings of the 21th International Conference on Machine Learning. New York, USA, 2002: 25512556.
［15］〖JP3〗NISKANEN M, SILVEN O. Comparison of dimensionality reduction methods for wood surface inspection［C］//Proceedings of the 6th International Conference on Quality Control by Artificial Vision. Gatlinburg, USA, 2003: 178188.
［16］KRUSKAL J B, WISH M. Multidimensional scaling［M］. Beverly Hills, USA: Sage Publications, 1977.
［17］ZHANG Zhenyue, ZHA Hongyuan. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment［J］. SIAM Journal of Scientific Computing, 2004, 26(1): 313338.
［18］LU Ke, HE Xiaofei. Image retrieval based on incremental subspace learning［J］. Pattern Recognition, 2005, 38(11): 20472054.
［19］YE Jieping, LI Qi, XIONG Hui, et al. IDR/QR: an incremental dimension reduction algorithm via QR decomposition［J］. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(9): 12081222.
［20］HAN Zhi, MENG Deyu, XU Zongben, et al. Incremental alignment manifold learning［J］. Journal of Computer Science and Technology, 2011, 26(1): 153165.
［21］LI Housen, JIANG Hao, BARRIO R, et al. Incremental manifold learning by spectral embedding methods［J］. Pattern Recognition Letters, 2011, 32(10): 14471455.
［22］KOUROPTEVA O, OKUN O, PIETIKAINEN M. Incremental locally linear embedding algorithm［C］//Proceedings of the 14th Scandinavian Conference Image Analysis. Joensuu, Finland, 2005: 521530.
［23］LAW M H C, JAIN A K. Incremental nonlinear dimensionality reduction by manifold learning［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(3): 337391.
［24］GOLUB G H, VAN LOAN C F. Matrix computations［M］. Baltimore, USA: Johns Hopkins University Press, 1996: 1694.
［25］SHI Lukui, YANG Qingxin, LIU Enhai, et al. An incremental manifold learning algorithm based on the small world model［C］//Proceedings of the 2010 International Conference on Life System Modeling and Intelligent Computing, and 2010 International Conference on Intelligent Computing for Sustainable Energy and Environment. Wuxi, China, 2010: 324332.
［26］SAUL L K, ROWEIS S T. Think globally, fit locally: unsupervised learning of low dimensional manifolds［J］. Journal of Machine Learning Research, 2003, 4: 119155.
［27］KOUROPTEVA O, OKUN O, PIETIKANEN M. Incremental locally linear embedding［J］. Pattern Recognition, 2005, 38(10): 17641767.
［28］朱明旱，罗大庸，易励群，等．基于正交迭代的增量LLE算法［J］．电子学报, 2009, 37(1): 132136. 
ZHU Minghan, LUO Dayong, YI Liqun, et al. Incremental locally linear embedding algorithm based on orthogonal iteration method［J］. Acta Electronica Sinica, 2009, 37(1): 132136.
［29］刘小明．数据降维及分类中的流形学习研究［D］．杭州：浙江大学, 2007: 1108. 
LIU Xiaoming. Research on data dimension reduction and manifold learning in classification［D］. Hangzhou: Zhejiang University, 2007: 1108.
［30］LIU Xiaoming, YIN Jianwei, FENG Zhilin, et al. Incremental manifold learning via tangent space alignment［C］//Proceedings of the Second International Conference on Artificial Neural Networks in Pattern Recognition. Ulm, Germany, 2006: 107121.
［31］YIN Jianwei, LIU Xiaoming, FENG Zhilin, et al. A local tangent space alignment based transductive classification algorithm［C］//Proceedings of the Second International Conference on Artificial Neural Networks in Pattern Recognition. Ulm, Germany, 2006: 93106.
［32］JIA Peng, YIN Junsong, HUANG Xinsheng, et al. Incremental Laplacian eigenmaps by preserving adjacent information between data points［J］. Pattern Recognition Letters, 2009, 30(16): 14571463.
［33］ZHAO Dongfang, YANG Li. Incremental isometric embedding of highdimensional data using connected neighborhood graphs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(1): 8698.
［34］ZHAO Dongfang, YANG Li. Incremental construction of neighborhood graphs for nonlinear dimensionality reduction［C］//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong, China, 2006: 177180.
［35］VIJAYAKUMAR S, DSOUZA A, SCHAAL S. Incremental online learning in high dimensions［J］. Neural Computation, 2005, 17(12): 26022634.
［36］FRITZKE B. Incremental learning of local linear mappings［C］//Proceedings of the International Conference on Artificial Neural Networks. Paris, France, 1995: 217222.
［37］WANG Yi, LIU Shixia, FENG Jianhua, et al. Mining naturally smooth evolution of clusters from dynamic data［C］//Proceedings of the SIAM International Conference on Data Mining. Minneapolis, USA, 2007: 125134.
［38］AHMED A, XING E. Dynamic nonparametric mixture models and the recurrent Chinese restaurant process: with applications to evolutionary clustering［C］//Proceedings of the SIAM International Conference on Data Mining. Atlanta, USA, 2008: 219230.
［39］SOUVENIR R, PLESS R. Manifold clustering［C］//Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China, 2005: 648653.
［40］CAO Wenbo, HARALICK R. Nonlinear manifold clustering by dimensionality［C］//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong, China, 2006: 920924.
［41］XU Rui, WUNSCHLL D. Survey on clustering algorithms［J］. IEEE Transactions on Neural Networks, 2003, 16(3): 645678.
［42］HARTIGAN J A, WONG M A. A kmeans clustering algorithm［J］. Applied Statistics, 1979, 28(1): 100108.
［43］NG A Y, JORDAN M I, WEISS Y. On spectral clustering: analysis and an algorithm［C］//Neural Information Processing Systems: Natural and Synthetic. Vancouver, Canada, 2001: 849856.
［44］CHAKRABARTI D, KUMAR R, TOMKINS A. Evolutionary clustering［C］//Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Philadelphia, USA, 2006: 554560.
［45］CHARIKAR M, CHEKURI C, FEDER T, et al. Incremental clustering and dynamic information retrieval［C］//Proceedings of the TwentyNinth Annual ACM Symposium on the Theory of Computing. El Paso, USA, 1997: 626635.
［46］CHI Yun, SONG Xiaodan, ZHOU Dengyong, et al. Evolutionary spectral clustering by incorporating temporal smoothness［C］//Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Jose, USA, 2007: 153162.
［47］TANG Lei, LIU Huan, ZHANG Jianping, et al. Community evolution in dynamic multimode networks［C］//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Las Vegas, USA, 2008: 677685.
［48］ZHANG Jianwen, SONG Yangqiu, CHEN Gang, et al. Online evolutionary exponential family mixture［C］//Proceedings of International Joint Conference on Artificial Intelligence. Pasadena, USA, 2009: 16101615.
［49］JIA Yangqing, YAN Shuicheng, ZHANG Changshui, et al. Semisupervised classification on evolutionary data［C］//Proceedings of International Joint Conference on Artificial Intelligence. Pasadena, USA, 2009: 10831088.
［50］SANGER T D. Optimal unsupervised learning in a singlelayer linear feedforward neural network［J］. IEEE Transactions on Neural Networks, 1989, 1(2): 459473.
［51］WEINBERGER K Q, SAUL L K. Unsupervised learning of image manifolds by semidefinite programming［J］. International Journal of Computer Vision, 2006, 70(1): 7790.
［52］GOH A, VIDAL R. Segmenting motions of different types by unsupervised manifold clustering［C］//IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA, 2007: 16.
［53］GOH A, VIDAL R. Unsupervised riemannian clustering of probability density functions［C］//Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases. Antwerp, Belgium, 2008: 377392.
［54］HE Xiaofei. Incremental semisupervised subspace learning for image retrieval［C］//Proceedings of the 12th Annual ACM International Conference on Multimedia. New York, USA, 2004: 28.
［55］BELKIN M, NIYOGI P. Semisupervised learning on riemannian manifolds［J］. Machine Learning, 2004, 56(1/2/3): 209239.
［56］孟德宇，徐宗本，戴明伟．一种新的有监督流形学习方法［J］．计算机研究与发展, 2007, 44(12): 20722077. 
MENG Deyu, XU Zongben, DAI Mingwei. A new supervised manifold learning method［J］. Journal of Computer Research and Development, 2007, 44(12): 20722077.
［57］CHENG Miao, FANG Bin, TANG Yuanyan, et al. Incremental embedding and learning in the local discriminant subspace with application to face recognition［J］. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2010, 40(5): 580591.
［58］XU Ye, SHEN Furao, HASEGAWA Q, et al. An online incremental learning vector quantization［C］//Proceedings of the 13th PacificAsia Conference on Advances in Knowledge Discovery and Data Mining. Bangkok, Thailand, 2009: 10461053.
［59］NING Huazhong, XU Wei, CHI Yun, et al. Incremental spectral clustering by efficiently updating the eigensystem［J］. Pattern Recognition, 2010, 43(1): 113127.
［60］ZHANG Zhenyue, WANG Jing, ZHA Hongyuan. Adaptive manifold learning［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(2): 253265.
相似文献/References:
[1]徐 蓉,姜 峰,姚鸿勋.流形学习概述[J].智能系统学报,2006,1(1):44.
　XU Rong,JIANG Feng,YAO Hong-xun.Overview of manifold learning[J].CAAI Transactions on Intelligent Systems,2006,1():44.
[2]谢朝霞,穆志纯,谢建军.基于LLE的多姿态人耳识别[J].智能系统学报,2008,3(4):321.
　XIE Zhao-xia,MU Zhi-chun,XIE J ian-jun.Multi-pose ear recogn ition based on locally linear embedding[J].CAAI Transactions on Intelligent Systems,2008,3():321.
[3]文贵华,江丽君,文　军.局部测地距离估计的Hessian局部线性嵌入[J].智能系统学报,2008,3(5):429.
　WEN Gui-hua,J IANG L i-jun,WEN Jun.Using locally estimated geodesic distances to improve Hessian local linear embedding[J].CAAI Transactions on Intelligent Systems,2008,3():429.
[4]刘? 琚,乔建苹.基于学习的超分辨率重建技术[J].智能系统学报,2009,4(3):199.
　LIU Ju,QIAO Jian-ping.Learningbased superresolution reconstruction[J].CAAI Transactions on Intelligent Systems,2009,4():199.
[5]练浩,曾宪华,李淑芳.有监督全局流形排序的图像检索算法[J].智能系统学报,2014,9(1):92.[doi:10.3969/j.issn.1673-4785.201303021]
　LIAN Hao,ZENG Xianhua,LI Shufang.Supervised global manifold ranking based image retrieval algorithm[J].CAAI Transactions on Intelligent Systems,2014,9():92.[doi:10.3969/j.issn.1673-4785.201303021]
[6]张钢,谢晓珊,黄英,等.面向大数据流的半监督在线多核学习算法[J].智能系统学报,2014,9(3):355.[doi:10.3969/j.issn.1673-4785.201403067]
　ZHANG Gang,XIE Xiaoshan,HUANG Ying,et al.An online multi-kernel learning algorithm for big data[J].CAAI Transactions on Intelligent Systems,2014,9():355.[doi:10.3969/j.issn.1673-4785.201403067]
[7]程旸,王士同.基于局部保留投影的多可选聚类发掘算法[J].智能系统学报,2016,11(5):600.[doi:10.11992/tis.201508022]
　CHENG Yang,WANG Shitong.A multiple alternative clusterings mining algorithm using locality preserving projections[J].CAAI Transactions on Intelligent Systems,2016,11():600.[doi:10.11992/tis.201508022]
[8]徐慧敏,陈秀宏.图正则化稀疏判别非负矩阵分解[J].智能系统学报,2019,14(6):1217.[doi:10.11992/tis.201811021]
　XU Huimin,CHEN Xiuhong.Graph-regularized, sparse discriminant, non-negative matrix factorization[J].CAAI Transactions on Intelligent Systems,2019,14():1217.[doi:10.11992/tis.201811021]
[9]邓廷权,王强.半监督类保持局部线性嵌入方法[J].智能系统学报,2021,16(1):98.[doi:10.11992/tis.202003007]
　DENG Tingquan,WANG Qiang.Semi-supervised class preserving locally linear embedding[J].CAAI Transactions on Intelligent Systems,2021,16():98.[doi:10.11992/tis.202003007]
[10]余沁茹,卢桂馥,李华.自适应图正则化的低秩非负矩阵分解算法[J].智能系统学报,2022,17(2):325.[doi:10.11992/tis.202102007]
　YU Qinru,LU Guifu,LI Hua.Nonnegative low-rank matrix factorization with adaptive graph neighbors[J].CAAI Transactions on Intelligent Systems,2022,17():325.[doi:10.11992/tis.202102007]

备注/Memo

收稿日期： 2012-05-02.
网络出版日期：2012-09-17.
基金项目：国家自然科学基金资助项目（61173118）.
通信作者：关佶红. E-mail: jhguan@tongji.edu.cn.
作者简介：谈超，女，1983年生，博士研究生，主要研究方向为机器学习与数据挖掘.
关佶红，女，1969年生，教授，博士生导师，中国计算机学会数据库专委会委员、开发系统专委会委员．主要研究方向为空间数据库、数据挖掘、生物信息学等．主持和参与国家自然科学基金、国家“863”计划项目、省部级以及其他科研项目30余项．2011年获教育部科技进步二等奖，发表学术论文200余篇．
周水庚，男，1966年生，教授，博士生导师，中国计算机学会数据库专委会和人工智能与模式识别专委会委员，中国人工智能学会机器学习专委会常委．主要研究方向为数据库、数据挖掘、生物信息学等．主持或参与国家“973”计划子项目、国家“863”计划项目、国家自然科学基金重大项目与面上项目及其他省部级科研项目20余项．获部级自然科学奖/科技进步奖二等奖6项、三等奖1项，发表学术论文150余篇．

更新日期/Last Update: 2012-11-13
Copyright © 《 智能系统学报》 编辑部
地址：（150001）黑龙江省哈尔滨市南岗区南通大街145-1号楼 电话：0451- 82534001、82518134　邮箱：tis@vip.sina.com