[1]贾晨,刘华平,续欣莹,等.基于宽度学习方法的多模态信息融合[J].智能系统学报,2019,14(01):150-157.[doi:10.11992/tis.201803022]
 JIA Chen,LIU Huaping,XU Xinying,et al.Multi-modal information fusion based on broad learning method[J].CAAI Transactions on Intelligent Systems,2019,14(01):150-157.[doi:10.11992/tis.201803022]
点击复制

基于宽度学习方法的多模态信息融合(/HTML)
分享到:

《智能系统学报》[ISSN:1673-4785/CN:23-1538/TP]

卷:
第14卷
期数:
2019年01期
页码:
150-157
栏目:
出版日期:
2019-01-05

文章信息/Info

Title:
Multi-modal information fusion based on broad learning method
作者:
贾晨1 刘华平23 续欣莹1 孙富春23
1. 太原理工大学 电气与动力工程学院, 山西 太原 030600;
2. 清华大学 计算机科学与技术系, 北京 100084;
3. 清华大学 智能技术与系统国家重点实验室, 北京 100084
Author(s):
JIA Chen1 LIU Huaping23 XU Xinying1 SUN Fuchun23
1. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030600, China;
2. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
3. State Key Laboratory of Intelligent Technology and Systems, Tsinghua University, Beijing 100084, China
关键词:
宽度学习方法多模态融合相关性分析特征提取非线性变换目标识别神经网络RGB-D图像分类
Keywords:
broad learning methodmulti-modal fusioncorrelation analysisfeature extractionnonlinear transformationobject recognitionneural networksRGB-D images classification
分类号:
TP391
DOI:
10.11992/tis.201803022
摘要:
多模态机器学习通过有效学习各个模态的丰富特征来解决不同模态数据的融合问题。考虑到模态间的差异性,基于宽度学习方法提出了一个能够学习和融合两种模态特征的框架,首先利用宽度学习方法分别提取不同模态的抽象特征,然后将高维特征表示在同一个特征空间进行相关性学习,并通过非线性融合得到最后的特征表达,输入分类器进行目标识别。相关实验建立在康奈尔大学抓取数据集和华盛顿大学RGB-D数据集上,实验结果验证了相比于传统的融合方法,所提出的方法具有更好的稳定性和快速性。
Abstract:
Multi-modal machine learning solves the fusion problem that arises in data with different modalites by effectively learning their rich characteristics. Considering the differences between various modalities, we propose a framework that can learn and fuse two kinds of modal characteristics based on the broad learning method. This method first extracts different abstract characteristics, then represents the high-dimension features in the same space to determine their correlation. We obtain a final representation of these characteristics by nonlinear fusion and inputs these characteristics into a classifier for target recognition. Relevant experiments are conducted on the Cornell Grasping Dataset and the Washington RGB-D Object Dataset, and our experimental results confirm that, compared with traditional fusion methods, the proposed algorithm has greater stability and rapidity.

参考文献/References:

[1] 雷俊, 王立辉, 何芸倩, 等. 适用于机器人视觉的图像分割方法[J]. 系统工程与电子技术, 2017, 39(7):1653-1659 LEI Jun, WANG Lihui, HE Yunqian, et al. Image segmentation method for robot vision[J]. Systems engineering and electronics, 2017, 39(7):1653-1659
[2] 毛玉仁, 郭松, 郑阳明, 等. 基于似物性判别的视觉目标检测方法[J]. 传感器与微系统, 2017, 36(11):147-150 MAO Yuren, GUO Song, ZHENG Yangming, et al. Visual object detection method based on objectness estimation[J]. Transducer and microsystem technologies, 2017, 36(11):147-150
[3] 齐静, 徐坤, 丁希仑. 机器人视觉手势交互技术研究进展[J]. 机器人, 2017, 39(4):565-584 QI Jing, XU Kun, DING Xilun. Vision-based hand gesture recognition for human-robot interaction:a review[J]. Robot, 2017, 39(4):565-584
[4] 王成济, 罗志明, 钟准, 等. 一种多层特征融合的人脸检测方法[J]. 智能系统学报, 2018, 13(1):138-146 WANG Chengji, LUO Zhiming, ZHONG Zhun, et al. Face detection method fusing multi-layer features[J]. CAAI transactions on intelligent systems, 2018, 13(1):138-146
[5] 吴钟强, 张耀文, 商琳. 基于语义特征的多视图情感分类方法[J]. 智能系统学报, 2017, 12(5):745-751 WU Zhongqiang, ZHANG Yaowen, SHANG Lin. Multi-view sentiment classification of microblogs based on semantic features[J]. CAAI transactions on intelligent systems, 2017, 12(5):745-751
[6] 温有福, 贾彩燕, 陈智能. 一种多模态融合的网络视频相关性度量方法[J]. 智能系统学报, 2016, 11(3):359-365 WEN Youfu, JIA Caiyan, CHEN Zhineng. A multi-modal fusion approach for measuring web video relatedness[J]. CAAI transactions on intelligent systems, 2016, 11(3):359-365
[7] 吴宗胜, 傅卫平, 韩改宁. 基于深度卷积神经网络的道路场景理解[J]. 计算机工程与应用, 2017, 53(22):8-15 WU Zongsheng, FU Weiping, HAN Gaining. Road scene understanding based on deep convolutional neural network[J]. Computer engineering and applications, 2017, 53(22):8-15
[8] 吴宗胜, 傅卫平. 移动机器人全局路径规划的模拟退火-教与学优化算法[J]. 机械科学与技术, 2016, 35(5):678-685 WU Zongsheng, FU Weiping. SA and teaching-learning -based optimization algorithm for mobile robots global path planning[J]. Mechanical science and technology for aerospace engineering, 2016, 35(5):678-685
[9] 张文, 刘勇, 张超凡, 等. 基于方向A*算法的温室机器人实时路径规划[J]. 农业机械学报, 2017, 48(7):22-28 ZHANG Wen, LIU Yong, ZHANG Chaofan, et al. Real-time path planning of greenhouse robot based on directional A* algorithm[J]. Transactions of the Chinese society for agricultural machinery, 2017, 48(7):22-28
[10] 张文, 刘勇, 张超凡, 等. 基于语义建图的室内机器人实时场景分类[J]. 传感器与微系统, 2017, 36(8):18-21, 28 ZHANG Wen, LIU Yong, ZHANG Chaofan, et al. Real-time scene category of indoor robot based on semantic mapping[J]. Transducer and microsystem technologies, 2017, 36(8):18-21, 28
[11] CHEN C L P, LIU Zhulin. Broad learning system:an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE transactions on neural networks and learning systems, 2018, 29(1):10-24.
[12] HUANG Guangbin, BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE transactions on neural networks, 1998, 9(1):224-229.
[13] PAO Y H, TAKEFUJI Y. Functional-link net computing:theory, system architecture, and functionalities[J]. Computer, 1992, 25(5):76-79.
[14] PAO Y H, PARK G H, SOBAJIC D J. Learning and generalization characteristics of the random vector functional-link net[J]. Neurocomputing, 1994, 6(2):163-180.
[15] IGELNIK B, PAO Y H. Stochastic choice of basis functions in adaptive function approximation and the functional-link net[J]. IEEE transactions on neural networks, 1995, 6(6):1320-1329.
[16] HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1/2/3):489-501.
[17] HUANG Guangbin, CHEN Lei. Convex incremental extreme learning machine[J]. Neurocomputing, 2007, 70(16/17/18):3056-3062.
[18] HUANG Guangbin, BAI Zuo, KASUN L L C, et al. Local receptive fields based extreme learning machine[J]. IEEE computational intelligence magazine, 2015, 10(2):18-29.
[19] HOTELLING H. Relations between two sets of variates[J]. Biometrika, 1936, 28(3/4):321-377.
[20] LENZ I, LEE H, SAXENA A. Deep learning for detecting robotic grasps[J]. The international journal of robotics research, 2015, 34(4/5):705-724.
[21] RASIWASIA N, MAHAJAN D, MAHADEVAN V, et al. Cluster canonical correlation analysis[C]//Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics. Reykjavik, Iceland, 2014:823-831.
[22] LAI K, BO Leifeng, REN Xiaofeng, et al. A large-scale hierarchical multi-view RGB-D object dataset[C]//Proceedings of IEEE International Conference on Robotics and Automation. Shanghai, China, 2011:1817-1824.

相似文献/References:

[1]温晓红,刘华平,阎高伟,等.基于超限学习机的非线性典型相关分析及应用[J].智能系统学报,2018,13(04):633.[doi:10.11992/tis.201703034]
 WEN Xiaohong,LIU Huaping,YAN Gaowei,et al.Nonlinear canonical correlation analysis and application based on extreme learning machine[J].CAAI Transactions on Intelligent Systems,2018,13(01):633.[doi:10.11992/tis.201703034]

备注/Memo

备注/Memo:
收稿日期:2018-03-16。
基金项目:国家自然科学基金项目(61673238);国家高技术研究发展计划课题(2015AA042306);山西省回国留学人员科研资助项目(2015-045,2016-044).
作者简介:贾晨,女,1992年生,硕士研究生,中国计算机学会会员,主要研究方向为智能控制、模式识别、机器视觉、多模态融合;刘华平,男,1976年生,副教授,博士生导师,主要研究方向为机器人感知、学习与控制、多模态信息融合;续欣莹,男,1979年生,副教授,主要研究方向为粗糙集、粒计算、数据挖掘、计算机视觉。
通讯作者:刘华平.E-mail:hpliu@tsinghua.edu.cn
更新日期/Last Update: 1900-01-01