[1]JIA Chen,LIU Huaping,XU Xinying,et al.Multi-modal information fusion based on broad learning method[J].CAAI Transactions on Intelligent Systems,2019,14(1):150-157.[doi:10.11992/tis.201803022]
Copy

Multi-modal information fusion based on broad learning method

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 14 Number of periods: 2019 1 Page number: 150-157 Column: 学术论文—机器学习 Public date: 2019-01-05
Title:
Multi-modal information fusion based on broad learning method
Author(s):
JIA Chen1;  LIU Huaping2; 3;  XU Xinying1;  SUN Fuchun2; 3
1. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030600, China;
2. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
3. State Key Laboratory of Intelligent Technology and Systems, Tsinghua University, Beijing 100084, China
Keywords:
broad learning method; multi-modal fusion; correlation analysis; feature extraction; nonlinear transformation; object recognition; neural networks; RGB-D images classification
CLC:
TP391
DOI:
10.11992/tis.201803022
Abstract:
Multi-modal machine learning solves the fusion problem that arises in data with different modalites by effectively learning their rich characteristics. Considering the differences between various modalities, we propose a framework that can learn and fuse two kinds of modal characteristics based on the broad learning method. This method first extracts different abstract characteristics, then represents the high-dimension features in the same space to determine their correlation. We obtain a final representation of these characteristics by nonlinear fusion and inputs these characteristics into a classifier for target recognition. Relevant experiments are conducted on the Cornell Grasping Dataset and the Washington RGB-D Object Dataset, and our experimental results confirm that, compared with traditional fusion methods, the proposed algorithm has greater stability and rapidity.
References:
[1] 雷俊, 王立辉, 何芸倩, 等. 适用于机器人视觉的图像分割方法[J]. 系统工程与电子技术, 2017, 39(7):1653-1659 LEI Jun, WANG Lihui, HE Yunqian, et al. Image segmentation method for robot vision[J]. Systems engineering and electronics, 2017, 39(7):1653-1659
[2] 毛玉仁, 郭松, 郑阳明, 等. 基于似物性判别的视觉目标检测方法[J]. 传感器与微系统, 2017, 36(11):147-150 MAO Yuren, GUO Song, ZHENG Yangming, et al. Visual object detection method based on objectness estimation[J]. Transducer and microsystem technologies, 2017, 36(11):147-150
[3] 齐静, 徐坤, 丁希仑. 机器人视觉手势交互技术研究进展[J]. 机器人, 2017, 39(4):565-584 QI Jing, XU Kun, DING Xilun. Vision-based hand gesture recognition for human-robot interaction:a review[J]. Robot, 2017, 39(4):565-584
[4] 王成济, 罗志明, 钟准, 等. 一种多层特征融合的人脸检测方法[J]. 智能系统学报, 2018, 13(1):138-146 WANG Chengji, LUO Zhiming, ZHONG Zhun, et al. Face detection method fusing multi-layer features[J]. CAAI transactions on intelligent systems, 2018, 13(1):138-146
[5] 吴钟强, 张耀文, 商琳. 基于语义特征的多视图情感分类方法[J]. 智能系统学报, 2017, 12(5):745-751 WU Zhongqiang, ZHANG Yaowen, SHANG Lin. Multi-view sentiment classification of microblogs based on semantic features[J]. CAAI transactions on intelligent systems, 2017, 12(5):745-751
[6] 温有福, 贾彩燕, 陈智能. 一种多模态融合的网络视频相关性度量方法[J]. 智能系统学报, 2016, 11(3):359-365 WEN Youfu, JIA Caiyan, CHEN Zhineng. A multi-modal fusion approach for measuring web video relatedness[J]. CAAI transactions on intelligent systems, 2016, 11(3):359-365
[7] 吴宗胜, 傅卫平, 韩改宁. 基于深度卷积神经网络的道路场景理解[J]. 计算机工程与应用, 2017, 53(22):8-15 WU Zongsheng, FU Weiping, HAN Gaining. Road scene understanding based on deep convolutional neural network[J]. Computer engineering and applications, 2017, 53(22):8-15
[8] 吴宗胜, 傅卫平. 移动机器人全局路径规划的模拟退火-教与学优化算法[J]. 机械科学与技术, 2016, 35(5):678-685 WU Zongsheng, FU Weiping. SA and teaching-learning -based optimization algorithm for mobile robots global path planning[J]. Mechanical science and technology for aerospace engineering, 2016, 35(5):678-685
[9] 张文, 刘勇, 张超凡, 等. 基于方向A*算法的温室机器人实时路径规划[J]. 农业机械学报, 2017, 48(7):22-28 ZHANG Wen, LIU Yong, ZHANG Chaofan, et al. Real-time path planning of greenhouse robot based on directional A* algorithm[J]. Transactions of the Chinese society for agricultural machinery, 2017, 48(7):22-28
[10] 张文, 刘勇, 张超凡, 等. 基于语义建图的室内机器人实时场景分类[J]. 传感器与微系统, 2017, 36(8):18-21, 28 ZHANG Wen, LIU Yong, ZHANG Chaofan, et al. Real-time scene category of indoor robot based on semantic mapping[J]. Transducer and microsystem technologies, 2017, 36(8):18-21, 28
[11] CHEN C L P, LIU Zhulin. Broad learning system:an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE transactions on neural networks and learning systems, 2018, 29(1):10-24.
[12] HUANG Guangbin, BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE transactions on neural networks, 1998, 9(1):224-229.
[13] PAO Y H, TAKEFUJI Y. Functional-link net computing:theory, system architecture, and functionalities[J]. Computer, 1992, 25(5):76-79.
[14] PAO Y H, PARK G H, SOBAJIC D J. Learning and generalization characteristics of the random vector functional-link net[J]. Neurocomputing, 1994, 6(2):163-180.
[15] IGELNIK B, PAO Y H. Stochastic choice of basis functions in adaptive function approximation and the functional-link net[J]. IEEE transactions on neural networks, 1995, 6(6):1320-1329.
[16] HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1/2/3):489-501.
[17] HUANG Guangbin, CHEN Lei. Convex incremental extreme learning machine[J]. Neurocomputing, 2007, 70(16/17/18):3056-3062.
[18] HUANG Guangbin, BAI Zuo, KASUN L L C, et al. Local receptive fields based extreme learning machine[J]. IEEE computational intelligence magazine, 2015, 10(2):18-29.
[19] HOTELLING H. Relations between two sets of variates[J]. Biometrika, 1936, 28(3/4):321-377.
[20] LENZ I, LEE H, SAXENA A. Deep learning for detecting robotic grasps[J]. The international journal of robotics research, 2015, 34(4/5):705-724.
[21] RASIWASIA N, MAHAJAN D, MAHADEVAN V, et al. Cluster canonical correlation analysis[C]//Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics. Reykjavik, Iceland, 2014:823-831.
[22] LAI K, BO Leifeng, REN Xiaofeng, et al. A large-scale hierarchical multi-view RGB-D object dataset[C]//Proceedings of IEEE International Conference on Robotics and Automation. Shanghai, China, 2011:1817-1824.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems