[1]王召新,续欣莹,刘华平,等.基于级联宽度学习的多模态材质识别[J].智能系统学报,2020,15(4):787-794.[doi:10.11992/tis.201908021]
 WANG Zhaoxin,XU Xinying,LIU Huaping,et al.Cascade broad learning for multi-modal material recognition[J].CAAI Transactions on Intelligent Systems,2020,15(4):787-794.[doi:10.11992/tis.201908021]
点击复制

基于级联宽度学习的多模态材质识别(/HTML)
分享到:

《智能系统学报》[ISSN:1673-4785/CN:23-1538/TP]

卷:
第15卷
期数:
2020年4期
页码:
787-794
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2020-07-05

文章信息/Info

Title:
Cascade broad learning for multi-modal material recognition
作者:
王召新1 续欣莹1 刘华平23 孙富春23
1. 太原理工大学 电气与动力工程学院,山西 太原 030600;
2. 清华大学 计算机科学与技术系,北京 100084;
3. 清华大学 智能技术系统国家重点实验室,北京 100084
Author(s):
WANG Zhaoxin1 XU Xinying1 LIU Huaping23 SUN Fuchun23
1. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030600, China;
2. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
3. State Key Laboratory of Intelligent Technology and System, Tsinghua University, Beijing 100084, China
关键词:
级联结构宽度学习方法多模态融合材质识别光谱数据同构数据特征提取神经网络
Keywords:
cascade structurebroad learning methodmulti-modal fusematerial recognitionspectral datahomogeneous datafeature extractionneural network
分类号:
TP391
DOI:
10.11992/tis.201908021
摘要:
材质识别在机器人与周围环境的相互作用中起着至关重要的作用,视觉、触觉和听觉模式可以提供不同材质的不同特性,如何利用不同模态的信号快速、高效地完成材质识别任务是亟待解决的问题。并且在现实应用中,传感器收集的数据量不大,无法为深度神经网络提供足够的数据进行学习训练。为此,本文将级联宽度学习这种泛化性能好的算法应用在小样本的材质识别任务上。首先,将两组同构多模态数据进行特征融合,之后使用级联特征节点的宽度学习进行特征学习,最终得到材质分类结果。最后,针对公开数据开展实验评估。结果表明,本文提出的方法与其他算法相比,在完成材质识别任务的同时,降低了训练时间,提高了分类性能。
Abstract:
Material recognition plays a vital role in the interaction betwee-n the robot and the surrounding environment. The visual, tactile and auditory modalities can provide different properties of various materials. How to use signals of different modalities to complete the task of material identification quickly and efficiently is an urgent problem to be solved. Moreover, in practical applications, the data collected by the sensor is limited, so it cannot provide enough data for deep neural network for learning and training. To this end, this paper applies the cascade broad learning with good generalization performance to the material recognition task of small samples. Firstly, feature fusion of two sets of homogeneous multi-modal data is carried out, and then feature learning is carried out by using the broad learning of cascading feature nodes, The results show that compared with other methods, the method proposed in this paper reduces the training time and improves the classification performance while completing the material recognition tasks.

参考文献/References:

[1] BELL S, UPCHERCH P, SNAVELY N, et al. Material recognition in the wild with the materials in context database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Massachusetts, Boston, 2015: 3479-3487.
[2] 齐静, 徐坤, 丁希仑. 机器人视觉手势交互技术研究进展[J]. 机器人, 2017, 39(4): 565-584
QI Jing, XU Kun, DING Xilun. Vision-based hand gesture recognition for human-robot interaction: a review[J]. Robot, 2017, 39(4): 565-584
[3] 吴钟强, 张耀文, 商琳. 基于语义特征的多视图情感分类方法[J]. 智能系统学报, 2017, 12(5): 167-173
WU Zhongqiang, ZHANG Yaowen, SHANG Lin. Multi-view sentiment classification of microblogs based on semantic features[J]. CAAI transactions on intelligent systems, 2017, 12(5): 167-173
[4] 温有福, 贾彩燕, 陈智能. 一种多模态融合的网络视频相关性度量方法[J]. 智能系统学报, 2016, 11(3): 359-365
WEN Youfu, JIA Caiyan, CHEN Zhineng. A multi-modal fusion approach for measuring web video relatedness[J]. CAAI transactions on intelligent systems, 2016, 11(3): 359-365
[5] 马蕊, 刘华平, 孙富春, 等. 基于触觉序列的物体分类[J]. 智能系统学报, 2015, 10(3): 362-368
MA Rui, LIU Huaping, SUN Fuchun, et al. Object classification based on the tactile sequence[J]. CAAI transactions on intelligent systems, 2015, 10(3): 362-368
[6] LIU H, WU Y, SUN F, et al. Weakly paired multimodal fusion for object recognition[J]. IEEE transactions on automation science and engineering, 2017, 15(2): 784-795.
[7] EGUíLUZ A G, RA?ó I, Coleman S A, et al. A multi-modal approach to continuous material identification through tactile sensing[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. Daejeon, Korea, 2016: 4912-4917.
[8] ZHENG H, FANG L, JI M, et al. Deep learning for surface material classification using haptic and visual information[J]. IEEE transactions on multimedia, 2016, 18(12): 2407-2416.
[9] ERICKSON Z, CHERNOVA S, KEMP C. Semi-supervised haptic material recognition for robots using generative adversarial networks[J]. arXiv: 1707.02796, 2017.
[10] CHEN C L P, LIU Z. Broad learning system: an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE transactions on neural networks and learning systems, 2017, 29(1): 10-24.
[11] LIU Z, ZHOU J, CHEN C L P. Broad learning system: Feature extraction based on K-means clustering algorithm[C]//2017 4th International Conference on Information, Cybernetics and Computational Social Systems. London, UK, 2017: 683-687.
[12] LIU Z, CHEN C L P. Broad learning system: Structural extensions on single-layer and multi-layer neural networks[C]//2017 International Conference on Security, Pattern Analysis, and Cybernetics. Shenzhen, China, 2017: 136-141.
[13] JIN J, LIU Z, CHEN C L P. Discriminative graph regularized broad learning system for image recognition[J]. Science China information sciences, 2018, 61(11): 112209.
[14] CHEN C L P, LIU Z, FENG S. Universal approximation capability of broad learning system and its structural variations[J]. IEEE transactions on neural networks and learning systems, 2018, 30(4): 1191-1204.
[15] LI D, SHUJUAN J, CHUNJIN Z. Improved broad learning system: partial weights modification based on BP algorithm[J]. Materials science and engineering, 2018, 439(3): 032083.
[16] ZHANG T L, CHEN R, YANG X, et al. Rich feature combination for cost-based broad learning system[J]. IEEE access, 2018, 7(1): 160-172.
[17] ZHAO H, ZHENG J, DENG W, et al. Semi-supervised broad learning system based on manifold regularization and broad network[J]. IEEE transactions on circuits and systems I: regular papers, 2020, 67(3): 983-994.
[18] KONG Y, WANG X, CHENG Y, et al. Hyperspectral imagery classification based on semi-supervised broad learning system[J]. Remote sensing, 2018, 10(5): 685.
[19] FENG S, CHEN C L P. Fuzzy broad learning system: A novel neuro-fuzzy model for regression and classification[J]. IEEE transactions on cybernetics, 2018, 50(2): 414-424.
[20] JIN J, CHEN C L P. Regularized robust broad learning system for uncertain data modeling[J]. Neurocomputing, 2018, 322(1): 58-69.
[21] LIU Z, SHEN Y, LAKSHMINARASIMHAN V B, et al. Efficient low-rank multimodal fusion with modality-specific factors[J]. arXiv: 1806.00064, 2018.
[22] 魏洁. 深度极限学习机的研究与应用[D]. 太原: 太原理工大学, 2016.
WEI Jie. Research and application of deep extreme learning machine[D]. Taiyuan: Taiyuan University of Technology, 2016.
[23] ERICKSON Z, LUSKEY N, CHERNOVA S, et al. Classification of household materials via spectroscopy[J]. IEEE robotics and automation letters, 2019, 4(2): 700-707.
[24] ZHENG W, LIU H, WANG B, et al. Cross-modal surface material retrieval using discriminant adversarial learning[J]. IEEE transactions on industrial informatics, 2019(1): 1-1.
[25] 贾晨, 刘华平, 续欣莹, 等. 基于宽度学习方法的多模态信息融合[J]. 智能系统学报, 2019, 14(1): 154-161
JIA Chen, LIU Huaping, XU Xinying, et al. Multi-modal information fusion based on broad learning method[J]. CAAI transactions on intelligent systems, 2019, 14(1): 154-161
[26] 方静. 基于LRF-ELM算法的研究及其在物体材质分类中的应用[D]. 太原: 太原理工大学, 2018.
FANG Jing. The research based on LRF-ELM algorithm and its application in the object material classification[D]. Taiyuan: Taiyuan University of Technology, 2018.

相似文献/References:

[1]贾晨,刘华平,续欣莹,等.基于宽度学习方法的多模态信息融合[J].智能系统学报,2019,14(1):150.[doi:10.11992/tis.201803022]
 JIA Chen,LIU Huaping,XU Xinying,et al.Multi-modal information fusion based on broad learning method[J].CAAI Transactions on Intelligent Systems,2019,14(4):150.[doi:10.11992/tis.201803022]

备注/Memo

备注/Memo:
收稿日期:2019-08-19。
基金项目:国家自然科学基金项目(U1613212);山西省自然科学基金项目(201801D121144,201801D221190)
作者简介:王召新,硕士研究生,主要研究方向为模式识别、计算机视觉,多模态融合;续欣莹,教授,主要研究方向为粒计算、计算机视觉、智能控制。;孙富春,教授,博士生导师,中国人工智能学会副理事长,主要研究方向为智能控制与机器人、多模态数据感知、模式识别。国家杰出青年基金获得者,IEEE Fellow,国家863计划专家组成员,荣获吴文俊科学技术奖创新奖一等奖、吴文俊科学技术奖进步奖一等奖。发表学术论文200余篇,出版专著3部、译书1部出版专著3部,译书1部
通讯作者:刘华平.E-mail:hpliu@tsinghua.edu.cn
更新日期/Last Update: 2020-07-25