[1]GAO Chunyan,LIU Qi,LI Manhong,et al.Review of robot tactile perception algorithms for complex environments[J].CAAI Transactions on Intelligent Systems,2026,21(4):834-848.[doi:10.11992/tis.202510008]
Copy

Review of robot tactile perception algorithms for complex environments

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 4 Page number: 834-848 Column: 综述 Public date: 2026-07-05
Title:
Review of robot tactile perception algorithms for complex environments
Author(s):
GAO Chunyan;  LIU Qi;  LI Manhong;  LIU Xuan
School of Mechanical Engineering, Hebei University of Technology, Tianjin 300401, China
Keywords:
tactile sensing; unstructured environments; artificial intelligence; robots; deep learning; reinforcement learning; machine learning; multimodal systems
CLC:
TP242.2
DOI:
10.11992/tis.202510008
Abstract:
To address the challenges posed by complex environments and improve the accuracy and robustness of robotic tactile perception, this study systematically reviews representative machine learning-, deep learning-, and reinforcement learning-based algorithms developed over the past five years, covering convolutional neural network(CNN), graph neural network(GNN), long short-term memory network(LSTM), multimodal fusion, and active exploration. The results indicate that existing methods have made significant progress in handling complex characteristics such as noise, irregular contact, and dynamic temporal patterns, while demonstrating distinct methodological advantages. Nevertheless, common bottlenecks remain, including data sparsity, limited publicly available datasets, weak cross-modal alignment, and challenges in simulation-to-real transfer. These findings provide valuable insights for future research on low-supervision learning, standardized dataset construction, unified multimodal modeling, and high-fidelity transfer in robotic tactile perception.
References:
[1] 孙世政, 何江, 秦鸿宇, 等. 基于FBG的机器手指尖触觉感知研究[J]. 仪器仪表学报, 2024, 45(5): 72-81 SUN Shizheng, HE Jiang, Qin Hongyu, et al. Research on tactile perception of machine fingertip based on FBG[J]. Chinese journal of scientific instrument, 2024, 45(5): 72-81
[2] YI Zhengkun, ZHANG Yilei, PETERS J. Bioinspired tactile sensor for surface roughness discrimination[J]. Sensors and actuators A: physical, 2017, 255: 46-53
[3] SUN Shizheng, HE Shenggang, HAN Yu, et al. Soft bionic skin with touch-slip and thermal perception using double-layer distributed FBG sensing array[J]. IEEE sensors journal, 2022, 22(5): 4096-4105
[4] 赵恒, 陈清淼, 李成钢, 等. 融合机器视觉和触觉的机器人自动接线方法研究[J]. 计算技术与自动化, 2022, 41(4): 84-90 ZHAO Heng, CHEN Qingmiao, LI Chenggang, et al. Research on robot automatic wiring method integrating machine vision and tactile sense[J]. Computing technology and automation, 2022, 41(4): 84-90
[5] SUN Jia, ZHANG Qifeng, LU Yu, et al. A review of touching-based underwater robotic perception and manipulation[J]. Machines, 2025, 13(1): 41
[6] MAZZEO A, AGUZZI J, CALISTI M, et al. Marine robotics for deep-sea specimen collection: a systematic review of underwater grippers[J]. Sensors, 2022, 22(2): 648
[7] GANDARIAS J M, GARCIA-CEREZO A J, GOMEZ-DE-GABRIEL J M. CNN-based methods for object recognition with high-resolution tactile sensors[J]. IEEE sensors journal, 2019, 19(16): 6872-6882
[8] CONTE ALCARAZ J, MOGHADDAMNIA S, PEISSIG J. Efficiency of deep neural networks for joint angle modeling in digital gait assessment[J]. EURASIP journal on advances in signal processing, 2021(1): 10
[9] LIU Chongyu, LIU Hong, CHEN Hu, et al. Touchformer: a transformer-based two-tower architecture for tactile temporal signal classification[J]. IEEE transactions on haptics, 2024, 17(3): 396-404
[10] MAO Qian, ZHU Rong. Enhanced robotic tactile perception with spatiotemporal sensing and logical reasoning for robust object recognition[J]. Applied physics reviews, 2024, 11(2): 021424
[11] HUANG Jinrong, GUO Yuchen, JIANG Yongchang, et al. Recent advances and future prospects in tactile sensors for normal and shear force detection, decoupling, and applications[J]. Journal of semiconductors, 2024, 45(12): 121601
[12] PARK M, BOK B G, AHN J H, et al. Recent advances in tactile sensing technology[J]. Micromachines, 2018, 9(7): 321
[13] 张宪民, 王浩楠, 黄沿江. 机器人抓取对象硬度触觉感知研究[J]. 机械工程学报, 2021, 57(23): 12-20 ZHANG Xianmin, WANG Haonan, HUANG Yanjiang. Research on robotic grasping object hardness perception based on tactile sensing[J]. Journal of mechanical engineering, 2021, 57(23): 12-20
[14] 朱家辰. 基于强化学习的模块化软硬结合的仿生象鼻抓取器设计与控制研究[D]. 上海: 上海师范大学, 2025. ZHU Jiachen. Research on the design and control of a modular soft-hard hybrid bionic elephant trunk gripper based on reinforcement learning[D]. Shanghai: Shanghai Normal University, 2025.
[15] LI Tianliang, ZHANG Ao, DU Mingchang, et al. A fingertip optical fiber composite sensor with conformal design for robotic perception of tactile force[J]. ASME transactions on mechatronics, 2025, 30(2): 1523-1535
[16] LECHUZ-SIERRA J G, MARTIN A E H, SUNDARAM A M, et al. Bayesian optimization for robust robotic grasping using a sensorized compliant hand[J]. IEEE robotics and automation letters, 2024, 9(11): 10503-10510
[17] ZHANG Xingxing, LI Shaobo, YANG Jing, et al. Tactile perception object recognition based on an improved support vector machine[J]. Micromachines, 2022, 13(9): 1538
[18] RUSK N. Deep learning[J]. Nature methods, 2016, 13(1): 35
[19] CAO Lele, SUN Fuchun, LIU Xiaolong, et al. End-to-end convNet for tactile recognition using residual orthogonal tiling and pyramid convolution ensemble[J]. Cognitive computation, 2018, 10(5): 718-736
[20] PASTOR F, GANDARIAS J M, GARC?A-CEREZO A J, et al. Using 3D convolutional neural networks for tactile object recognition with robotic palpation[J]. Sensors, 2019, 19(24): 5356
[21] FUNABASHI S, GANG Yan, FEI Hongyi, et al. Tactile transfer learning and object recognition with a multifingered hand using morphology specific convolutional neural networks[J]. IEEE transactions on neural networks and learning systems, 2024, 35(6): 7587-7601
[22] PARK H, PARK K, MO S, et al. Deep neural network based electrical impedance tomographic sensing methodology for large-area robotic tactile sensing[J]. IEEE transactions on robotics, 2021, 37(5): 1570-1583
[23] WEERAKKODI MUDALIGE N D, NAZAROVA E, BABATAEV I, et al. DogTouch: CNN-based recognition of surface textures by quadruped robot with high density tactile sensors[C]//2022 IEEE 95th Vehicular Technology Conference. Helsinki: IEEE, 2022.
[24] MA Feihong, LI Yuliang, CHEN Meng, et al. A data-driven robotic tactile material recognition system based on electrode array bionic finger sensors[J]. Sensors and actuators A: physical, 2023, 363: 114727
[25] LI Qian, DU Zhijiang, LIU Feng, et al. Tactile perception for surgical status recognition in robot-assisted laminectomy[J]. IEEE transactions on industrial electronics, 2022, 69(11): 11425-11435
[26] GU Fuqiang, SNG W, TAUNYAZOV T, et al. TactileSGNet: a spiking graph neural network for event-based tactile object recognition[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020.
[27] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016-09-09)[2025-09-16]. https://arxiv.org/abs/1609.02907.
[28] LIAO Junjie, XIONG Pengwen, LIU P X, et al. Enhancing robotic tactile exploration with multireceptive graph convolutional networks[J]. IEEE transactions on industrial electronics, 2024, 71(8): 9297-9308
[29] YANG Jing, YU Zukun, LI Shaobo, et al. GGT-SNN: graph learning and Gaussian prior integrated spiking graph neural network for event-driven tactile object recognition[J]. Information sciences, 2024, 677: 120998
[30] YANG Jing, LIU Tingqing, REN Yaping, et al. AM-SGCN: tactile object recognition for adaptive multichannel spiking graph convolutional neural networks[J]. IEEE sensors journal, 2023, 23(24): 30805-30820
[31] 吴培良, 林为梁, 毛秉毅, 等. 一种基于触觉信息的脉冲图注意力残差卷积物体检测算法[J]. 控制与决策, 2023, 38(9): 2537-2544 WU Peiliang, LIN Weiliang, MAO Bingyi, et al. An object detection algorithm based on convolution of attention residuals of pulse graph based on tactile information[J]. Control and decision, 2023, 38(9): 2537-2544
[32] BENGIO Y, SIMARD P, FRASCONI P. Learning long-term dependencies with gradient descent is difficult[J]. IEEE transactions on neural networks, 1994, 5(2): 157-166
[33] SHERSTINSKY A. Fundamentals of recurrent neural network(RNN) and long short-term memory(LSTM) network[J]. Physica D: nonlinear phenomena, 2020, 404: 132306
[34] CHEN Zhengqi, VERSACE E, JAMONE L. 3D localization of objects buried within granular material using a distributed 3-axis tactile sensor[C]//2024 IEEE/RSJ International Conference on Intelligent Robots and Systems. Abu Dhabi: IEEE, 2024.
[35] XU Zhe, YI Wei, CHEN Muxin, et al. Robotic tactile recognition system based on AM-LSTM model[C]//2021 IEEE International Conference on Robotics and Biomimetics. Sanya: IEEE, 2021.
[36] YAN Youcan, HU Zhe, SHEN Yajing, et al. Surface texture recognition by deep learning-enhanced tactile sensing[J]. Advanced intelligent systems, 2022, 4: 2100076
[37] 陆军, 赵颢然, 鲁林超. 基于多模态融合的三维目标检测方法研究[J]. 智能系统学报, 2025, 20(5): 1167-1177 LU Jun, ZHAO Haoran, LU Linchao. Research on 3D object detection based on multi-modal fusion[J]. CAAI transactions on intelligent systems, 2025, 20(5): 1167-1177
[38] 王路遥, 王凤随, 闫涛, 等. 结合多尺度特征与混淆学习的跨模态行人重识别[J]. 智能系统学报, 2024, 19(4): 898-908 WANG Luyao, WANG Fengsui, YAN Tao, et al. Cross-modal person re-identification combining multi-scale features and confusion learning[J]. CAAI transactions on intelligent systems, 2024, 19(4): 898-908
[39] WU Renjie, WANG Hu, CHEN H T, et al. Deep multimodal learning with missing modality: a survey[EB/OL]. (2024-09-12)[2025-09-16]. https://arxiv.org/abs/2409.07825.
[40] YANG Fengyu, FENG Chao, CHEN Ziyang, et al. Binding touch to everything: learning unified multimodal tactile representations[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024.
[41] DING Zihao, CHEN Guodong, WANG Zhenhua, et al. Adaptive visual–tactile fusion recognition for robotic operation of multi-material system[J]. Frontiers in neurorobotics, 2023, 17: 1181383
[42] YI Xiongfeng, WANG Furui, ZUO Wenyu, et al. Robotics assisted smart-touch pipeline inspection[J]. International journal of intelligent robotics and applications, 2021, 5(3): 326-336
[43] MURALI P K, DUTTA A, GENTNER M, et al. Active visuo-tactile interactive robotic perception for accurate object pose estimation in dense clutter[J]. IEEE robotics and automation letters, 2022, 7(2): 4686-4693
[44] L? Chengang, DAI J, DU Yanxia, et al. Tactile sensing system based on FBG and SA-LSTM[J]. Sensors and actuators A: physical, 2025, 388: 116483
[45] LIU Xueqian, LI Yuanyue, LI Yiang, et al. Bionic fingerprint tactile sensor with deep learning-decoupled multimodal perception for simultaneous pressure-friction mapping[J]. Advanced functional materials, 2025, 35(49): e06158
[46] RUAN Wenjun, ZHU Wenbo, ZHAO Zhijia, et al. RFCT: multimodal sensing enhances grasping state detection for weak-stiffness targets[J]. Mathematics, 2023, 11(18): 3969
[47] 彭欢, 熊鹏文, 张宇, 等. 面向机器人交互识别的抗干扰多模态柔性电子皮肤[J]. 机器人, 2025, 47(1): 55-63 PENG Huan, XIONG Pengwen, ZHANG Yu, et al. Interference-resistant multimodal flexible electronic skin for robotic interactive recognition[J]. Robot, 2025, 47(1): 55-63
[48] PASTOR F, GARCIA-GONZALEZ J, GANDARIAS J M, et al. Bayesian and neural inference on LSTM-based object recognition from tactile and kinesthetic information[J]. IEEE robotics and automation letters, 2021, 6(1): 231-238
[49] CHI Xinrui, GUO Zhanbin, CHENG Fu. A probabilistic neural network-based bimanual control method with multimodal haptic perception fusion[J]. Alexandria engineering journal, 2025, 127: 892-919
[50] DUTTA A, BURDET E, KABOLI M. Predictive visuo-tactile interactive perception framework for object properties inference[J]. IEEE transactions on robotics, 2025, 41: 1386-1403
[51] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Long Beach: NIPS Foundation, 2017.
[52] L? Chengang, DAI J, DU Yanxia. Dual-parameter bionic tactile sensing system[J]. IEEE internet of things journal, 2025, 12(11): 16777-16788
[53] L? Chengang, LI Lin, ZHANG Ze, et al. FBG tactile sensing system based on SVP-transformer for material classification[J]. IEEE sensors journal, 2025, 25(3): 5370-5379
[54] QIU Shengjie, LI Baojiang, WANG Xichao, et al. DT-transformer: a text-tactile fusion network for object recognition[J]. IEEE transactions on haptics, 2025, 18(1): 164-174
[55] HAN Yunhai, YU Kelin, BATRA R, et al. Learning generalizable vision-tactile robotic grasping strategy for deformable objects via transformer[J]. ASME transactions on mechatronics, 2025, 30(1): 554-566
[56] LI Baojiang, LI Liang, WANG Haiyan, et al. TVT-transformer: a tactile-visual-textual fusion network for object recognition[J]. Information fusion, 2025, 118: 102943
[57] HU E J, SHEN Yelong, WALLIS P, et al. LoRA: low-rank adaptation of large language models[EB/OL]. (2021-06-17)[2025-09-16]. https://arxiv.org/abs/2106.09685.
[58] SHAHIDZADEH A H, YOO S J, MANTRIPRAGADA P, et al. AcTExplore: active tactile exploration on unknown objects[C]//2024 IEEE International Conference on Robotics and Automation. Yokohama: IEEE, 2024.
[59] CHEN Rui, WANG Huigang, WANG Haoji, et al. A motion-sensing integrated soft robot with triboelectric nanogenerator for pipeline inspection[J]. Advanced intelligent systems, 2025, 7(6): 2400643
[60] 沈健, 宋智功. 基于深度学习的双臂系统协同控制综述[J/OL]. 控制工程, (2025-09-09)[2025-10-03]. https://doi.org/10.14107/j.cnki.kzgc.20250283. SHEN Jian, SONG Zhigong. A review of cooperative control for dual-arm systems based on deep learning control[J/OL]. Engineering of China, (2025-09-09)[2025-10-03]. https://doi.org/10.14107/j.cnki.kzgc.20250283.
[61] OLIFF H, LIU Ying, KUMAR M, et al. Reinforcement learning for facilitating human-robot-interaction in manufacturing[J]. Journal of manufacturing systems, 2020, 56: 326-340
[62] FLEER S, MORINGEN A, KLATZKY R L, et al. Learning efficient haptic shape exploration with a rigid tactile sensor array[J]. PLoS one, 2020, 15(1): e0226880
[63] DUAN Boyi, QIAN Kun, LIU Aohua, et al. Visual-tactile learning of robotic cable-in-duct installation skills[J]. Automation in construction, 2025, 170: 105905
[64] TANG Jiaxian, YUAN Xiaogang, LI Shaodong. Visual-tactile fusion and SAC-based learning for robot peg-in-hole assembly in uncertain environments[J]. Machines, 2025, 13(7): 605
[65] QI Wen, FAN Haoyu, ZHENG Cankun, et al. Human-like dexterous grasping through reinforcement learning and multimodal perception[J]. Biomimetics, 2025, 10(3): 186
[66] WANG Shaoxiong, LAMBETA M, CHOU Powei, et al. TACTO: a fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors[J]. IEEE robotics and automation letters, 2022, 7(2): 3930-3937
[67] JIANU T, GOMES D F, LUO Shan. Reducing tactile Sim2Real domain gaps via deep texture generation networks[C]//2022 International Conference on Robotics and Automation. Philadelphia: IEEE, 2022.
[68] KERR J, HUANG H, WILCOX A, et al. Learning self-supervised representations from vision and touch for active sliding perception of deformable surfaces[EB/OL]. (2022-09-26)[2025-09-10]. https://arxiv.org/abs/2209.13042v1.
[69] HIGUERA C, SHARMA A, BODDULURI C K, et al. Sparsh: self-supervised touch representations for vision-based tactile sensing[EB/OL]. (2024-10-31)[2025-09-10]. https://arxiv.org/abs/2410.24090.
[70] MIYATO T, MAEDA S I, KOYAMA M, et al. Virtual adversarial training: a regularization method for supervised and semi-supervised learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(8): 1979-1993
[71] ZHENG Wendong, LIU Huaping, WANG Bowen, et al. Cross-modal surface material retrieval using discriminant adversarial learning[J]. IEEE transactions on industrial informatics, 2019, 15(9): 4978-4987
[72] ZHENG Wendong, LIU Huaping, WANG Bowen, et al. Cross-modal material perception for novel objects: a deep adversarial learning method[J]. IEEE transactions on automation science and engineering, 2020, 17(2): 697-707
[73] LIU Huaping, WANG Feng, SUN Fuchun, et al. Surface material retrieval using weakly paired cross-modal learning[J]. IEEE transactions on automation science and engineering, 2019, 16(2): 781-791
[74] LIU Huaping, SUN Fuchun, FANG Bin, et al. Multimodal measurements fusion for surface material categorization[J]. IEEE transactions on instrumentation and measurement, 2018, 67(2): 246-256
[75] ZHAO Wenying, XU Qian, WANG Haoyu, et al. Cross-modal hashing for material surface properties fusion[C]//2023 International Wireless Communications and Mobile Computing. Marrakesh: IEEE, 2023.
[76] LI Fengling, WANG Tong, ZHU Lei, et al. Task-adaptive asymmetric deep cross-modal hashing[J]. Knowledge-based systems, 2021, 219: 106851
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems