[1]ZHANG Weinan,LIU Ting.Research and application of embodied intelligence[J].CAAI Transactions on Intelligent Systems,2025,20(1):255-262.[doi:10.11992/tis.202406044]
Copy

Research and application of embodied intelligence

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 1 Page number: 255-262 Column: 人工智能院长论坛 Public date: 2025-01-05
Title:
Research and application of embodied intelligence
Author(s):
ZHANG Weinan;  LIU Ting
Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Keywords:
embodied AI; embodied perception; embodied cognition; embodied action optimization; deep learning; artificial intelligence; simulation environment; humanoid robots
CLC:
TP391; TP242.6
DOI:
10.11992/tis.202406044
Abstract:
With the continuous enhancement of deep learning and large model technologies, artificial intelligence has evolved from studying simple, closed virtual environments to exploring more complex, open real-world scenarios. The research focus has shifted from early small-scale corpus and web text data processing to multimodal integrated processing architectures and research paradigms. Meanwhile, the emergence of physical world approximation and simulation models, represented by OpenAI Sora, marks another step forward in the progress towards general artificial intelligence (AGI). However, for AI to truly reach the standard of AGI and become human-like intelligence, it must possess the ability to interact and learn from the physical world—this is known as embodied intelligence. Therefore, this paper primarily focuses on the research content and progress of embodied intelligence, specifically including embodied perception, embodied cognition, and embodied action optimization. Additionally, considering recent developments in humanoid robots, this paper outlines the application of embodied intelligence technologies in carriers such as humanoid robots and provides an outlook on research and applications in this field.
References:
[1] CAMPBELL M, HOANE J. Deep blue[J]. Artificial intelligence, 2002, 134(1/2): 57-83.
[2] SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.
[3] OpenAI. Introducing ChatGPT[EB/OL]. (2022-11-30)[2024-06-13]. https://openai.com/index/chatgpt.
[4] OpenAI. Video generation models as world simulators [EB/OL]. (2024-02-15)[2024-06-13]. https://openai.com/index/video-generation-models-as-world-simulators.
[5] 唐佩佩, 叶浩生. 作为主体的身体: 从无身认知到具身认知[J]. 心理研究, 2012, 5(3): 3-8.
TANG Peipei, YE Haosheng. Body as the subject: from the disembodied cognition to embodied cognition[J]. Psychological research, 2012, 5(3): 3-8.
[6] 叶浩生. 认知与身体: 理论心理学的视角[J]. 心理学报, 2013, 45(4): 481-488.
YE Haosheng. Cognition and body: a perspective from theoretical psychology[J]. Acta psychologica sinica, 2013, 45(4): 481-488.
[7] 卢策吾, 王鹤. 具身智能(embodied artificial intelligence)[EB/OL]. (2023-07-22)[2024-06-13]. https://www.ccf.org.cn/Media_list/gzwyh/jsjsysdwyh/2023-07-22/794317.shtml.
LU Cewu, WANG He. Embodied AI(embodied artificial intelligence)[EB/OL]. (2023-07-22)[2024-06-13]. https://www.ccf.org.cn/Media_list/gzwyh/jsjsysdwyh/2023-07-22/794317.shtml.
[8] ZHONG Licheng, YANG Lixin, LI Kailin, et al. Color-NeuS: reconstructing neural implicit surfaces with color[EB/OL]. (2023-12-19)[2024-06-13]. https://arxiv.org/abs/2308.06962v2.
[9] LI Kailin, YANG Lixin, ZHEN Haoyu, et al. Chord: category-level hand-held object reconstruction via shape deformation[C]//2023 IEEE/CVF International Conference on Computer Vision. Paris: IEEE, 2023: 9410-9420.
[10] XU Wenqiang, YU Zhenjun, XUE Han, et al. Visual-tactile sensing for in-hand object reconstruction[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 8803-8812.
[11] LIU Liu, XUE Han, XU Wenqiang, et al. Toward real-world category-level articulation pose estimation[J]. IEEE transactions on image processing, 2022, 31: 1072-1083.
[12] XUE Han, XU Wenqiang, ZHANG Jieyi, et al. GarmentTracking: category-level garment pose tracking[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 21233-21242.
[13] RAMAKRISHNAN S K, JAYARAMAN D, GRAUMAN K. An exploration of embodied visual exploration[J]. International journal of computer vision, 2021, 129(5): 1616-1649.
[14] LYU Jun, YU Qiaojun, SHAO Lin, et al. SAGCI-system: towards sample-efficient, generalizable, compositional, and incremental robot learning[C]//2022 International Conference on Robotics and Automation. Philadelphia: IEEE, 2022: 98-105.
[15] LI Yonglu, LIU Xinpeng, WU Xiaoqian, et al. HAKE: a knowledge engine foundation for human activity understanding[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(7): 8494-8506.
[16] MEES O, HERMANN L, ROSETE-BEAS E, et al. CALVIN: a benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks[J]. IEEE robotics and automation letters, 2022, 7(3): 7327-7334.
[17] MENDEZ-MENDEZ J, KAELBLING L P, LOZANO-PéREZ T. Embodied lifelong learning for task and motion planning[C]//Conference on Robot Learning. New York: PMLR, 2023: 2134-2150.
[18] JIANG Yunfan, GUPTA A, ZHANG Zichen, et al. VIMA: robot manipulation with multimodal prompts[C]// Proceedings of the 40th International Conference on Machine Learning. New York: PMLR, 2023: 14975-15022.
[19] AHN M, BROHAN A, BROWN N, et al. Do As I can, not As I say: grounding language in robotic affordances[EB/OL]. (2022-04-04)[2024-06-13]. https://arxiv.org/abs/2204.01691v2.
[20] DAMEN Dima, DOUGHTY H, FARINELLA G M, et al. Scaling egocentric vision: the ?quation missing\"dataset[C]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 753-771.
[21] DRIESS D, XIA Fei, SAJJADI M S M, et al. PaLM-E: an embodied multimodal language model[C]//International Conference on Machine Learning. New York: PMLR, 2023: 8469-8488.
[22] MA Y J, LIANG W, WANG Guanzhi, et al. Eureka: human-level reward design via coding large language models[EB/OL]. (2023-10-19)[2024-06-13]. https://arxiv.org/abs/2310.12931v2.
[23] BROHAN A, BROWN N, CARBAJAL J, et al. RT-2: vision-language-action models transfer web knowledge to robotic control[C]// Proceedings of the 7th Conference on Robot Learning. New York: PMLR, 2023: 2165-2183.
[24] JAMES S, WOHLHART P, KALAKRISHNAN M, et al. Sim-to-real via sim-to-sim: data-efficient robotic grasping via randomized-to-canonical adaptation networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 12619-12629.
[25] ZAGAL J C, RUIZ-DEL-SOLAR J. Combining simulation and reality in evolutionary robotics[J]. Journal of intelligent and robotic systems, 2007, 50(1): 19-39.
[26] KADIAN A, TRUONG J, GOKASLAN A, et al. Sim2Real predictivity: does evaluation in simulation predict real-world performance?[J]. IEEE robotics and automation letters, 2020, 5(4): 6670-6677.
[27] FANG Haoshu, WANG Chenxi, FANG Hongjie, et al. AnyGrasp: robust and efficient grasp perception in spatial and temporal domains[J]. IEEE transactions on robotics, 2023, 39(5): 3929-3945.
[28] ONGGO B S S, HILL J. Data identification and data collection methods in simulation: a case study at ORH Ltd[J]. Journal of simulation, 2014, 8(3): 195-205.
[29] DEITKE M, HAN W, HERRASTI A, et al. RoboTHOR: an open simulation-to-real embodied AI platform[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3164-3174.
[30] KROEMER O, NIEKUM S, KONIDARIS G. A review of robot learning for manipulation: challenges, representations, and algorithms[J]. Journal of machine learning research, 2021, 22(1): 1395-1476.
[31] FU Zipeng, ZHAO T Z, FINN C. Mobile ALOHA: learning bimanual mobile manipulation with low-cost whole-body teleoperation[EB/OL]. (2024-01-04)[2024-06-13]. https://arxiv.org/abs/2401.02117v1.
[32] ZHANG Gu, FANG Haoshu, FANG Hongjie, et al. Flexible handover with real-time robust dynamic grasp trajectory generation[C]//2023 IEEE/RSJ International Conference on Intelligent Robots and Systems. Detroit: IEEE, 2023: 3192-3199.
[33] LI Shoujie, YU Haixin, DING Wenbo, et al. Visual-tactile fusion for transparent object grasping in complex backgrounds[J]. IEEE transactions on robotics, 2023, 39(5): 3838-3856.
[34] SHEN Bokui, XIA Fei, LI Chengshu, et al. IGibson 1.0: a simulation environment for interactive tasks in large realistic scenes[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. Prague: IEEE, 2021: 7520-7527.
[35] JING Mingxuan, MA Xiaojian, HUANG Wenbing, et al. Task transfer by preference-based cost learning[C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. New York: ACM, 2019: 2471-2478.
[36] KATO I, OHTERU S, KOBAYASHI H, et al. Information-power machine with senses and limbs[M]//On Theory and Practice of Robots and Manipulators. Vienna: Springer Vienna, 1974: 11-24.
[37] KUSUDA Y. The humanoid robot scene in Japan[J]. Industrial robot, 2002, 29(5): 412-419.
Similar References:

Memo

-

Last Update: 2025-01-05

Copyright © CAAI Transactions on Intelligent Systems