[1]李晓飞,刘宏.机器人听觉声源定位研究综述[J].智能系统学报,2012,7(1):9-20.
　LI Xiaofei,LIU Hong.A survey of sound source localization for robot audition[J].CAAI Transactions on Intelligent Systems,2012,7(1):9-20.
点击复制

机器人听觉声源定位研究综述

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 7 期数: 2012年第1期 页码: 9-20 栏目: 综述 出版日期: 2012-02-25
Title:
A survey of sound source localization for robot audition
文章编号:
1673-4785(2012)01-0009-12
作者:
李晓飞1,刘宏1,2
1.北京大学深圳研究生院 集成微系统科学工程与应用重点实验室，广东 深圳 518055；
2.北京大学 机器感知与智能教育部重点实验室，北京 100871
Author(s):
LI Xiaofei1， LIU Hong1,2
1.Key Laboratory of Integrated Microsystems, Shenzhen Graduate School of Peking University, Shenzhen 518055, China;
2.Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing 100871, China
关键词:
机器人; 机器人听觉; 声源定位; 麦克风阵列
Keywords:
robot;  robot audition;  sound source localization;  microphone array
分类号:
TP242.6;TN912.3
文献标志码:
A
摘要:
声源定位技术定位出外界声源相对于机器人的方向和位置，机器人听觉声源定位系统可以极大地提高机器人与外界交互的能力.总结和分析面向机器人听觉的声源定位技术对智能机器人技术的发展有着重要的意义.首先总结了面向机器人听觉的声源定位系统的特点，综述了机器人听觉声源定位的关键技术，包括到达时间差、可控波束形成、高分辨率谱估计、双耳听觉、主动听觉和视听融合技术.其次对麦克风阵列模型进行了分类，比较了基于三维麦克风阵列、二维麦克风阵列和双耳的7个典型系统的性能.最后总结了机器人听觉声源定位系统的应用，并分析了存在的问题和未来的发展趋势.
Abstract:
The technology of sound source localization can localize the direction and position of a sound source relative to a robot. Sound source localization system for robot audition can greatly improve the ability of a robot to interact with external conditions. The summary and analysis of sound source localization for robot audition can significantly promote the development of intelligent robots. In this work, first, the characteristics of sound source localization for robot audition were summarized. The key technologies were summarized, including the time delay of arrival, steered beamforming, high resolution spectral estimation, binaural, active audition, and audiovisual. Then, the models of a microphone array were classified, and the performances of seven typical systems based on a 3D microphone array, 2D microphone array, and binaural were compared. Finally, the applications of a sound source localization system of robot audition were summarized. Several issues that sound source localization systems face as well as development trends were analyzed. 
参考文献/References:
［1］IRIE R E. Robust sound localization: an application of an auditory perception system for a humanoid robot［D］. Cambridge, USA: Department of Electrical Engineering and Computer Science, MIT, 1995.
［2］KNAPP C H, CARTER G C. The generalized correlation method for estimation of time delay［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1976, 24(4): 320327.
［3］OMOLOGO M, SVAIZER P. Acoustic source location in noisy and reverberant environment using CSP analysis［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Atlanta, USA, 1996: 921924.
［4］BENESTY J. Adaptive eigenvalue decomposition algorithm for passive acoustic source localization［J］. Journal of Acoustical Society of America, 2000, 107(1): 384391.
［5］DVORKIND T G, GANNOT S. Time difference of arrival estimation of speech source in a noisy and reverberant environment［J］. IEEE Transactions on Signal Processing, 2005, 85(1): 177204.
［6］HAHN W, TRETTER S. Optimum processing for delayvector estimation in passive signal arrays［J］. IEEE Transactions on Information Theory, 1973, 19(5): 608614.
［7］WANG H, CHU P. Voice source localization for automatic camera pointing system in videoconferencing［C］//IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, USA, 1997: 187190.
［8］SCHAU H, ROBINSON A. Passive source localization employing intersection spherical surfaces from timeofarrival difference［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1987, 35(8): 12231225.
［9］HUANG Jie, SUPAONGPRAPA T, TERAKURA I, et al. Mobile robot and sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Grenoble, France, 1997: 683689.
［10］HUANG Jie, KUME K, SAJI A, et al. Robotic spatial sound localization and its 3D sound human interface［C］//First International Symposium on Cyber Worlds (CW 2002). Tokyo, Japan, 2002: 191197.
［11］LI H K, YOSIARA T, ZHAO Q F. A spatial sound localization system for mobile robots［C］//IEEE Instrumentation and Measurement Technology Conference. Warsaw, Poland, 2007: 16.
［12］SEKMEN A S, WIKES M, KAWAMURA K. An application of passive humanrobot interaction: human tracking based on attention distraction［J］. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 2002, 32(2): 248259.
［13］VALIN J M, MICHAUD F, ROUAT J, et al. Robust sound source localization using a microphone array on a mobile robot［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Las Vegas, USA, 2003: 12281233.
［14］RUDZYN B, KADOUS W, SAMMUT C. Real time robot audition system incorporating both 3D sound source localisation and voice characterization［C］//IEEE International Conference on Robotics and Automation. Roma, Italy, 2007: 47334738.
［15］KWAK K C, KIM S S. Sound source localization with the aid of excitation source information in home robot environments［J］. IEEE Transactions on Consumer Electronics, 2008, 54(2): 852856.
［16］HU J S, CHAN C Y, WANG C K, et al. Simultaneous localization of mobile robot and multiple sound sources using microphone array［C］//IEEE International Conference on Robotics and Automation. Kobe, Japan, 2009: 2934.
［17］HU J S, YANG C H, WANG C K. Estimation of sound source number and directions under a multisource environment［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Louis, USA, 2009: 181186.
［18］LEE B, CHOI J S. Multisource sound localization using the competitive Kmeans clustering［C］//IEEE Conference on Emerging Technologies and Factory Automation. Bilbao, Spain, 2010: 17.
［19］HAHN W R. Optimum signal processing for passive sonar range and bearing estimation［J］. Journal of Acoustical Society of America, 1975, 58(1): 201207.
［20］CARTER G. Variance bounds for passively locating an acoustic source with a symmetric line array［J］. Journal of Acoustical Society of America, 1977, 62(4): 922926.
［21］RAMOS L L, HOLM S, GUDYANGEN S, et al. Delayandsum beamforming for direction of arrival estimation applied to gunshot acoustics［C］//Proceedings of SPIE Defense, Security, and Sensing. Orlando, USA, 2011.
［22］KAJALA M. Filterandsum beamformer with adjustable filter characteristics［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Salt Lake City, USA, 2001: 29172920.
［23］曹玮玮.基于麦克风阵列的声源定位与语音增强方法研究［D］.北京:清华大学, 2008.
CAO Weiwei. Study on methods of microphone array based sound source localization and speech enhancement［D］. Beijing: Tsinghua University, 2008
［24］MATSUI T, ASOH H, FRY J, et al. Integrated natural spoken dialogue system of Jijo2 mobile robot for office services［C］//Proceedings of the Sixteenth National Conference on Artificial Intelligence and the Eleventh Conference on Innovative Applications of Artificial Intelligence. Menlo Park, USA, 1999: 621627.
［25］VALIN J M, MICHAUD F, HADJOU B. Localization of simultaneous moving sound sources for mobile robot using a frequencydomain steered beamformer approach［C］//IEEE International Conference on Robotics and Automation. New Orleans, USA, 2004: 10331038.
［26］BADALI A, VALIN J M, MICHAUD F. Evaluating realtime audio localization algorithms for artificial audition in robotics［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St.Louis, USA, 2009: 20332038.
［27］TAMAI Y, KAGAMI S, AMEMIYA Y, et al. Circular microphone array for robot’s audition［C］//IEEE International Conference on Sensors. Vienna, Austria, 2004: 565570.
［28］TAMAI Y, SASAKI Y, KAGAMI S. Three ring microphone array for 3D sound localization and separation for mobile robot audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 41724177.
［29］NAKADAI K, NAKAJIMA H, YAMADA K, et al. Sound source tracking with directivity pattern estimation using a 64 ch microphone array［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 16901696.
［30］NAKADAI K, NAKAJIMA H, MURASE M, et al. Robust tracking of multiple sound sources by spatial integration of room and robot microphone arrays［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Toulouse, France, 2006: 929932. 
［31］SASAKI Y, KAGAMI S, MIZOGUCHI H. Multiple sound source mapping for a mobile robot by selfmotion triangulation［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 380385.
［32］SASAKI Y, KAGAMI S, MIZOGUCHI H. Mainlobe canceling method for multiple sound sources localization on mobile robot［C］//IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Zurich, Switzerland, 2007: 16.
［33］KAGAMI S, THOMPSON S, SASAKI Y, et al. 2D sound source mapping from mobile robot using beamforming and particle filtering［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Taipei, China, 2009: 36893692.
［34］SASAKI Y, THOMPSON S, KANEYOSHI M, et al. Mapgeneration and identification of multiple sound sources from robot in motion［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 437443.
［35］SCHMIDT R O. Multiple emitter location and signal parameter estimation［J］. IEEE Transactions on Antennas and Propagation, 1986, 34(33): 276280.
［36］WANG H, KAVEH M. Coherent signal subspace processing for the detection and estimation of angles of arrival of multiple wideband sources［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1985, 33(4): 823831.
［37］居太亮.基于麦克风阵列的声源定位算法研究［D］.成都:电子科技大学, 2006.
JU Tailiang. Research on speech source localization methods based on microphone arrays［D］. Chengdu: University of Electronic Science and Technology of China, 2006.
?［38］ASANO F, ASOH H, MATSUI T. Sound source localization and signal separation for office robot “Jijo2”［C］//IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems. Taipei, China, 1999: 243248.
［39］ARGENTIERI S. Broadband variations of the MUSIC highresolution method for sound source localization in robotics［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Diego, USA, 2007: 20092014.
［40］NAKAMURA K, NAKADAI K, ASANO F, et al. Intelligent sound source localization for dynamic environments［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis, USA, 2009: 664669.
［41］NAKAMURA K, NAKADAI K, ASANO F, et al. Intelligent sound source localization and its application to multimodal human tracking［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 143148.
［42］ISHI C T, CHATOT O, ISHIGURO H, et al. Evaluation of a MUSICbased realtime sound localization of multiple sound sources in real noisy environments［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis, USA, 2009: 20272032.
［43］LYON R F. A computational model of binaural localization and separation［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Boston, USA, 1983: 11481151.
［44］ALGAZU V R, DUDA R O, MORRISON R P, et al. Structural composition and decomposition of HRTFs［C］//IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New York, USA, 2001: 103106.
［45］HANDZEL A A, KRISHNAPRASAD P S. Biomimetic soundsource localization［J］. IEEE Journal on Sensors, 2002, 2(6): 607616.
［46］NAKADAI K, OKUNOT H G, KITANO H. Epipolar geometry based sound localization and extraction for humanoid audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Maui, USA, 2001: 13951401.
［47］NAKADAI K, HIDAI K, MIZOGUCHI H, et al. Realtime auditory and visual multipleobject tracking for humanoids［C］//Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. Seattle, USA, 2001: 14251436.
［48］NAKADAI K, MATSUURA D, OKUNO H G, et al. Applying scattering theory to robot audition system: robust sound source localization and extraction［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, USA, 2003: 11471152.
［49］KUMON M, SHIMODA T, KOHZAWA R. Audio servo for robotic systems with pinnae［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 18811886.
［50］SHIMODA T, NAKASHIMA T, KUMON M, et al. Spectral cues for robust sound localization with pinnae［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 386391.
［51］HOMSTEIN J, LOPES M, SANTOSVICTOR J, et al. Sound localization for humanoid robotsbuilding audiomotor maps based on the HRTF［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 11701176.
［52］KEYROUZ F, MAIER W, DIEPOLD K. A novel humanoid binaural 3D sound localization and separation algorithm［C］//IEEERAS International Conference on Humanoid Robot. Genova, Italy, 2006: 296301.
［53］RODEMANN T, INCE G, JOUBLIN F, et al. Using binaural and spectral cues for azimuth and elevation localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice, France, 2008: 21852190.
［54］RODEMANN T. A study on distance estimation in binaural sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 425430.
［55］KIM U H, MIZUMOTO T, OGATA T, et al. Improvement of speaker localization by considering multipath interference of sound wave for binaural robot audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 29102915.
［56］SKAF A. Optimal positioning of a binaural sensor on a humanoid head for sound source localization［C］//IEEERAS International Conference on Humanoid Robot. Bled, Slovenia, 2011: 165170.
［57］SAXENA A, NG A Y. Learning sound location from a single microphone［C］//IEEE International Conference on Robotics and Automation. Kobe, Japan, 2009: 17371742.
［58］NAKADAI K, LAURENS T, OKUNO H G, et al. Active audition for humanoid［C］//Proceedings of the 17th National Conference on Artificial Intelligence. Austin, USA, 2000: 832839.
［59］ANDERSSON S B, HANDZEL A A, SHAH V, et al. Robot phonotaxis with dynamic soundsource localization［C］//IEEE International Conference on Robotics and Automation. Barcelona, Spain, 2004: 48334838.
［60］MARTINSON E, APKER T, BUGAJSKA M. Optimizing a reconfigurable robotic microphone array［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 125130.
［61］PORTELLO A. Acoustic models and Kalman filtering strategies for active binaural sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 137142.
［62］KUMON M, NODA Y. Active soft pinnae for robots［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 112117.
［63］OKUNO H G, NAKADAI K, HIDAI K, et al. Humanrobot interaction through realtime auditory and visual multipletalker tracking［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Maui, USA, 2001: 14021409.
［64］OKUNO H G, NAKADAI K, KITANO K. Social interaction of humanoid robot based on audiovisual tracking［C］//International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert System. Cairns, Australia, 2002: 110.
［65］LV Xiaoling, ZHANG Minglu. Sound source localization based on robot hearing and vision［C］//International Conference on Computer Science and Information Technology. Singapore, 2008: 942946.
［66］LEE B, CHOI J S, KIM D, et al. Sound source localization in reverberant environment using visual information［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 35423547.
?［67］LIU Hong, SHEN Miao. Continuous sound source localization based on microphone array for mobile robots［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 43324339.
［68］LI Xiaofei, LIU Hong, YANG Xuesong. Sound source localization for mobile robot based on time difference feature and space grid matching［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 28792886. 
［69］YOUNG S H, SCANLON M V. Detection and localization with an acoustic array on a small robotic platform in urban environments, technical report ADA410432［R］. Adelphi, USA: U.S. Army Research Laboratory, 2003.
［70］SUN Hao, YANG Peng, LIU Zuojun, et al. Microphone array based auditory localization for rescue robot［C］//Chinese Control and Decision Conference. Taiyuan, China, 2011: 606609.
［71］LUO R C, HUANG C H, LIN T T. Human tracking and following using sound source localization for multisensor based mobile assistive companion robot［C］//IEEE Conference on Industrial Electronics Society. Glendale, USA, 2010: 15521557.
相似文献/References:
[1]方勇纯.机器人视觉伺服研究综述[J].智能系统学报,2008,3(2):109.
　FANG Yong-chun.A survey of robot visual servoing[J].CAAI Transactions on Intelligent Systems,2008,3():109.
[2]王立权,刘秉昊,吴健荣,等.6R关节型机器人运动学建模[J].智能系统学报,2010,5(2):156.
　WANG Li-quan,LIU Bing-hao,WU Jian-rong,et al.Modeling and implementing the inverse kinematics ofa six revolute joint robot[J].CAAI Transactions on Intelligent Systems,2010,5():156.
[3]孙宁,方勇纯.一类欠驱动系统的控制方法综述[J].智能系统学报,2011,6(3):200.
　SUN Ning,FANG Yongchun.A review for the control of a class of underactuated systems[J].CAAI Transactions on Intelligent Systems,2011,6():200.
[4]蒲兴成,张军,张毅.基于神经网络的改进行为协调控制及其在智能轮椅路径规划中的应用[J].智能系统学报,2011,6(5):456.
　PU Xingcheng,ZHANG Jun,ZHANG Yi.Modified behavior coordination for intelligent wheelchair path planning based on a neural network[J].CAAI Transactions on Intelligent Systems,2011,6():456.
[5]孙凤池,宋萌,刘光.一种无线传感器信号衰减自适应测距模型[J].智能系统学报,2012,7(3):214.
　SUN Fengchi,SONG Meng,LIU Guang.An adaptive ranging model based on energy distance loss of wireless sensors[J].CAAI Transactions on Intelligent Systems,2012,7():214.
[6]孙玉香,曹会彬,冯勇,等.具有拨土功能的轮腿一体化机器人结构设计[J].智能系统学报,2012,7(5):409.
　SUN Yuxiang,CAO Huibin,FENG Yong,et al.Structure design on a legwheeledintegration robotwith an earthmoving function[J].CAAI Transactions on Intelligent Systems,2012,7():409.
[7]伍明,李琳琳,李承剑.基于协方差交集的多机器人协作目标跟踪算法[J].智能系统学报,2013,8(1):66.[doi:10.3969/j.issn.1673-4785.201204022]
　WU Ming,LI Linlin,LI Chengjian.An algorithm of multi robot cooperative object tracking based on covariance intersection[J].CAAI Transactions on Intelligent Systems,2013,8():66.[doi:10.3969/j.issn.1673-4785.201204022]
[8]肖国宝,严宣辉.一种基于改进Theta *的机器人路径规划算法[J].智能系统学报,2013,8(1):58.[doi:10.3969/j.issn.1673-4785.201208032]
　XIAO Guobao,YAN Xuanhui.A path planning algorithm based on improved Theta * for mobile robot[J].CAAI Transactions on Intelligent Systems,2013,8():58.[doi:10.3969/j.issn.1673-4785.201208032]
[9]李大伟,贾鹏飞,李卫国,等.一种基于卡尔曼滤波与模糊算法的变电站机器人组合导航及控制系统设计[J].智能系统学报,2013,8(3):226.
　LI Dawei,JIA Pengfei,LI Weiguo,et al.A kind of integrated navigation and control system design for substation robot based on the Kalman filtering and fuzzy algorithm[J].CAAI Transactions on Intelligent Systems,2013,8():226.
[10]裴振兵,陈雪波.改进蚁群算法及其在机器人避障中的应用[J].智能系统学报,2015,10(1):90.[doi:10.3969/j.issn.1673-4785.201311018]
　PEI Zhenbing,CHEN Xuebo.Improved ant colony algorithm and its application in obstacle avoidance for robot[J].CAAI Transactions on Intelligent Systems,2015,10():90.[doi:10.3969/j.issn.1673-4785.201311018]

备注/Memo

收稿日期： 2012-01-10.
网络出版时间： 2012-02-19.
基金项目：国家“863”计划资助项目（2006AA04Z247）；国家自然科学基金资助项目（60675025,60875050）；深圳市科技计划及基础研究计划资助项目（JC20090316039）.
通信作者：刘宏.????????????E-mail：hongliu@pku.edu.cn.
作者简介：
李晓飞，男，1987年生，博士研究生，主要研究方向为语音识别、声源定位.
刘宏，男，1967年生，教授，博士生导师，中国人工智能学会常务理事、副秘书长、青年工作委员会主任，主要研究方向为智能机器人、计算机视听觉.先后承担国家自然科学基金项目7项，国家“863”、“973”计划课题项目5项，曾获国家航天科技进步奖.发表学术论文100余篇，其中60余篇被SCI、EI检索.

更新日期/Last Update: 2012-05-07
Copyright © 《 智能系统学报》 编辑部
地址：（150001）黑龙江省哈尔滨市南岗区南通大街145-1号楼 电话：0451- 82534001、82518134　邮箱：tis@vip.sina.com