[1]LI Xiaofei,LIU Hong.A survey of sound source localization for robot audition[J].CAAI Transactions on Intelligent Systems,2012,7(1):9-20.
Copy

A survey of sound source localization for robot audition

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 7 Number of periods: 2012 1 Page number: 9-20 Column: 综述 Public date: 2012-02-25
Title:
A survey of sound source localization for robot audition
Author(s):
LI Xiaofei1;  LIU Hong1; 2
1.Key Laboratory of Integrated Microsystems, Shenzhen Graduate School of Peking University, Shenzhen 518055, China;
2.Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing 100871, China
Keywords:
robot;  robot audition;  sound source localization;  microphone array
CLC:
TP242.6;TN912.3
DOI:
-
Abstract:
The technology of sound source localization can localize the direction and position of a sound source relative to a robot. Sound source localization system for robot audition can greatly improve the ability of a robot to interact with external conditions. The summary and analysis of sound source localization for robot audition can significantly promote the development of intelligent robots. In this work, first, the characteristics of sound source localization for robot audition were summarized. The key technologies were summarized, including the time delay of arrival, steered beamforming, high resolution spectral estimation, binaural, active audition, and audiovisual. Then, the models of a microphone array were classified, and the performances of seven typical systems based on a 3D microphone array, 2D microphone array, and binaural were compared. Finally, the applications of a sound source localization system of robot audition were summarized. Several issues that sound source localization systems face as well as development trends were analyzed. 
References:
［1］IRIE R E. Robust sound localization: an application of an auditory perception system for a humanoid robot［D］. Cambridge, USA: Department of Electrical Engineering and Computer Science, MIT, 1995.
［2］KNAPP C H, CARTER G C. The generalized correlation method for estimation of time delay［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1976, 24(4): 320327.
［3］OMOLOGO M, SVAIZER P. Acoustic source location in noisy and reverberant environment using CSP analysis［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Atlanta, USA, 1996: 921924.
［4］BENESTY J. Adaptive eigenvalue decomposition algorithm for passive acoustic source localization［J］. Journal of Acoustical Society of America, 2000, 107(1): 384391.
［5］DVORKIND T G, GANNOT S. Time difference of arrival estimation of speech source in a noisy and reverberant environment［J］. IEEE Transactions on Signal Processing, 2005, 85(1): 177204.
［6］HAHN W, TRETTER S. Optimum processing for delayvector estimation in passive signal arrays［J］. IEEE Transactions on Information Theory, 1973, 19(5): 608614.
［7］WANG H, CHU P. Voice source localization for automatic camera pointing system in videoconferencing［C］//IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, USA, 1997: 187190.
［8］SCHAU H, ROBINSON A. Passive source localization employing intersection spherical surfaces from timeofarrival difference［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1987, 35(8): 12231225.
［9］HUANG Jie, SUPAONGPRAPA T, TERAKURA I, et al. Mobile robot and sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Grenoble, France, 1997: 683689.
［10］HUANG Jie, KUME K, SAJI A, et al. Robotic spatial sound localization and its 3D sound human interface［C］//First International Symposium on Cyber Worlds (CW 2002). Tokyo, Japan, 2002: 191197.
［11］LI H K, YOSIARA T, ZHAO Q F. A spatial sound localization system for mobile robots［C］//IEEE Instrumentation and Measurement Technology Conference. Warsaw, Poland, 2007: 16.
［12］SEKMEN A S, WIKES M, KAWAMURA K. An application of passive humanrobot interaction: human tracking based on attention distraction［J］. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 2002, 32(2): 248259.
［13］VALIN J M, MICHAUD F, ROUAT J, et al. Robust sound source localization using a microphone array on a mobile robot［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Las Vegas, USA, 2003: 12281233.
［14］RUDZYN B, KADOUS W, SAMMUT C. Real time robot audition system incorporating both 3D sound source localisation and voice characterization［C］//IEEE International Conference on Robotics and Automation. Roma, Italy, 2007: 47334738.
［15］KWAK K C, KIM S S. Sound source localization with the aid of excitation source information in home robot environments［J］. IEEE Transactions on Consumer Electronics, 2008, 54(2): 852856.
［16］HU J S, CHAN C Y, WANG C K, et al. Simultaneous localization of mobile robot and multiple sound sources using microphone array［C］//IEEE International Conference on Robotics and Automation. Kobe, Japan, 2009: 2934.
［17］HU J S, YANG C H, WANG C K. Estimation of sound source number and directions under a multisource environment［C］//IEEE/RSJ International Conference on Intelligent Robots and System. Louis, USA, 2009: 181186.
［18］LEE B, CHOI J S. Multisource sound localization using the competitive Kmeans clustering［C］//IEEE Conference on Emerging Technologies and Factory Automation. Bilbao, Spain, 2010: 17.
［19］HAHN W R. Optimum signal processing for passive sonar range and bearing estimation［J］. Journal of Acoustical Society of America, 1975, 58(1): 201207.
［20］CARTER G. Variance bounds for passively locating an acoustic source with a symmetric line array［J］. Journal of Acoustical Society of America, 1977, 62(4): 922926.
［21］RAMOS L L, HOLM S, GUDYANGEN S, et al. Delayandsum beamforming for direction of arrival estimation applied to gunshot acoustics［C］//Proceedings of SPIE Defense, Security, and Sensing. Orlando, USA, 2011.
［22］KAJALA M. Filterandsum beamformer with adjustable filter characteristics［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Salt Lake City, USA, 2001: 29172920.
［23］曹玮玮.基于麦克风阵列的声源定位与语音增强方法研究［D］.北京:清华大学, 2008.
CAO Weiwei. Study on methods of microphone array based sound source localization and speech enhancement［D］. Beijing: Tsinghua University, 2008
［24］MATSUI T, ASOH H, FRY J, et al. Integrated natural spoken dialogue system of Jijo2 mobile robot for office services［C］//Proceedings of the Sixteenth National Conference on Artificial Intelligence and the Eleventh Conference on Innovative Applications of Artificial Intelligence. Menlo Park, USA, 1999: 621627.
［25］VALIN J M, MICHAUD F, HADJOU B. Localization of simultaneous moving sound sources for mobile robot using a frequencydomain steered beamformer approach［C］//IEEE International Conference on Robotics and Automation. New Orleans, USA, 2004: 10331038.
［26］BADALI A, VALIN J M, MICHAUD F. Evaluating realtime audio localization algorithms for artificial audition in robotics［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St.Louis, USA, 2009: 20332038.
［27］TAMAI Y, KAGAMI S, AMEMIYA Y, et al. Circular microphone array for robot’s audition［C］//IEEE International Conference on Sensors. Vienna, Austria, 2004: 565570.
［28］TAMAI Y, SASAKI Y, KAGAMI S. Three ring microphone array for 3D sound localization and separation for mobile robot audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 41724177.
［29］NAKADAI K, NAKAJIMA H, YAMADA K, et al. Sound source tracking with directivity pattern estimation using a 64 ch microphone array［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 16901696.
［30］NAKADAI K, NAKAJIMA H, MURASE M, et al. Robust tracking of multiple sound sources by spatial integration of room and robot microphone arrays［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Toulouse, France, 2006: 929932. 
［31］SASAKI Y, KAGAMI S, MIZOGUCHI H. Multiple sound source mapping for a mobile robot by selfmotion triangulation［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 380385.
［32］SASAKI Y, KAGAMI S, MIZOGUCHI H. Mainlobe canceling method for multiple sound sources localization on mobile robot［C］//IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Zurich, Switzerland, 2007: 16.
［33］KAGAMI S, THOMPSON S, SASAKI Y, et al. 2D sound source mapping from mobile robot using beamforming and particle filtering［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Taipei, China, 2009: 36893692.
［34］SASAKI Y, THOMPSON S, KANEYOSHI M, et al. Mapgeneration and identification of multiple sound sources from robot in motion［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 437443.
［35］SCHMIDT R O. Multiple emitter location and signal parameter estimation［J］. IEEE Transactions on Antennas and Propagation, 1986, 34(33): 276280.
［36］WANG H, KAVEH M. Coherent signal subspace processing for the detection and estimation of angles of arrival of multiple wideband sources［J］. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1985, 33(4): 823831.
［37］居太亮.基于麦克风阵列的声源定位算法研究［D］.成都:电子科技大学, 2006.
JU Tailiang. Research on speech source localization methods based on microphone arrays［D］. Chengdu: University of Electronic Science and Technology of China, 2006.
?［38］ASANO F, ASOH H, MATSUI T. Sound source localization and signal separation for office robot “Jijo2”［C］//IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems. Taipei, China, 1999: 243248.
［39］ARGENTIERI S. Broadband variations of the MUSIC highresolution method for sound source localization in robotics［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Diego, USA, 2007: 20092014.
［40］NAKAMURA K, NAKADAI K, ASANO F, et al. Intelligent sound source localization for dynamic environments［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis, USA, 2009: 664669.
［41］NAKAMURA K, NAKADAI K, ASANO F, et al. Intelligent sound source localization and its application to multimodal human tracking［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 143148.
［42］ISHI C T, CHATOT O, ISHIGURO H, et al. Evaluation of a MUSICbased realtime sound localization of multiple sound sources in real noisy environments［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis, USA, 2009: 20272032.
［43］LYON R F. A computational model of binaural localization and separation［C］//IEEE International Conference on Acoustics, Speech, and Signal Processing. Boston, USA, 1983: 11481151.
［44］ALGAZU V R, DUDA R O, MORRISON R P, et al. Structural composition and decomposition of HRTFs［C］//IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New York, USA, 2001: 103106.
［45］HANDZEL A A, KRISHNAPRASAD P S. Biomimetic soundsource localization［J］. IEEE Journal on Sensors, 2002, 2(6): 607616.
［46］NAKADAI K, OKUNOT H G, KITANO H. Epipolar geometry based sound localization and extraction for humanoid audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Maui, USA, 2001: 13951401.
［47］NAKADAI K, HIDAI K, MIZOGUCHI H, et al. Realtime auditory and visual multipleobject tracking for humanoids［C］//Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. Seattle, USA, 2001: 14251436.
［48］NAKADAI K, MATSUURA D, OKUNO H G, et al. Applying scattering theory to robot audition system: robust sound source localization and extraction［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, USA, 2003: 11471152.
［49］KUMON M, SHIMODA T, KOHZAWA R. Audio servo for robotic systems with pinnae［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, 2005: 18811886.
［50］SHIMODA T, NAKASHIMA T, KUMON M, et al. Spectral cues for robust sound localization with pinnae［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 386391.
［51］HOMSTEIN J, LOPES M, SANTOSVICTOR J, et al. Sound localization for humanoid robotsbuilding audiomotor maps based on the HRTF［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, 2006: 11701176.
［52］KEYROUZ F, MAIER W, DIEPOLD K. A novel humanoid binaural 3D sound localization and separation algorithm［C］//IEEERAS International Conference on Humanoid Robot. Genova, Italy, 2006: 296301.
［53］RODEMANN T, INCE G, JOUBLIN F, et al. Using binaural and spectral cues for azimuth and elevation localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice, France, 2008: 21852190.
［54］RODEMANN T. A study on distance estimation in binaural sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 425430.
［55］KIM U H, MIZUMOTO T, OGATA T, et al. Improvement of speaker localization by considering multipath interference of sound wave for binaural robot audition［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 29102915.
［56］SKAF A. Optimal positioning of a binaural sensor on a humanoid head for sound source localization［C］//IEEERAS International Conference on Humanoid Robot. Bled, Slovenia, 2011: 165170.
［57］SAXENA A, NG A Y. Learning sound location from a single microphone［C］//IEEE International Conference on Robotics and Automation. Kobe, Japan, 2009: 17371742.
［58］NAKADAI K, LAURENS T, OKUNO H G, et al. Active audition for humanoid［C］//Proceedings of the 17th National Conference on Artificial Intelligence. Austin, USA, 2000: 832839.
［59］ANDERSSON S B, HANDZEL A A, SHAH V, et al. Robot phonotaxis with dynamic soundsource localization［C］//IEEE International Conference on Robotics and Automation. Barcelona, Spain, 2004: 48334838.
［60］MARTINSON E, APKER T, BUGAJSKA M. Optimizing a reconfigurable robotic microphone array［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 125130.
［61］PORTELLO A. Acoustic models and Kalman filtering strategies for active binaural sound localization［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 137142.
［62］KUMON M, NODA Y. Active soft pinnae for robots［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 112117.
［63］OKUNO H G, NAKADAI K, HIDAI K, et al. Humanrobot interaction through realtime auditory and visual multipletalker tracking［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Maui, USA, 2001: 14021409.
［64］OKUNO H G, NAKADAI K, KITANO K. Social interaction of humanoid robot based on audiovisual tracking［C］//International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert System. Cairns, Australia, 2002: 110.
［65］LV Xiaoling, ZHANG Minglu. Sound source localization based on robot hearing and vision［C］//International Conference on Computer Science and Information Technology. Singapore, 2008: 942946.
［66］LEE B, CHOI J S, KIM D, et al. Sound source localization in reverberant environment using visual information［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 35423547.
?［67］LIU Hong, SHEN Miao. Continuous sound source localization based on microphone array for mobile robots［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China, 2010: 43324339.
［68］LI Xiaofei, LIU Hong, YANG Xuesong. Sound source localization for mobile robot based on time difference feature and space grid matching［C］//IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, USA, 2011: 28792886. 
［69］YOUNG S H, SCANLON M V. Detection and localization with an acoustic array on a small robotic platform in urban environments, technical report ADA410432［R］. Adelphi, USA: U.S. Army Research Laboratory, 2003.
［70］SUN Hao, YANG Peng, LIU Zuojun, et al. Microphone array based auditory localization for rescue robot［C］//Chinese Control and Decision Conference. Taiyuan, China, 2011: 606609.
［71］LUO R C, HUANG C H, LIN T T. Human tracking and following using sound source localization for multisensor based mobile assistive companion robot［C］//IEEE Conference on Industrial Electronics Society. Glendale, USA, 2010: 15521557.
Similar References:

Memo

-

Last Update: 2012-05-07

Copyright © CAAI Transactions on Intelligent Systems