[1]张 菁,沈兰荪,David Dagan FENG.图像搜索中人机交互技术的新进展[J].智能系统学报,2007,2(04):14-20.
 ZHANG Jing,SHEN Lan-sun,David Dagan FENG.computer interaction technology in image searches: a survey[J].CAAI Transactions on Intelligent Systems,2007,2(04):14-20.





computer interaction technology  in image searches: a survey
张 菁1沈兰荪1David Dagan FENG23
ZHANG Jing1SHEN Lan-sun1David Dagan FENG23
1. Signal & Information Processing Lab., Beijing University of Technology, Bei jing 100022, China; 2. School of Information Technologies, the University of Syd ney, NSW 2006, Australia; 3. Department of Electronic & Information Engineering, Hong Kong Polytechnic University, Hong Kong, China
humancomputer interaction image search relevance feedback semant ic gap
人机交互在图像搜索中起着重要的作用.研究下一代人机交互接口以更好地表达用户搜索意图,具有广大的应用前景.如何充分利用人类的感觉器官,提供拟人化的交互方式已经成为信息科学的一个研究热点.除了提供自然友好的人机交互,还需要研究如何采用相关反馈技术获取用户的真实需求,以弥补图像底层特征和高层语义之间的鸿沟,优化查询结果,实现个性化搜索.首先对图像搜索的发展概况做了简要介绍,在对人机交互、相关反馈和个性化搜索的研究进展进行讨论后,描述了人眼跟踪、语音和触摸导航在图像检索中的应用 .最后指出了图像搜索中人机交互技术进一步的发展前景.
Humancomputer interaction plays an important role in image searches. Next generation humancomputer interactions which can identify users’ search i n tentions are a promising research field. Ways to do this by fully utilizing huma n sense organs and providing humanlike interaction have become a lively topic i n informatics. Based on a natural and friendly humancomputer interaction, rele v ance feedback is used to determine a user’s requirements and narrow the gap bet ween lowlevel image features and highlevel semantic concepts in order to opt im ize query results and perform a personalized search. Developments in the area of image searches are briefly addressed. The current state of humancomputer inte r action, relevance feedback, and personalized search are discussed. Applications for image retrieval using eyetracking, speech, and haptical navigation are als o described. Finally current challenges and future trends are outlined.


[1]沈兰荪,卓 力.小波编码与网络视频传输[M]. 北京:科学出版社,2005.
[2]LI Xiaohua, SHEN Lansun. Detecting faces in the wavelet compressed domain[A ]. In Proceedings of SPIE: Visual Communications and Image Processing 2005[C] . Beijing,2005.
[3]LIU Danghui, SHEN Lansun, LAN Kinman, et al. Face recognition based on ill um ination restoration[A]. In Proceeding of 2004 International Symposium Multime dia: Video and Speech Proceeding[C]. Hong Kong,China,2004.
[4]FASEL B, LUETTIN J. Automatic facial expression analysis: a survey[J]. Pattern Recognition, 2003, 36(1): 259-275.
[5]OUDEVER P. The production and recognition of emotions in speech: features and algorithms[J].Int J of HumanComputer Studies, 2003, 59(1-2):157-183 .
[6]MARCEL S. Gestures for multimodal interfaces: a review[R]. Technical Report IDIAPRR 02-34,2002.
[7]HU Weiming, TAN Tieniu, WANG Liang, et al. A survey on visual surveillance o f object motion and behaviors [J]. IEEE Trans on Systems, Man, and Cybernetic s, 2004, 34(8):3.
[8]DUCHOWSKI A. A breadthfirst survey of eye tracking applications[J]. B ehavior Research Methods, Instruments, and Computer, 2002, 34(4):455-470.
[9]PORTA M. Visionbased user interfaces: methods and applications[J]. Int J Humancomputer Studies, 2002, 57(1):27-73.
[10]DURIC Z, GRAY W, HEISHMAN R, et al. Integrating perceptual and co gnitive mo deling for adaptive and intelligent humancomputer interaction [J]. Proceedin gs of the IEEE, 2002, 90(7):1272-1289.
[11]OVIATT S, DARRELL T, FLICKNER M. Multimodal interfaces that flex, ada pt, and persist[J]. Communications of the ACM, 2004, 47( 1 ): 30-75.
[12]KISACANIN B, PAVLOVIC V, HUANG T. Realtime vision for humancomputer i nteraction [M]. New York:SpringerVerlag,2005.
[13]QVARFORDT P, ZHAI Shumin. Conversing with the user based on eyegaze pat terns[A].Conf HumanFactors in Computing System[C]. New York,2005.
[14]TURK M, KOLSCH M. Perceptual interfaces[M]. Englewood Cliffs:Prentice Hall, 2004.
[15]TURK M, ROBERTSON G. Perceptual interfaces[J]. Communications of the A CM, 2000, 43(3):32-34.
[16]SELKER T. Visual attentive interfaces[J]. BT Technology Journal, 2004, 22(4):146-150.
[17]CHEN J,BOUMAN C,DALTON J.Hierarchical browsing and search of large ima ge databases[J]. IEEE Trans Image Process, 2000, 9(3): 442-445.
[18]ISHIKAWA Y, SUBRAMANYA R, FALOUTSOS C. MindReader: query databases throug h multiple examples[A]. International Conf on Very Large Data Bases (VLDB) [C]. New York, USA, 1998.
[19]RUI Y, HUANG T. Optimizing learning in image retrieval[A]. IEEE Conf Computer Vision and Pattern Recognition[C]. South Carolina, USA, 2000.
[20]ZHOU X, HUANG T. Small sample learning during multimedia retrieval using Bi asMap[A]. IEEE Int Conf Computer Vision and Pattern Recognition[C]. Hawai i, USA, 2001.
[21]CHEN Y, ZHOU X, HUANG T. Oneclass SVM for learning in image retrieval [A]. International Conf on Image Processing[C]. Thessaloniki, 2001.
[22]WU Y, TIAN Q, HUANG T S. Discriminant EM algorithm with application to ima ge retrieval[A]. IEEE Conf Computer Vision and Pattern Recognition[C].South Carolina, USA,2000.
[23]MACARTUR S, BRODLEY C, SHYU C. Relevance feedback decision trees in conte nt based image retrieval[A]. IEEE Workshop CBAIVL[C]. South Carolina, USA, 2000.
[24]TIEU K, VIOLA P. Image retrieval[A]. IEEE Conf Computer Vision an d Pattern Recognition[C]. South Carolina, USA, 2000.
[25]TONG S, CHANG E. Support vector machine active learning for image retriev al[A]. ACM Multimedia[C]. Ottawa, Canada, 2001.
[26]TONG S, KOLLER D. Support vector machine active learning with application s to text classification[A]. International Conf on Machine Learning[C]. Stanford, USA, 2000.
[27]VASCONCELOS N, LIPPMAN A. Bayesian relevance feedback for contentbased image retrieval[A]. IEEE Workshop CBAIVL[C]. South Carolina, USA, 2000.
[28]WONG S, ZIARKO W, WONG P. Generalized vector space model in information r et rieval[A]. Proceedings of the 8th ACM SIGIR Conference on Research and Develo pment in Information Retrieval[C]. Montreal, Canada, 1985.
[29]RUI Y, HUANG T, ORTEGA M, et al. Relevance feedback: a power tool in inte ra ctive contentbased image retrieval[J]. IEEE Trans Circuits System Video Tec hnology, 1998, 8(5):644-655.
[30]PICARD R, MINKA T, SZUMMER M. Modeling user subjectivity in image librari es[A]. International Conf on Image Processing[C]. Lausanne, Switzerland, 1 996.
[31]WOOD M, CAMPBELL N, THOMAS B. Iterative refinement by relevance feedback in contentbased digital image retrieval[A]. ACM Multimedia[C].Bristol, UK, 1998.
[32]LAAKSONEN J, KOSKELA M, OJA E. PicSOM: selforganizing maps for content bas ed image retrieval[A]. INNSIEEE International Joint Conference on Neural Ne tworks[C]. Washington, DC, USA, 1999.
[33]SALTON G. Automatic text processing[M]. MA: AddisonWesley, 1989.
[34]SCHETTINI R, CIOCCA G, GAGLIARDI I. Contentbased color image retrieval wit h relevance feedback[A]. International Conf on Image Processing[C]. Kobe, J apan, 1999.
[35]GUO G, ZHANG H, LI S. Boosting for contentbased audio classification an d r etrieval: an evaluation[R]. Microsoft Research Technical Report: MSRTR200 1-15,2001.
[36]XU Y, SABER E, TEKALP A. Hierarchical content description and object for mation by learning[A]. IEEE Workshop CBAIVL[C]. Colorado, USA, 1999.
[37]RATAN A, GRIMSON M, LOZANO P. A framework for learning query concepts in im age classification[A]. IEEE Conf Computer Vision and Pattern Recognition[C ]. Fort Collins, USA, 1999.
[38]FORSYTH D, FLECK M. Finding people and animals by guided assembly[A]. International Conf on Image Processing[C].Santa Barbara, USA, 1997.
【39]HONG P, HUANG T. Spatial pattern discovering by learning the isomorphic s ub graph from multiple attributed relation graphs[A]. 8th International Wo rkshop on Combinatorial Image Analysis[C]. Pniladelphia,USA, 2001.
[40]TSAI W, FU K. Errorcorrecting isomorphism of attributed relational grap hs for pattern analysis[J]. IEEE Transaction System Man Cybern, 1979, 9(12): 757 -768.
[41]OYEKOYA O, STENTIFORD F. Eye tracking as a new interface for image retrie val[J]. BT Technology, 2004, 22(7):161-169.
[42]KASTER T, PFEIFFER M, BAUCKHAGE C. Combining speech and haptics for intui ti ve and efficient navigation through image databases[A]. ICMI 2003[C]. Vanco uver, Canada, 2003.


[1]王 巍,王志良,郑思仪,等.人机交互中的个性化情感模型[J].智能系统学报,2010,5(01):10.
 WANG Wei,WANG Zhi-liang,ZHENG Si-yi,et al.Affective model in humanrobot interaction[J].CAAI Transactions on Intelligent Systems,2010,5(04):10.
 CHI Jiannan,WANG Zhiliang,XIE Xiuzhen,et al.A survey of multitouch humancomputer interaction technology[J].CAAI Transactions on Intelligent Systems,2011,6(04):28.
 XIN Yuxuan,YAN Zifei.Research progress of image retrieval based on hand-drawn sketches[J].CAAI Transactions on Intelligent Systems,2015,10(04):167.[doi:10.3969/j.issn.1673-4785.201401045]
 CHENG Yu,ZHANG Mingyu,TAO Linmi.One-Bit human-computer interactive system[J].CAAI Transactions on Intelligent Systems,2015,10(04):528.[doi:10.3969/j.issn.1673-4785.201501015]
 XIN Jizhong,KE Xianxin,YANG Yang,et al.Development of the system of a humanoid robot head with facial expressions[J].CAAI Transactions on Intelligent Systems,2015,10(04):555.[doi:10.3969/j.issn.1673-4785.201503025]
 JIANG Jiliang,TU Dawei.Human-robot collaboration navigation of service robots for the elderly and disabled in an intelligent space[J].CAAI Transactions on Intelligent Systems,2014,9(04):560.[doi:10.3969/j.issn.1673-4785.201307001]
 ZHANG Yi,YIN Chunlin,CAI Jun.On a hybrid electroencephalograph and visual information intelligentwheelchair human-machine interactive system[J].CAAI Transactions on Intelligent Systems,2016,11(04):648.[doi:10.11992/tis.201511004]
[8]JABEEN Farzana,田琳琳,任怡,等.针对可穿戴设备的虚拟鼠标[J].智能系统学报,2017,12(02):133.[doi:10.11992/tis.201608003]
 JABEEN Farzana,TIAN Linlin,REN Yi,et al.Virtual mouse for wearable display[J].CAAI Transactions on Intelligent Systems,2017,12(04):133.[doi:10.11992/tis.201608003]
 LI Xue,JIANG Shuqiang.Incremental learning and object recognition system based on intelligent HCI: a survey[J].CAAI Transactions on Intelligent Systems,2017,12(04):140.[doi:10.11992/tis.201701006]
 MAO Lina,LI Weihua.Research on enhancing the human-machine modeling ability for an extension model using the intelligent guide and KDML[J].CAAI Transactions on Intelligent Systems,2017,12(04):348.[doi:10.11992/tis.201610017]


教育部博士点基金资助项目(2004 0005015);
the PolyU/UGC grants (B-Q698)
作者简介: 
张  菁,女,1975年生,讲师,博士研究生,主要研究方向为多媒体信息检索,发表学术论文10余篇. E-mail:zhj@biut.edu.cn.
 David Dagan FENG,男,1950年生,悉尼大学教授、香港理工大学教授,ACS、 ATSE、 H KIE、 IEE和IEEE会员,主要研究方向为生物医学和多媒体信息处理、功能图像、模拟与仿真、快速算法与数据压缩等,发表学术论文300余篇.
更新日期/Last Update: 2009-05-07