CHI Jiannan,WANG Zhiliang,XIE Xiuzhen,et al.A survey of multitouch humancomputer interaction technology[J].CAAI Transactions on Intelligent Systems,2011,6(01):28-37.





A survey of multitouch humancomputer interaction technology
1.北京科技大学 信息工程学院,北京 100083;
2.北京控制工程研究所,北京 100190
CHI Jiannan1 WANG Zhiliang1 XIE Xiuzhen1 XU Kai2 FANG Hongliang1 XU Yongyun1 LI Ying1
1.School of Information Engineering, University of Science and Technology Beijing, Beijing 100083, China;
2.Beijing Institute of Control Engineering, Beijing 100190, China
multitouch humancomputer interaction hardware system structure software system framework
Multitouch technology was reviewed based on optics from four aspects: hardware, software, application domains, and development trends. For hardware, the structure of the hardware system was introduced along with five technologies which are used to build optical multitouch systems and the projection technology that is used to build a large multitouch screen. Concerning software, the way to retrieve the information of contacts from a hardware platform was introduced first, and then the relevant contents of gesture recognition were shown. The multitouch software framework was given, and at the same time the method of passing the information of contacts to the upper applications through the TUIO protocol was explained. Finally, this paper describes the application domains and development trends of multitouch technology.


[1]HAN J Y. Low cost multitouch sensing through frustrated total internal refection[C]//Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology. Seattle, USA, 2005: 115118.
[2]HAN J Y. Multitouch interaction wall[C]// Proceedings of the 33rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2006). New York, USA: ACM, 2006: 2526.
 [3]MOTAMEDI N. HD touch: multitouch and object sensing on a high definition LCD TV[C]//Proceedings of CHI’08 Extended Abstracts on Human Factors in Computing Systems. New York, USA: ACM, 2008: 30693074.
[4]NUI Group. Diffused illumination(DI)[EB/OL]. [20100920]. http://wiki.nuigroup.com/Diffused_Illumination.
[5]NUI Group. Laser light plane illumination(LLP)[EB/OL]. [20100920]. http://wiki.nuigroup.com/LLP.
[6]T Roth. DSI—diffused surface illumination[EB/OL]. [20100920]. http://iad.projects.zhdk.ch/multitouch/.
[7]About multitouch:基于光学原理的多点触摸硬件技术[EB/OL]. [20100920]. http://files.cnblogs.com/ Thons/基于光学原理的多点触摸技术.pdf.
[8]CHEN C H, NIEN K H, WU F G. Design a multitouch table and apply to interior furniture allocation[J]. Lecture Notes in Computer Science, 2009, 5615: 1319.
[9]PETER B. Visualization and prototyping of a multitouch display device[EB/OL]. [20100920]. http://epubl.ltu.se/14021773/2007/174/LTUCUPP07174SE.p.
[10]CRUZNEIRA C, SANDIN D J, DEFANTI T A. Surround screen projectionbased virtual reality: the design and implementation of the CAVE[C]//Proceedings of the 20st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 1993). New York, USA: ACM, 1993: 135142.
[11]HUMPHEYS G, ELDRIDGE M, BUCK I, et al. Wire GL: a scalable graphics system for clusters[C]//Proceedings of the 28th Annual Conference on Computer Graphics (SIGGRAPH 2001). Los Angeles, USA: ACM, 2001: 129140.
[12]CHEN Yuqun, CLARK D W, FINKELSTEIN A, et al. Automatic alignment of highresolution multiprojector displays using an uncalibrated camera[C]//Proceedings of IEEE Visualization. Salt Lake City, USA: IEEE Computer Society and ACM, 2000: 813.
[13]RASKAR R, JEROEN B, CHAI Jinxiang. A lowcost projector mosaic with fast registration[C]//The 5th Asian Conference on Computer Vision. Melbourne, Australia, 2002: 2325.
[14]PAILTHORPE B, BORDES N, BLEHA W. High resolution display with uniform illumination[C]//Proceedings Asia Display Conference. Nagoya, Japan, 2001: 12951298.
[15]MORELAND J L, REINSCH S. HDD: a single light source uniform tiled display[EB/OL]. [20100920]. http://www.sidchapters.org/sandiego/HDD_Talk.pdf.
[16]STONE M C. Color balancing experimental projection displays[C]//Proceedings of the 9th IST/SID Color Imaging Conference. [S.l.], 2001: 7983.
[17]WALLACE G, CHEN Han, LI Kai. Color gamut matching for tiled display walls[C]//Proceedings of the Workshop on Virtual Environments. New York, USA: ACM, 2003: 2027.
[18]侯洪涛,朱一凡,韦庆,等.基于PC集群的多屏无缝拼接技术研究[J].计算机仿真, 2006, 23(11): 2528. HOU Hongtao, ZHU Yifan, WEI Qing, et al. The study of multiscreen seamless technology based on PC cluster[J]. Computer Simulation, 2006, 23(11): 2528.
[19]夏季.多投影仪画面的融合技术探讨[J].中国科技成果, 2006, 20(19): 3338. XIA Ji. The discussion of the technology of the integration of multiprojection screen[J]. China Science and Technology Achievements, 2006, 20(19): 3338. 
 [20]王胜正,杨杰.自动多投影仪非线性几何校正与图像边缘融合方法[J].上海交通大学学报, 2008, 25(4): 6569.
WANG Shengzheng, YANG Jie. Autononlinear geometry calibration and edge blending of multiprojector display system[J]. Journal of Shanghai Jiaotong University, 2008, 25(4): 6569.
[21]NUI Group. Frustrated total internal reflection (FTIR)[EB/OL]. [20100921]. http://wiki.nuigroup.com/ FTIR.
[22]WANG Feng, REN Xiangshi, LIU Zhen. A robust blob recognition and tracking method in visionbased multitouch technique[C]//IEEE International Symposium on Parallel and Distributed Processing with Applications. Sydney, Australia, 2008: 971974.
[23]VARCHOLIK P, LAVIOLA J J Jr, NICHOLSON D. TACTUS: a hardware and software testbed for research in multitouch interaction[EB/OL]. [20100921]. http://www.eecs.ucf.edu/~jjl/pubs/HCI2009_final.pdf.
[24]祁彩萍.图像图形技术研究与应用[M].北京:中国传媒大学出版社, 2010: 4046.
[25]张雪峰.触摸屏技术浅谈[J].现代物理知识, 2004, 16(3): 4345.
ZHANG Xuefeng. Discussion of touch screen technology[J]. Modern Physics, 2004, 16(3): 4345.
[26]FLORIAN E, HUBER M, KLINKER G. Hand tracking for enhanced gesture recognition on interactive multitouch surfaces[EB/OL]. [20100921]. http://ar.in.tum.de/pub/echtler2007handtrack/echtler2007handtrack.pdf.
[27]JUMDERA M, JONES D, MCCRORY M. Using a camera to capture and correct spatial photometric variation in multiprojector displays[C]//Proceedings of IEEE International Workshop on Projectorcamera Systems. Nice, France, 2003: 118139.
[28]黄菁.基于单目视觉的触摸屏技术研究[D].杭州:浙江工商大学, 2009: 163.
 HUANG Jing. Researches on monocular visionbased touch screen technology[D]. Hangzhou: Zhejiang Gongshang University, 2009: 163.
[29]BUXTON W, MYERS B. A study in two handed input[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 1986: 321326.
[30]凌云翔.基于多点触摸的自然手势识别方法研究[J].国防科技大学学报, 2010, 32(1): 127132.
LING Yunxiang. Research on natural gesture recognition method based on multitouch[J]. Journal of National University of Defense Technology, 2010, 32(1): 127132.
[31]MURAKAMIT H. Gesture recognition using recurrent neural networks[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching through Technology. New Orleans, USA, 1991: 237242.
[32]Touchlib hompage[EB/OL]. [2010921]. http:// www.whitenoiseaudio.com/touchlib/.
[33]陈大炜.基于光学感应的大尺寸多点触摸控制技术[J].电子产品世界, 2009, 5(1): 2226.
CHEN Dawei. Largesize multitouch technology based on optical sensing[J]. Electronic Engineering & Product World, 2009, 5(1): 2226.
[34]李正民.Touchlib源码注释[EB/OL]. [20100921]. http://blog.csdn.net/li_007/archive/2010/01/05/5134270.as .


[1]张 菁,沈兰荪,David Dagan FENG.图像搜索中人机交互技术的新进展[J].智能系统学报,2007,2(04):14.
 ZHANG Jing,SHEN Lan-sun,David Dagan FENG.computer interaction technology in image searches: a survey[J].CAAI Transactions on Intelligent Systems,2007,2(01):14.
[2]王 巍,王志良,郑思仪,等.人机交互中的个性化情感模型[J].智能系统学报,2010,5(01):10.
 WANG Wei,WANG Zhi-liang,ZHENG Si-yi,et al.Affective model in humanrobot interaction[J].CAAI Transactions on Intelligent Systems,2010,5(01):10.
 XIN Yuxuan,YAN Zifei.Research progress of image retrieval based on hand-drawn sketches[J].CAAI Transactions on Intelligent Systems,2015,10(01):167.[doi:10.3969/j.issn.1673-4785.201401045]
 CHENG Yu,ZHANG Mingyu,TAO Linmi.One-Bit human-computer interactive system[J].CAAI Transactions on Intelligent Systems,2015,10(01):528.[doi:10.3969/j.issn.1673-4785.201501015]
 XIN Jizhong,KE Xianxin,YANG Yang,et al.Development of the system of a humanoid robot head with facial expressions[J].CAAI Transactions on Intelligent Systems,2015,10(01):555.[doi:10.3969/j.issn.1673-4785.201503025]
 JIANG Jiliang,TU Dawei.Human-robot collaboration navigation of service robots for the elderly and disabled in an intelligent space[J].CAAI Transactions on Intelligent Systems,2014,9(01):560.[doi:10.3969/j.issn.1673-4785.201307001]
 ZHANG Yi,YIN Chunlin,CAI Jun.On a hybrid electroencephalograph and visual information intelligentwheelchair human-machine interactive system[J].CAAI Transactions on Intelligent Systems,2016,11(01):648.[doi:10.11992/tis.201511004]
[8]JABEEN Farzana,田琳琳,任怡,等.针对可穿戴设备的虚拟鼠标[J].智能系统学报,2017,12(02):133.[doi:10.11992/tis.201608003]
 JABEEN Farzana,TIAN Linlin,REN Yi,et al.Virtual mouse for wearable display[J].CAAI Transactions on Intelligent Systems,2017,12(01):133.[doi:10.11992/tis.201608003]
 LI Xue,JIANG Shuqiang.Incremental learning and object recognition system based on intelligent HCI: a survey[J].CAAI Transactions on Intelligent Systems,2017,12(01):140.[doi:10.11992/tis.201701006]
 MAO Lina,LI Weihua.Research on enhancing the human-machine modeling ability for an extension model using the intelligent guide and KDML[J].CAAI Transactions on Intelligent Systems,2017,12(01):348.[doi:10.11992/tis.201610017]


更新日期/Last Update: 2011-04-13