[1]ZHU Wenlin,LIU Huaping,WANG Bowen,et al.An intelligent blind guidance system based on visual-touch cross-modal perception[J].CAAI Transactions on Intelligent Systems,2020,15(1):33-40.[doi:10.11992/tis.201908015]
Copy

An intelligent blind guidance system based on visual-touch cross-modal perception

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 15 Number of periods: 2020 1 Page number: 33-40 Column: 学术论文—智能系统 Public date: 2020-01-05
Title:
An intelligent blind guidance system based on visual-touch cross-modal perception
Author(s):
ZHU Wenlin1;  LIU Huaping2;  WANG Bowen1;  SUN Fuchun2
1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300130, China;
2. State Key Lab. of Intelligent Technology and Systems, Tsinghua University, Beijing 100084, China
Keywords:
blind users; electronic cane; cross-modal technology; touch; data set; deep learning; computer vision; GANs
CLC:
TP391.4
DOI:
10.11992/tis.201908015
Abstract:
Blind mobility aid is a primary part of the daily life for blind people. Most of these technologies are used to help them navigate and avoid obstacles, and few researches have been done to convert ground information into tactile sensation that gives the user an intuitive feeling. To meet the above requirements, we proposed an intelligent guided rod system, which can provide tactile feedback to assist the blind to recognize ground information. We attempted to generate the vibrotactile stimuli leveraging the power of deep generative adversarial training. Specifically, we used an improved DiscoGAN training an end-to-end generated network. To train the network, we also built GroVib, a visual touch cross-modal dataset. We set up computer experiments and physical experiments to evaluate the feasibility of the scheme. The results of computer experiments showed that the accuracy rate of the participants in recognizing the ground by tactile sense was 84.7%, and the average real feeling score of tactile sense was 71.3. In real scene experiments, the participants needed only 3.25 times of attempts on average to recognize the ground based on tactile feedback.
References:
[1] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005:539–546
[2] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, USA, 2014: 2672–2680.
[3] MIRZA M, OSINDERO S. Conditional generative adversarial nets[J]. arXiv: 1411.1784, 2014.
[4] REED S, AKATA Z, YAN Xinchen, et al. Generative adversarial text to image synthesis[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York, USA, 2016: 1060–1069.
[5] BAHADIR S K, KONCAR V, KALAOGLU F. Wearable obstacle detection system fully integrated to textile structures for visually impaired people[J]. Sensors and actuators A: physical, 2012, 179: 297–311.
[6] SHIN B S, LIM C S. Obstacle detection and avoidance system for visually impaired people[C]//Proceedings of the 2nd International Workshop on Haptic and Audio Interaction Design. Seoul, South Korea, 2007: 78–85.
[7] BOUSBIA-SALAH M, BETTAYEB M, LARBI A. A navigation aid for blind people[J]. Journal of intelligent & robotic systems, 2011, 64(3/4): 387–400.
[8] HASANUZZAMAN F M, YANG Xiaodong, TIAN Yingli. Robust and effective component-based banknote recognition for the blind[J]. IEEE transactions on systems, man, and cybernetics, part C (applications and reviews), 2012, 42(6): 1021–1030.
[9] GUEST S, DESSIRIER J, MEHRABYAN A. The development and validation of sensory and emotional scales of touch perception[J]. Attention perception & psychophysics, 2011, 73(2): 531–550.
[10] KIM D Y, YI K Y. A user-steered guide robot for the blind[C]//Proceedings of 2008 IEEE International Conference on Robotics and Biomimetics. Bangkok, Thailand, 2009: 114–119.
[11] TIWANA M, REDMOND S, LOVELL N. A review of tactile sensing technologies with applications in biomedical engineering[J]. Sensors and actuators: a physical, 2012, 179(5):17–31.
[12] TANG T J J, LUI W L D, LI W H. Plane-based detection of staircases using inverse depth[C]//Proceedings of 2012 Australasian Conference on Robotics and Automation. New Zealand, 2012: 1–10.
[13] AL KALBANI J, SUWAILAM R B, AL YAFAI A, et al. Bus detection system for blind people using RFID[C]//Proceedings of the 2015 IEEE 8th GCC Conference & Exhibition. Muscat, Oman, 2015: 1–6.
[14] KULKARNI A, BHURCHANDI K. Low cost E-book reading device for blind people[C]//Proceedings of 2015 International Conference on Computing Communication Control and Automation. Pune, India, 2015: 516–520.
[15] THILAGAVATHI B. Recognizing clothes patterns and colours for blind people using neural network[C]//Proceedings of 2015 International Conference on Innovations in Information, Embedded and Communication Systems. Coimbatore, India, 2015: 1–5.
[16] NICHOLLS H, LEE M. A survey of robot tactile sensing technology[J]. The international journal of robotics research, 1989, 8(3):3–30.
[17] STRESE M, SCHUWERK C, IEPURE A, et al. Multimodal feature-based surface material classification[J]. IEEE transactions on haptics, 2017, 10(2): 226–239.
[18] LI Xinwu, LIU Huaping, ZHOU Junfeng, et al. Learning cross-modal visual-tactile representation using ensembled generative adversarial networks[J]. Cognitive computation and systems, 2019, 1(2): 40–44.
[19] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 5967–5976.
[20] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2242–2251.
[21] KIM T, CHA M, KIM H, et al. Learning to discover cross-domain relations with generative adversarial networks[C]//Proceedings of the 34th International Conference on Machine Learning. Sydney, Australia, 2017: 1857–1865.
[22] GRIFFIN D, LIM J. Signal estimation from modified short-time fourier transform[J]. IEEE transactions on acoustics, speech, and signal processing, 1984, 32(2): 236–243.
[23] UJITOKO Y, BAN Y. Vibrotactile signal generation from texture images or attributes using generative adversarial network[C]//Proceedings of the 11th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Pisa, Italy, 2018: 25–36.
[24] HUANG G, WANG D, LAN Y. Extreme learning machines: a survey[J]. International journal of machine learning & cybernetics, 2011, 2(2): 107–122.
[25] LEE K A, HICKS G, NINO-MURCIA G. Validity and reliability of a scale to assess fatigue[J]. Psychiatry research, 1991, 36(3): 291–298.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems