[1]MA Zhiqiang,LI Tuya,YANG Shuangtao,et al.Mongolian acoustic modeling based on deep neural network[J].CAAI Transactions on Intelligent Systems,2018,13(3):486-492.[doi:10.11992/tis.201710029]
Copy
Mongolian acoustic modeling based on deep neural network
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
13
Number of periods:
2018 3
Page number:
486-492
Column:
学术论文—自然语言处理与理解
Public date:
2018-05-05
- Title:
- Mongolian acoustic modeling based on deep neural network
- Keywords:
- speech recognition; acoustic model; GMM-HMM; DNN-HMM; supervised learning; pre-training; over-fitting; dropout
- CLC:
- TP391
- DOI:
- 10.11992/tis.201710029
- Abstract:
- Considering the difficulty of using the Gaussian mixture model (GMM) to adequately describe the correlation and independence hypothesis of the Mongolian acoustic features in the acoustic modeling of Mongolian speech recognition, this study investigates an acoustic model based on deep neural network (DNN). Firstly, using DNN, the internal structure of phonetic features were classified and learned to extract the Mongolian acoustic features, and a DNN-HMM Mongolian acoustic model was constructed. Secondly, a training algorithm was designed by combining unsupervised pre-training and supervised training tuning. In addition, dropout technology was added into the DNN-HMM Mongolian acoustic model training to avoid the over-fitting phenomenon. Finally, a comparative experiment was conducted for the GMM-HMM and DNN-HMM Mongolian acoustic models on basis of the small-scale corpus and Kaldi experimental platform. Experimental results show that the word recognition error rate of DNN-HMM Mongolian model was reduced by 7.5% and sentence recognition error rate was reduced by 13.63%. In addition, the over-fitting of DNN-HMM Mongolian acoustic model can be effectively avoided by adopting the dropout technique during training.
- References:
-
[1] 马志强, 张泽广, 闫瑞, 等. 基于N-Gram模型的蒙古语文本语种识别算法的研究[J]. 中文信息学报, 2016, 30(1):133-140. MA Zhiqiang, ZHANG Zeguang, YAN Rui, et al. N-Gram based language identification for Mongolian text[J]. Journal of Chinese information processing, 2016, 30(1):133-140.
[2] RABINER L R. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2):257-286.
[3] RABINER L, JUANG B H. Fundamentals of Speech Recognition[M]. Upper Saddle River, USA:Prentice-Hall, 1993.
[4] RENALS S, MORGAN N, BOURLARD H, et al. Connectionist probability estimators in HMM speech recognition[J]. IEEE transactions on speech and audio processing, 1994, 2(1):161-174.
[5] LI Deng, HINTON G, KINGSBURY B. New types of deep neural network learning for speech recognition and related applications:an overview[C]//Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, Canada, 2013:8599-8603.
[6] HINTON G, DENG Li, YU Dong, et al. Deep neural networks for acoustic modeling in speech recognition:the shared views of four research groups[J]. IEEE signal processing magazine, 2012, 29(6):82-97.
[7] YU Dong, DENG Li, DAHL G E. Roles of pre-training and fine-tuning in context-dependent DBN-HMMs for real-world speech recognition[C]//Proceedings of NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2010.
[8] DAHL G E, YU Dong, DENG Li, et al. Large vocabulary continuous speech recognition with context-dependent DBN-HMMs[C]//Proceedings of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing. Prague, Czech Republic, 2011:4688-4691.
[9] DAHL G E, YU Dong, DENG Li, et al. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition[J]. IEEE transactions on audio, speech, and language processing, 2012, 20(1):30-42.
[10] HINTON G E. Training products of experts by minimizing contrastive divergence[J]. Neural computation, 2002, 14(8):1771-1800.
[11] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural computation, 2006, 18(7):1527-1554.
[12] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[M]//SCH?LKOPF B, PLATT J, HOFFMAN T. Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2007:19-153.
[13] HINTON G E. A practical guide to training restricted Boltzmann machines[R]. Toronto:University of Toronto, 2010:926-927.
[14] KHALTA B O, FUJⅡ A. A lemmatization method for Mongolian and its application to indexing for information retrieval[J]. Information processing & management, 2009, 45(4):438-451.
[15] JAIMAI P, ZUNDUI T, CHAGNAA A, et al. PC-KIMMO-based description of Mongolian morphology[J]. International journal of information processing systems, 2005, 1(1):41-48.
[16] GAO Guanglai, BILIGETU, NABUQING, et al. A Mongolian speech recognition system based on HMM[C]//Proceedings of 2006 International Conference on Intelligent Computing. Kunming, China, 2006:667-676.
[17] 飞龙, 高光来, 闫学亮, 等. 基于分割识别的蒙古语语音关键词检测方法的研究[J]. 计算机科学, 2013, 40(9):208-211. FEI Long, GAO Guanglai, Yan Xueliang, et al. Research on Mongolian spoken term detection method based on segmentation recognition[J]. Computer science, 2013, 40(9):208-211.
[18] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[J]. arXiv:1207.0580, 2012.
[19] SRIVASTAVA N. Improving neural networks with dropout[D]. Toronto:University of Toronto, 2013.
[20] DENG Li, YU Dong. Deep learning:methods and applications[J]. Foundations and trends in signal processing, 2014, 7(3/4):197-387.
- Similar References:
Memo
-
Last Update:
2018-06-25