[1]WANG Quanbin,TAN Ying.Chinese grammatical error correction method based on data augmentation and copy mechanism[J].CAAI Transactions on Intelligent Systems,2020,15(1):99-106.[doi:10.11992/tis.202001014]
Copy

Chinese grammatical error correction method based on data augmentation and copy mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 15 Number of periods: 2020 1 Page number: 99-106 Column: 学术论文—自然语言处理与理解 Public date: 2020-01-05
Title:
Chinese grammatical error correction method based on data augmentation and copy mechanism
Author(s):
WANG Quanbin;  TAN Ying
School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China
Keywords:
self-attention mechanism; copy mechanism; sequence to sequence learning; Chinese; grammatical error correction; neural networks; text generation; fluency
CLC:
TP389.1
DOI:
10.11992/tis.202001014
Abstract:
Chinese is a widely used language. However, due to its natural difference between Indo-European languages, Chinese learners tend to make various grammatical errors. This article proposes an automatic grammar correction method for those who will make errors like typos or improper words order. First, we built the C-Transformer model that adopts copy mechanism in the self-attention model to translate wrong text sequence to the correct one. Second, based on the public data set, a pure sequence to sequence method is utilized to generate wrong text corresponding to the correct one, and an error text filter is designed based on fluency, semantic, and syntactic measurements. Finally, since Chinese words are pictographic, based on the collected homographs and homophones dictionaries, some error samples are artificially constructed to expand training data. The experimental results show that our method can well correct typos, improper word order, missing, redundancy and other errors, and achieved the state-of-the-art performance on the standard test set of Chinese text grammatical error correction.
References:
[1] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Long Beach, USA, 2017: 5998–6008.
[2] VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, USA, 2015: 2692–2700.
[3] MACDONALD N, FRASE L, GINGRICH P, et al. The writer’s workbench: computer aids for text analysis[J]. IEEE transactions on communications, 1982, 30(1): 105–110.
[4] FRANCIS W N, KUCERA H. A standard corpus of present-day edited American English, for use with digital computers[R]. Providence, RI: Department of Linguistics, Brown University, 1979.
[5] MANGU L, BRILL E. Automatic rule acquisition for spelling correction[C]//Proceedings of the Fourteenth International Conference on Machine Learning. San Francisco, USA, 1997: 187–194.
[6] CAHILL A, MADNANI N, TETREAULT J, et al. Robust systems for preposition error correction using Wikipedia revisions[C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Atlanta, USA, 2013: 507–517.
[7] BROCKETT C, DOLAN W B, GAMON M. Correcting ESL errors using phrasal SMT techniques[C]//Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Stroudsburg, USA, 2006: 249–256.
[8] JUNCZYS-DOWMUNT M, GRUNDKIEWICZ R. Phrase-based machine translation is state-of-the-art for automatic grammatical error correction[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas, USA, 2016: 1546–1556.
[9] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, USA, 2014: 3104–3112.
[10] XIE Z, AVATI A, ARIVAZHAGAN N, et al. Neural language correction with character-based attention[J]. arXiv preprint arXiv: 1603.09727, 2016.
[11] WANG Quanbin, TAN Ying. Automatic grammatical error correction based on edit operations information[C]//Proceedings of 25th International Conference on Neural Information Processing. Siem Reap, Cambodia, 2018: 494–505.
[12] GE Tao, WEI Furu, ZHOU Ming. Fluency boost learning and inference for neural grammatical error correction[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia, 2018: 1055–1065.
[13] ZHAO Wei, WANG Liang, SHEN Kewei, et al. Improving grammatical error correction via pre-training a copy-augmented architecture with unlabeled data[J]. arXiv preprint arXiv: 1903.00138, 2019.
[14] RAO Gaoqi, ZHANG Baolin, XUN Endong, et al. IJCNLP-2017 Task 1: Chinese grammatical error diagnosis[C]//Proceedings of the IJCNLP 2017. Taipei, China, 2017: 1–8.
[15] WU S H, LIU Chaolin, LEE L H. Chinese spelling check evaluation at SIGHAN Bake-off 2013[C]//Proceedings of the Seventh SIGHAN Workshop on Chinese Language Processing. Nagoya, Japan, 2013: 35–42.
[16] ZHAO Yuanyuan, JIANG Nan, SUN Weiwei, et al. Overview of the NLPCC 2018 shared task: grammatical error correction[C]//Proceedings of 7th CCF International Conference on Natural Language Processing and Chinese Computing. Hohhot, China, 2018: 439–445.
[17] ZHOU Junpei, LI Chen, LIU Hengyou, et al. Chinese grammatical error correction using statistical and neural models[C]//Proceedings of 7th CCF International Conference on Natural Language Processing and Chinese Computing. Hohhot, China, 2018: 117–128.
[18] FU Kai, HUANG Jin, DUAN Yitao. Youdao’s winning solution to the NLPCC-2018 Task 2 challenge: a neural machine translation approach to Chinese grammatical error correction[C]//Proceedings of 7th CCF International Conference on Natural Language Processing and Chinese Computing. Hohhot, China, 2018: 341–350.
[19] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735–1780.
[20] LECUN Y, BENGIO Y. Convolutional networks for images, speech, and time-series[M]//ARBIB M A. The Handbook of Brain Theory and Neural Networks. Cambridge, USA: MIT Press, 1995: 3361.
[21] REN Honghai, YANG Liner, XUN Endong. A sequence to sequence learning for Chinese grammatical error correction[C]//Proceedings of 7th CCF International Conference on Natural Language Processing and Chinese Computing. Hohhot, China, 2018: 401–410.
[22] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473, 2014.
[23] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint arXiv: 1412.3555, 2014.
[24] SEE A, LIU P J, MANNING C D. Get to the point: summarization with pointer-generator networks[J]. arXiv preprint arXiv: 1704.04368, 2017.
[25] JIA R, LIANG P. Data recombination for neural semantic parsing[J]. arXiv preprint arXiv: 1606.03622, 2016.
[26] LIU CHAOLIN, LAI MINHUA, TIEN K W, et al. Visually and phonologically similar characters in incorrect Chinese words: analyses, identification, and applications[J]. ACM transactions on Asian language information processing, 2011, 10(2): 10.
[27] A project for conversion between traditional and simplified Chinese[EB/OL]. [2019-12-20].https://github.com/BYVoid/OpenCC.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems