[1]LI Qiang,WU Zhengbiao,GUAN Xin.Piano fingering generation with deep musical score feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(6):1287-1294.[doi:10.11992/tis.202303018]
Copy

Piano fingering generation with deep musical score feature fusion

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 6 Page number: 1287-1294 Column: 学术论文—智能系统 Public date: 2023-11-05
Title:
Piano fingering generation with deep musical score feature fusion
Author(s):
LI Qiang;  WU Zhengbiao;  GUAN Xin
School of Microelectronics, Tianjin University, Tianjin 300072, China
Keywords:
artificial intelligence; music; information retrieval; long short-term memory; recurrent neural networks; data processing; feature extraction; time series
CLC:
TP391
DOI:
10.11992/tis.202303018
Abstract:
Fingering is a key technique in piano playing. However, most musical scores have no finger notation except in beginners’ textbooks. The HMM and LSTM models used for automatic piano fingering only model pitch information and ignore speed information, which will influence the fingering. This condition results in insufficient extraction of comprehensive features and a low accuracy rate for generated fingerings. A feature extraction method was first designed using the pitch and speed information of the musical score simultaneously to address these problems. The Word2Vec-CBOW model was then introduced to produce a fused feature vector. Further, data enhancement and joint training of left and right hand sequences were conducted on the original data according to the mirror symmetric characteristics of human left and right hands. Finally, the generation of fingering was realized by combining the bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model. Experimental results show that the proposed algorithm is considerably better than commonly used statistical and deep learning methods, which confirms the rationality and effectiveness of the proposed model.
References:
[1] PURWINS H, LI Bo, VIRTANEN T, et al. Deep learning for audio signal processing[J]. IEEE journal of selected topics in signal processing, 2019, 13(2): 206–219.
[2] PARNCUTT R, SLOBODA J A, CLARKE E F, et al. An ergonomic model of keyboard fingering for melodic fragments[J]. Music perception, 1997, 14(4): 341–382.
[3] BALLIAUW M, HERREMANS D, PALHAZI CUERVO D, et al. A variable neighborhood search algorithm to generate piano fingerings for polyphonic sheet music[J]. International transactions in operational research, 2017, 24(3): 509–535.
[4] AL K, ALI A. A simple algorithm for automatic generation of polyphonic piano fingerings[C]//The International Society for Music Information Retrieval. Vienna: ISMIR, 2007.
[5] XIAO Yisheng, WU Lijun, GUO Junliang, et al. A survey on non-autoregressive generation for neural machine translation and beyond[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(10): 11407–11427.
[6] LI Jing, SUN Aixin, HAN Jianglei, et al. A survey on deep learning for named entity recognition[J]. IEEE transactions on knowledge and data engineering, 2022, 34(1): 50–70.
[7] LI Jing, HAN Peng, REN Xiangnan, et al. Sequence labeling with meta-learning[J]. IEEE transactions on knowledge and data engineering, 2023, 35(3): 3072–3086.
[8] YONEBAYASHI Y, KAMEOKA H, SAGAYAMA S. Automatic decision of piano fingering based on hidden Markov models[C]//Proceedings of the 20th International Joint Conference on Artifical Intelligence. Hyderabad: ACM, 2007: 2915-2921.
[9] NAKAMURA E, ONO N, SAGAYAMA S. Merged-output HMM for piano fingering of both hands[C]//Proceedings of the 15th International Society for Conference on Music Information Retrieval. Taipei: ISMIR, 2014: 531-536
[10] SAHA S, BOVOLO F, BRUZZONE L. Change detection in image time-series using unsupervised LSTM[J]. IEEE geoscience and remote sensing letters, 2022, 19: 1–5.
[11] WEI Yuanyuan, JANG-JACCARD J, XU Wen, et al. LSTM-autoencoder-based anomaly detection for indoor air quality time-series data[J]. IEEE sensors journal, 2023, 23(4): 3787–3800.
[12] YAN Jingyang, DIMEO P, SUN Lu, et al. LSTM-based model predictive control of piezoelectric motion stages for high-speed autofocus[J]. IEEE transactions on industrial electronics, 2023, 70(6): 6209–6218.
[13] 于润羽, 杜军平, 薛哲, 等. 面向科技学术会议的命名实体识别研究[J]. 智能系统学报, 2022, 17(1): 50–58
YU Runyu, DU Junping, XUE Zhe, et al. Research on named entity recognition for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(1): 50–58
[14] 王一成, 万福成, 马宁. 融合多层次特征的中文语义角色标注[J]. 智能系统学报, 2020, 15(1): 107–113
WANG Yicheng, WAN Fucheng, MA Ning. Chinese semantic role labeling with multi-level linguistic features[J]. CAAI transactions on intelligent systems, 2020, 15(1): 107–113
[15] SIAMI-NAMINI S, TAVAKOLI N, NAMIN A S. The performance of LSTM and BiLSTM in forecasting time series[C]//2019 IEEE International Conference on Big Data (Big Data). Los Angeles: IEEE, 2020: 3285-3292.
[16] WANG Jianyou, XUE M, CULHANE R, et al. Speech emotion recognition with dual-sequence LSTM architecture[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 6474-6478.
[17] LIU Gang, GUO Jiabao. Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J]. Neurocomputing, 2019, 337: 325–338.
[18] NAKAMURA E, SAITO Y, YOSHII K. Statistical learning and estimation of piano fingering[J]. Information sciences, 2020, 517: 68–85.
[19] RAMONEDA P, JEONG D, NAKAMURA E, et al. Automatic piano fingering from partially annotated scores using autoregressive neural networks[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 6502-6510.
[20] STEWART M. Automatic piano fingerings estimation using recurrent neural networks[EB/OL]. (2021-12-29)[2023-03-25].https://api.semanticscholar.org/CorpusID:252005276.
[21] CHANG C C. Fundamentals Of piano practice[M/OL]. [S. l.]: CreateSpace Independent Publishing Platform, 2007.
[22] TITON J T, COOLEY T J. Worlds of music: an introduction to the music of the world’s peoples[M]. Array Boston: Cengage Learning, 2016
[23] CHEN Yichen, HUANG S F, LEE H Y, et al. Audio Word2Vec: sequence-to-sequence autoencoding for unsupervised learning of audio segmentation and representation[J]. IEEE/ACM transactions on audio, speech, and language processing, 2019, 27(9): 1481–1493.
[24] AMIN S, UDDIN M I, ALI ZEB M, et al. Detecting dengue/flu infections based on tweets using LSTM and word embedding[J]. IEEE access, 2020, 8: 189054–189068.
[25] QIU Xipeng, SUN Tianxiang, XU Yige, et al. Pre-trained models for natural language processing: a survey[J]. Science China technological sciences, 2020, 63(10): 1872–1897.
[26] YU Yong, SI Xiaosheng, HU Changhua, et al. A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural computation, 2019, 31(7): 1235–1270.
[27] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735–1780.
[28] VAN HOUDT G, MOSQUERA C, NáPOLES G. A review on the long short-term memory model[J]. Artificial intelligence review, 2020, 53(8): 5929–5955.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems