[1]李锵,吴正彪,关欣.结合深度乐谱特征融合的钢琴指法生成方法[J].智能系统学报,2023,18(6):1287-1294.[doi:10.11992/tis.202303018]
LI Qiang,WU Zhengbiao,GUAN Xin.Piano fingering generation with deep musical score feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(6):1287-1294.[doi:10.11992/tis.202303018]
点击复制
LI Qiang,WU Zhengbiao,GUAN Xin.Piano fingering generation with deep musical score feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(6):1287-1294.[doi:10.11992/tis.202303018]
结合深度乐谱特征融合的钢琴指法生成方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第6期
页码:
1287-1294
栏目:
学术论文—智能系统
出版日期:
2023-11-05
- Title:
- Piano fingering generation with deep musical score feature fusion
- Keywords:
- artificial intelligence; music; information retrieval; long short-term memory; recurrent neural networks; data processing; feature extraction; time series
- 分类号:
- TP391
- 摘要:
- 指法是钢琴演奏的关键技术，但是除了初学者的教科书外，大多数乐谱都没有指法注释。目前用于钢琴指法自动生成的隐马尔可夫模型（hidden Markov model，HMM）和长短时记忆网络（long short-term memory，LSTM）模型，仅针对乐谱的音高建立模型，忽略同样影响指法的速度信息，存在对乐谱综合特征提取能力不足、生成的指法正确率低等问题。针对这些问题，设计一种可以同时利用乐谱的音高信息与速度信息的特征提取方法，并引入Word2Vec-CBOW（continuous bag-of-words）模型得到融合特征向量，根据人体左右手镜像对称的特点对原始数据进行左右手序列的数据增强与联合训练，最后结合双向长短时记忆网络-条件随机场（bidirectional LSTM conditional random field，BiLSTM-CRF）模型实现指法的生成。实验结果显示，本文提出的算法相比常用的统计学习方法和深度学习方法均有明显提高，验证了其合理性和有效性。
- Abstract:
- Fingering is a key technique in piano playing. However, most musical scores have no finger notation except in beginners’ textbooks. The HMM and LSTM models used for automatic piano fingering only model pitch information and ignore speed information, which will influence the fingering. This condition results in insufficient extraction of comprehensive features and a low accuracy rate for generated fingerings. A feature extraction method was first designed using the pitch and speed information of the musical score simultaneously to address these problems. The Word2Vec-CBOW model was then introduced to produce a fused feature vector. Further, data enhancement and joint training of left and right hand sequences were conducted on the original data according to the mirror symmetric characteristics of human left and right hands. Finally, the generation of fingering was realized by combining the bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model. Experimental results show that the proposed algorithm is considerably better than commonly used statistical and deep learning methods, which confirms the rationality and effectiveness of the proposed model.
- 参考文献/References:
-
[1] PURWINS H, LI Bo, VIRTANEN T, et al. Deep learning for audio signal processing[J]. IEEE journal of selected topics in signal processing, 2019, 13(2): 206–219.
[2] PARNCUTT R, SLOBODA J A, CLARKE E F, et al. An ergonomic model of keyboard fingering for melodic fragments[J]. Music perception, 1997, 14(4): 341–382.
[3] BALLIAUW M, HERREMANS D, PALHAZI CUERVO D, et al. A variable neighborhood search algorithm to generate piano fingerings for polyphonic sheet music[J]. International transactions in operational research, 2017, 24(3): 509–535.
[4] AL K, ALI A. A simple algorithm for automatic generation of polyphonic piano fingerings[C]//The International Society for Music Information Retrieval. Vienna: ISMIR, 2007.
[5] XIAO Yisheng, WU Lijun, GUO Junliang, et al. A survey on non-autoregressive generation for neural machine translation and beyond[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(10): 11407–11427.
[6] LI Jing, SUN Aixin, HAN Jianglei, et al. A survey on deep learning for named entity recognition[J]. IEEE transactions on knowledge and data engineering, 2022, 34(1): 50–70.
[7] LI Jing, HAN Peng, REN Xiangnan, et al. Sequence labeling with meta-learning[J]. IEEE transactions on knowledge and data engineering, 2023, 35(3): 3072–3086.
[8] YONEBAYASHI Y, KAMEOKA H, SAGAYAMA S. Automatic decision of piano fingering based on hidden Markov models[C]//Proceedings of the 20th International Joint Conference on Artifical Intelligence. Hyderabad: ACM, 2007: 2915-2921.
[9] NAKAMURA E, ONO N, SAGAYAMA S. Merged-output HMM for piano fingering of both hands[C]//Proceedings of the 15th International Society for Conference on Music Information Retrieval. Taipei: ISMIR, 2014: 531-536
[10] SAHA S, BOVOLO F, BRUZZONE L. Change detection in image time-series using unsupervised LSTM[J]. IEEE geoscience and remote sensing letters, 2022, 19: 1–5.
[11] WEI Yuanyuan, JANG-JACCARD J, XU Wen, et al. LSTM-autoencoder-based anomaly detection for indoor air quality time-series data[J]. IEEE sensors journal, 2023, 23(4): 3787–3800.
[12] YAN Jingyang, DIMEO P, SUN Lu, et al. LSTM-based model predictive control of piezoelectric motion stages for high-speed autofocus[J]. IEEE transactions on industrial electronics, 2023, 70(6): 6209–6218.
[13] 于润羽, 杜军平, 薛哲, 等. 面向科技学术会议的命名实体识别研究[J]. 智能系统学报, 2022, 17(1): 50–58
YU Runyu, DU Junping, XUE Zhe, et al. Research on named entity recognition for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(1): 50–58
[14] 王一成, 万福成, 马宁. 融合多层次特征的中文语义角色标注[J]. 智能系统学报, 2020, 15(1): 107–113
WANG Yicheng, WAN Fucheng, MA Ning. Chinese semantic role labeling with multi-level linguistic features[J]. CAAI transactions on intelligent systems, 2020, 15(1): 107–113
[15] SIAMI-NAMINI S, TAVAKOLI N, NAMIN A S. The performance of LSTM and BiLSTM in forecasting time series[C]//2019 IEEE International Conference on Big Data (Big Data). Los Angeles: IEEE, 2020: 3285-3292.
[16] WANG Jianyou, XUE M, CULHANE R, et al. Speech emotion recognition with dual-sequence LSTM architecture[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 6474-6478.
[17] LIU Gang, GUO Jiabao. Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J]. Neurocomputing, 2019, 337: 325–338.
[18] NAKAMURA E, SAITO Y, YOSHII K. Statistical learning and estimation of piano fingering[J]. Information sciences, 2020, 517: 68–85.
[19] RAMONEDA P, JEONG D, NAKAMURA E, et al. Automatic piano fingering from partially annotated scores using autoregressive neural networks[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 6502-6510.
[20] STEWART M. Automatic piano fingerings estimation using recurrent neural networks[EB/OL]. (2021-12-29)[2023-03-25].https://api.semanticscholar.org/CorpusID:252005276.
[21] CHANG C C. Fundamentals Of piano practice[M/OL]. [S. l.]: CreateSpace Independent Publishing Platform, 2007.
[22] TITON J T, COOLEY T J. Worlds of music: an introduction to the music of the world’s peoples[M]. Array Boston: Cengage Learning, 2016
[23] CHEN Yichen, HUANG S F, LEE H Y, et al. Audio Word2Vec: sequence-to-sequence autoencoding for unsupervised learning of audio segmentation and representation[J]. IEEE/ACM transactions on audio, speech, and language processing, 2019, 27(9): 1481–1493.
[24] AMIN S, UDDIN M I, ALI ZEB M, et al. Detecting dengue/flu infections based on tweets using LSTM and word embedding[J]. IEEE access, 2020, 8: 189054–189068.
[25] QIU Xipeng, SUN Tianxiang, XU Yige, et al. Pre-trained models for natural language processing: a survey[J]. Science China technological sciences, 2020, 63(10): 1872–1897.
[26] YU Yong, SI Xiaosheng, HU Changhua, et al. A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural computation, 2019, 31(7): 1235–1270.
[27] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735–1780.
[28] VAN HOUDT G, MOSQUERA C, NáPOLES G. A review on the long short-term memory model[J]. Artificial intelligence review, 2020, 53(8): 5929–5955.
- 相似文献/References:
-
[1]李德毅.网络时代人工智能研究与发展[J].智能系统学报,2009,4(1):1.
LI De-yi.AI research and development in the network age[J].CAAI Transactions on Intelligent Systems,2009,4():1.
[2]赵克勤.二元联系数A+Bi的理论基础与基本算法及在人工智能中的应用[J].智能系统学报,2008,3(6):476.
ZHAO Ke-qin.The theoretical basis and basic algorithm of binary connection A+Bi and its application in AI[J].CAAI Transactions on Intelligent Systems,2008,3():476.
[3]徐玉如,庞永杰,甘?? 永,等.智能水下机器人技术展望[J].智能系统学报,2006,1(1):9.
XU Yu-ru,PANG Yong-jie,GAN Yong,et al.AUV—state-of-the-art and prospect[J].CAAI Transactions on Intelligent Systems,2006,1():9.
[4]王志良.人工心理与人工情感[J].智能系统学报,2006,1(1):38.
WANG Zhi-liang.Artificial psychology and artificial emotion[J].CAAI Transactions on Intelligent Systems,2006,1():38.
[5]赵克勤.集对分析的不确定性系统理论在AI中的应用[J].智能系统学报,2006,1(2):16.
ZHAO Ke-qin.The application of uncertainty systems theory of set pair analysis (SPU)in the artificial intelligence[J].CAAI Transactions on Intelligent Systems,2006,1():16.
[6]秦裕林,朱新民,朱? 丹.Herbert Simon在最后几年里的两个研究方向[J].智能系统学报,2006,1(2):11.
QIN Yu-lin,ZHU Xin-min,ZHU Dan.Herbert Simons two research directions in his lost years[J].CAAI Transactions on Intelligent Systems,2006,1():11.
[7]谷文祥,李 丽,李丹丹.规划识别的研究及其应用[J].智能系统学报,2007,2(1):1.
GU Wen-xiang,LI Li,LI Dan-dan.Research and application of plan recognition[J].CAAI Transactions on Intelligent Systems,2007,2():1.
[8]杨春燕,蔡 文.可拓信息-知识-智能形式化体系研究[J].智能系统学报,2007,2(3):8.
YANG Chun-yan,CAI Wen.A formalized system of extension information-knowledge-intelligence[J].CAAI Transactions on Intelligent Systems,2007,2():8.
[9]赵克勤.SPA的同异反系统理论在人工智能研究中的应用[J].智能系统学报,2007,2(5):20.
ZHAO Ke-qin.The application of SPAbased identicaldiscrepancycontrary system theory in artificial intelligence research[J].CAAI Transactions on Intelligent Systems,2007,2():20.
[10]王志良,杨?? 溢,杨?? 扬,等.一种周期时变马尔可夫室内位置预测模型[J].智能系统学报,2009,4(6):521.[doi:10.3969/j.issn.1673-4785.2009.06.009]
WANG Zhi-liang,YANG Yi,YANG Yang,et al.A periodic time-varying Markov model for indoor location prediction[J].CAAI Transactions on Intelligent Systems,2009,4():521.[doi:10.3969/j.issn.1673-4785.2009.06.009]
备注/Memo
收稿日期:2023-3-10。
基金项目:国家自然科学基金项目（61872267）；天津市自然科学基金项目（16JCZDJC31100）；天津大学创新基金项目（2021XZC-0024）.
作者简介:李锵,教授,博士生导师,博士,主要研究方向为智能信息处理、医学图像处理、音乐信息检索、数字系统和微系统设计。发表学术论文130余篇;吴正彪,硕士研究生,主要研究方向为音乐信号处理、音乐信息检索;关欣,副教授,博士,主要研究方向为智能信息处理、统计学习和音乐信息检索。发表学术论文60余篇
通讯作者:关欣.E-mail:guanxin@tju.edu.cn
更新日期/Last Update:
1900-01-01