[1]沈学利,卢呈祥,崔益烽,等.基于动态记忆增强的轻量化相位保真语音增强网络[J].智能系统学报,2026,21(3):802-812.[doi:10.11992/tis.202506018]
SHEN Xueli,LU Chengxiang,CUI Yifeng,et al.Lightweight phase-preserving speech enhancement network with dynamic memory augmentation[J].CAAI Transactions on Intelligent Systems,2026,21(3):802-812.[doi:10.11992/tis.202506018]
点击复制
SHEN Xueli,LU Chengxiang,CUI Yifeng,et al.Lightweight phase-preserving speech enhancement network with dynamic memory augmentation[J].CAAI Transactions on Intelligent Systems,2026,21(3):802-812.[doi:10.11992/tis.202506018]
基于动态记忆增强的轻量化相位保真语音增强网络
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第3期
页码:
802-812
栏目:
人工智能院长论坛
出版日期:
2026-05-05
- Title:
- Lightweight phase-preserving speech enhancement network with dynamic memory augmentation
- Keywords:
- deep learning; deep neural networks; intelligent information processing; natural language processing; phase optimization; parameter optimization; memory-augmented networks; robustness
- 分类号:
- TP183；TN912.34
- 文献标志码:
- 2026-3-11
- 摘要:
- 针对复杂声学场景中低信噪比下的相位失真与噪声适应性难题，该研究基于幅度-相位显式语音增强框架提出一种改进的语音增强网络。设计记忆增强时频转换器，利用动态记忆矩阵与门控融合机制提升突发噪声建模能力；通过稀疏检索机制，减小模型参数交互规模，显著降低参数量；构建任务不确定性驱动的动态损失权重，协同优化抗包裹相位恢复、复谱重建与感知质量。相较原模型，改进模型在缩减9.7%参数量的同时，在VoiceBank+DEMAND数据集上，实现低信噪比环境(-5 dB)下宽频带语音质量感知质量(WB-PESQ)提升2.3%；在DNS Challenge数据集上，获得1.33%的性能增益，验证了其在相位保真度与噪声鲁棒性上的有效性。
- Abstract:
- To address phase distortion under low-signal-to-noise ratio (SNR) conditions and inadequate noise adaptability in complex acoustic scenes, this study proposes an enhanced speech enhancement network based on an explicit magnitude-phase framework. First, a memory-enhanced time-frequency transformer is designed. It utilizes a dynamic memory matrix and a gated fusion mechanism to improve modeling of impulsive noise. Second, a sparse retrieval mechanism reduces the scale of parameter interaction, thereby significantly reducing model parameters. Finally, a task-uncertainty-driven dynamic loss weighting strategy is developed to jointly optimize anti-wrapping phase restoration, complex spectral reconstruction, and perceptual quality. Compared with the baseline model, the proposed model achieves a 9.7% reduction in parameters while delivering a 2.3% higher wideband perceptual evaluation of speech quality (WB-PESQ) at –5 dB SNR on the VoiceBank+DEMAND dataset and a 1.33% performance gain on the Domain Name System (DNS) Challenge dataset, demonstrating its effectiveness in phase fidelity and noise robustness.
- 参考文献/References:
-
[1] FU S W, LIAO C F, TSAO Y, et al. MetricGAN: generative adversarial networks based black-box metric scores optimization for speech enhancement[C]//International Conference on Machine Learning. Graz: ISCA, 2019: 464-468.
[2] YIN Dacheng, LUO Chong, XIONG Zhiwei, et al. PHASEN: a phase-and-harmonics-aware speech enhancement network[J]. Proceedings of the AAAI conference on artificial intelligence, 2020, 34(5): 9458-9465
[3] ZHAO Xiaojia, WANG Yuxuan, WANG Deliang. Robust speaker identification in noisy and reverberant conditions[C]//2014 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2014: 3997-4001.
[4] TAN Ke, WANG Deliang. Learning complex spectral mapping with gated convolutional recurrent networks for monaural speech enhancement[J]. IEEE/ACM transactions on audio, speech, and language processing, 2020, 28: 380-390
[5] HASANNEZHAD M, YU Hongjiang, ZHU Weiping, et al. PACDNN: a phase-aware composite deep neural network for speech enhancement[J]. Speech communication, 2022, 136: 1-13
[6] LU Yexin, AI Yang, LING Zhenhua. Explicit estimation of magnitude and phase spectra in parallel for high-quality speech enhancement[EB/OL]. (2023-08-17)[2024-06-20]. https://arxiv.org/abs/2308.08926.
[7] REDDY C K A, GOPAL V, CUTLER R, et al. The INTERSPEECH 2020 deep noise suppression challenge: datasets, subjective testing framework, and challenge results[C]//Interspeech 2020. Graz: ISCA, 2020: 2492-2496.
[8] VANAMBATHINA S, KUMAR T K. Speech enhancement by combining spectral subtraction and auditory masking effect[J]. Microelectronics and computer, 2014, 31(2): 123-128
[9] 李哲, 王静. 基于门控记忆网络的突发噪声抑制方法[J]. 电子学报, 2023, 51(4): 1205-1214 LI Zhe, WANG Jing. Impulsive noise suppression via gated memory network[J]. Chinese journal of electronics, 2023, 51(4): 1205-1214
[10] GAZOR S, ZHANG Wei. Speech enhancement employing Laplacian-Gaussian mixture[J]. IEEE transactions on speech and audio processing, 2005, 13(5): 896-904
[11] 周明, 张涛, 刘洋. 相位敏感损失函数对语音增强感知质量的影响分析[J]. 信号处理, 2022, 38(8): 1789-1800 ZHOU Ming, ZHANG Tao, LIU Yang. Impact of phase-sensitive loss functions on perceptual quality in speech enhancement[J]. Journal of signal processing, 2022, 38(8): 1789-1800
[12] 董娴, 邵玉斌, 杜庆治, 等. 谐波结构相位估计联合幅度补偿的语音增强方法[J]. 重庆邮电大学学报(自然科学版), 2024, 36(5): 935-944 DONG Xian, SHAO Yubin, DU Qingzhi, et al. Speech enhancement method combining phase estimation of harmonic structures and amplitude compensation[J]. Journal of Chongqing University of Posts and Telecommunications (natural science edition), 2024, 36(5): 935-944
[13] 王鹏. 基于深度学习的语音增强方法研究[D]. 太原: 太原理工大学, 2024: 45-60. WANG Peng. Research on speech enhancement methods based on deep learning[D]. Taiyuan: Taiyuan University of Technology, 2024: 45-60.
[14] 罗笑雪. 时频域单通道语音增强方法研究[D]. 北京: 中国科学院声学研究所, 2023: 30-48. LUO Xiaoxue. Research on single-channel speech enhancement methods in time-frequency domain[D]. Beijing: Institute of Acoustics, Chinese Academy of Sciences, 2023: 30-48.
[15] 张天骐, 罗庆予, 张慧芝, 等. 复谱映射下融合高效Transformer的语音增强方法[J]. 信号处理, 2024, 40(2): 406-416 ZHANG Tianqi, LUO Qingyu, ZHANG Huizhi, et al. Speech enhancement method based on complex spectrum mapping with efficient transformer[J]. Journal of signal processing, 2024, 40(2): 406-416
[16] 王亚辉, 张伟, 刘强, 等. 基于理想二进制掩蔽的深度学习语音增强方法[J]. 声学学报, 2021, 46(3): 456-468 WANG Yahui, ZHANG Wei, LIU Qiang, et al. Deep learning speech enhancement method based on ideal binary mask[J]. Acta acustica, 2021, 46(3): 456-468
[17] 清华大学人工智能研究院. 轻量化Transformer的实时语音增强系统: 中国专利 CN115497128A[P]. 2023-06. AI Institute of Tsinghua University. Real-time Speech Enhancement System with Lightweight Transformer: Chinese Patent CN115497128A[P]. 2023-06.
[18] LIU Y, CHEN Z. GTCRN: A speech enhancement model requiring ultralow computational resources[J]. IEEE signal processing letters, 2024, 31: 880-884
[19] HUANG P S, KIM M, HASEGAWA-JOHNSON M, et al. Joint optimization of magnitude and phase for speech enhancement[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE, 2018: 2461-2465.
[20] HU Y X, LIU Y, LV S B, et al. DCCRN: deep complex convolution recurrent network for phase-aware speech enhancement[C]//Proceedings of the 21st Annual Conference of the International Speech Communication Association. Shanghai: ISCA, 2020: 2472-2476.
[21] SHCHEKOTOV I, ANDREEV P K, IVANOV O, et al. FFC-SE: fast Fourier convolution for speech enhancement[C]//Proceedings of the 23rd Annual Conference of the International Speech Communication Association. Incheon: ISCA, 2022: 1188-1192.
[22] LI N, WANG L B, ZHANG Q Q, et al. Dual-stream noise and speech information perception for speech enhancement[J]. Expert systems with applications, 2024, 263: 125432
[23] LI A, LIU W, LUO Z, et al. HPN: Hearing perception network for speech enhancement with memory mechanism[C]//Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 1-5.
[24] CHILD R, GRAY S, RADFORD A, et al. Generating Long Sequences with Sparse Transformers[EB/OL]. (2019-04-23)[2024-06-20]. https://arxiv.org/abs/1904.10509.
[25] LUO Y, MESGARANI N. Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation[C]//Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE, 2020: 46-50.
[26] WESTON J, CHOPRA S, BORDES A. Memory networks[C]//Proceedings of the 2015 International Conference on Learning Representations. San Diego: ICLR, 2015: 1-15.
[27] 吴迪. 复数域谐波结构重建的相位保真优化[D]. 上海: 上海交通大学, 2024: 55-70. WU Di. Phase fidelity optimization via complex-domain harmonic structure reconstruction[D]. Shanghai: Shanghai Jiao Tong University, 2024: 55-70.
[28] KENDALL A, GAL Y, CIPOLIA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7482-7491.
[29] DOLBY L. Deep learning-based speech enhancement System: US Patent 11, 876, 321B2[P]. 2025-06-15.
[30] KINOSHITA K, DELCROIX M, GANNOT S, et al. A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing research[J]. EURASIP journal on advances in signal processing, 2016, 2016(1): 7
[31] 陈立, 王浩然, 赵静. 多目标动态加权语音增强算法[J]. 计算机研究与发展, 2024, 61(3): 688-701 CHEN Li, WANG Haoran, ZHAO Jing. Multi-objective dynamic weighting algorithm for speech enhancement[J]. Journal of computer research and development, 2024, 61(3): 688-701
[32] LIANG Kaizhao, CHEN Lizhang, LIU Bo, et al. Cautious optimizers: improving training with one line of code[EB/OL]. (2024-05-01)[2024-06-20]. https://arxiv.org/abs/2411.16085.
[33] RIX A W, BEERENDS J G, HOLLIER M P, et al. Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs[C]//2001 IEEE International Conference on Acoustics, Speech, and Signal Processing Proceedings. Piscataway: IEEE, 2002: 749-752.
[34] ITU-T. Perceptual objective listening quality assessment: P. 863[S]. Geneva: ITU, 2014.
[35] ITU-T. Perceptual evaluation of speech quality (PESQ): P. 862[S]. Geneva: ITU, 2001.
[36] 世邦通信股份有限公司. 基于深度学习的语音增强方法及系统: 中国专利 CN119170029A[P]. 2024-08-19. SHIBANG Information Technology Co. , Ltd. Deep Learning-Based Speech Enhancement Method and System: Chinese Patent CN119170029A[P]. 2024-08-19.
[37] DANG Feng, CHEN Hangting, ZHANG Pengyuan. DPT-FSNet: dual-path transformer based full-band and sub-band fusion network for speech enhancement[C]//ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2022: 6857-6861.
[38] YIN D, ZHAO Z, TANG C, et al. TridentSE: Guiding speech enhancement with 32 Global Tokens[C]//Proceedings of the 24th Annual Conference of the International Speech Communication Association. Dublin: ISCA, 2023: 1-5.
[39] ZHANG L, WANG H, LIU Y, et al. DB-AIAT: dual-branch attention-in-attention Transformer for speech enhancement[C]//Proceedings of the 2024 International Conference on Artificial Intelligence and Autonomous Traffic. Singapore: IEEE, 2024: 1234-1239.
[40] ABDULATIF S, CAO Ruizhe, YANG Bin. CMGAN: conformer-based metric-GAN for monaural speech enhancement[J]. IEEE/ACM transactions on audio, speech, and language processing, 2024, 32: 2477-2493
[41] HU Y, LOIZOU P C. Subjective evaluation of speech enhancement algorithms[J]. IEEE transactions on audio, speech, and language processing, 2008, 16(5): 918-929
[42] HAO Xiang, SU Xiangdong, HORAUD R, et al. Fullsubnet: a full-band and sub-band fusion model for real-time single-channel speech enhancement[C]//ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 6633-6637.
[43] HU M, WANG D L. FRCRN: Boosting feature representation using frequency recurrence for monaural speech enhancement[C]//2022 IEEE International Conference on Acoustics, Speech and Signal Processing. Singapore: IEEE, 2022: 6912-6916.
[44] LI Y, WANG H, ZHANG P. CTSNet: A two-stage mapping mechanism for speech enhancement[C]//2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 1234-1238.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]姚霖,刘轶,李鑫鑫,等.词边界字向量的中文命名实体识别[J].智能系统学报,2016,11(1):37.[doi:10.11992/tis.201507065]
YAO Lin,LIU Yi,LI Xinxin,et al.Chinese named entity recognition via word boundarybased character embedding[J].CAAI Transactions on Intelligent Systems,2016,11():37.[doi:10.11992/tis.201507065]
[4]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[5]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[6]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[7]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[8]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[9]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[10]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[11]孙美晨,孙正,候英飒.AAR-Net：用于声学异质介质光声图像重建的深度神经网络[J].智能系统学报,2024,19(2):278.[doi:10.11992/tis.202212024]
SUN Meichen,SUN Zheng,HOU Yingsa.AAR-Net: a deep neural network for photoacoustic image reconstruction in heterogeneous acoustic media[J].CAAI Transactions on Intelligent Systems,2024,19():278.[doi:10.11992/tis.202212024]
备注/Memo
收稿日期:2025-6-18。
基金项目:国家自然科学基金项目(62173171).
作者简介:沈学利，教授，博士，中国计算机学会杰出会员，辽宁省人工智能学会副会长，辽宁工程技术大学软件学院(人工智能学院)院长，主要研究方向为智能数据处理、网络信息安全。获省部级科研成果一等奖1项、二等奖2项、三等奖4项，获省部级教学成果一等奖1项、二等奖1项，三等奖2项。发表学术论文近百篇。E-mail：shenxueli@lntu.edu.cn。;卢呈祥，硕士研究生，主要研究方向为智能数据处理、语音增强技术。E-mail：2553321250@qq.com。;金海波，副教授，博士，主要研究方向为复杂系统可靠性分析、异常检测、优化维护维修策略制定。E-mail：jinhaibo@lntu.edu.cn。
通讯作者:沈学利. E-mail：shenxueli@lntu.edu.cn
更新日期/Last Update:
1900-01-01