[1]WANG Yaru,YANG Chunwang,QU Zhuo,et al.Image quality assessment based on bilinear feature fusion and gate recurrent unit quality polymerization[J].CAAI Transactions on Intelligent Systems,2025,20(4):946-957.[doi:10.11992/tis.202407028]
Copy

Image quality assessment based on bilinear feature fusion and gate recurrent unit quality polymerization

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 4 Page number: 946-957 Column: 学术论文—机器学习 Public date: 2025-08-05
Title:
Image quality assessment based on bilinear feature fusion and gate recurrent unit quality polymerization
Author(s):
WANG Yaru;  YANG Chunwang;  QU Zhuo;  ZHAO Shun;  ZHANG Shiyin;  ZHAI Yongjie
Department of Automation, North China Electric Power University, Baoding 071003, China
Keywords:
deep learning; image quality; bilinear pooling; gate recurrent unit; deformable convolution; feature extraction; feature selection; features fusion
CLC:
TP391.4
DOI:
10.11992/tis.202407028
Abstract:
Current image quality assessment methods suffer from simple feature fusion strategies, insufficient extraction and utilization of quality information, and neglect of the correlation between different image regions. This paper proposes an image quality assessment method based on bilinear feature fusion and gate recurrent unit (GRU) quality aggregation. We extract global and local features of images and perform selection operations on local features based on deformable convolution. Under the guidance of semantic and contextual information, information unrelated to distortion is filtered out. A bilinear feature fusion module is constructed to enhance the interaction between global and local features, capturing changes in image quality in terms of spatial relationships and contextual information. A quality aggregation module based on GRU is constructed, combining block-wise quality prediction and global dependency modeling. This dynamically adjusts the weight proportion of each image block, ultimately aggregating the quality information of all blocks to generate a quality score for the entire image. For the CSIQ, TID2013, and PIPAL datasets across different distortion types and various scenarios, the proposed method achieved optimal Pearson linear cor-relation coefficient (PLCC) and Spearman rank-order correlation coefficient (SROCC) metrics. Notably, on the PIPAL dataset, the PLCC improved by 3.9% and the SROCC improved by 3.1% compared with the second-best method.
References:
[1] 方玉明, 眭相杰, 鄢杰斌, 等. 无参考图像质量评价研究进展[J]. 中国图象图形学报, 2021, 26(2): 265-286.
FANG Yuming, SUI Xiangjie, YAN Jiebin, et al. Progress in no-reference image quality assessment[J]. Journal of image and graphics, 2021, 26(2): 265-286.
[2] 曹玉东, 刘海燕, 贾旭, 等. 基于深度学习的图像质量评价方法综述[J]. 计算机工程与应用, 2021, 57(23): 27-36.
CAO Yudong, LIU Haiyan, JIA Xu, et al. Overview of image quality assessment method based on deep learning[J]. Computer engineering and applications, 2021, 57(23): 27-36.
[3] HU Runze, LIU Yutao, GU Ke, et al. Toward a No-reference quality metric for camera-captured images[J]. IEEE transactions on cybernetics, 2023, 53(6): 3651-3664.
[4] 秦小倩, 杜浩. 基于自然场景统计的图像质量评价算法[J]. 现代电子技术, 2023, 46(23): 36-42.
QIN Xiaoqian, DU Hao. Image quality assessment algorithm based on natural scene statistics[J]. Modern electronics technique, 2023, 46(23): 36-42.
[5] 李沛钊, 王同罕, 贾惠珍, 等. USformer-Net: 基于U-Net和Swin Transformer的脑部MRI图像质量评价方法[J]. 现代电子技术, 2024, 47(7): 1-7.
LI Peizhao, WANG Tonghan, JIA Huizhen, et al. USformer-Net: brain MRI image quality assessment fusing U-Net and Swin Transformer[J]. Modern electronics technique, 2024, 47(7): 1-7.
[6] 江本赤, 卞仕磊, 史晨阳, 等. 基于色貌尺度相位一致性的全参考图像质量评价[J]. 光学精密工程, 2023, 31(10): 1509-1521.
JIANG Benchi, BIAN Shilei, SHI Chenyang, et al. Full reference image quality assessment based on color appearance-based phase consistency[J]. Optics and precision engineering, 2023, 31(10): 1509-1521.
[7] 赵文清, 许丽娇, 陈昊阳, 等. 多层特征融合与语义增强的盲图像质量评价[J]. 智能系统学报, 2024, 19(1): 132-141.
ZHAO Wenqing, XU Lijiao, CHEN Haoyang, et al. Blind image quality assessment based on multi-level feature fusion and semantic enhancement[J]. CAAI transactions on intelligent systems, 2024, 19(1): 132-141.
[8] 王伟, 刘辉, 杨俊安. 一种特征字典映射的图像盲评价方法研究[J]. 智能系统学报, 2018, 13(6): 989-993.
WANG Wei, LIU Hui, YANG Jun’an. Blind quality evaluation with image features codebook mapping[J]. CAAI transactions on intelligent systems, 2018, 13(6): 989-993.
[9] 王成, 刘坤, 杜砾. 全参考图像质量指标评价分析[J]. 现代电子技术, 2023, 46(21): 39-43.
WANG Cheng, LIU Kun, DU Li. Evaluation and analysis of full reference image quality indicators[J]. Modern electronics technique, 2023, 46(21): 39-43.
[10] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing, 2004, 13(4): 600-612.
[11] SAMPAT M P, WANG Zhou, GUPTA S, et al. Complex wavelet structural similarity: a new image similarity index[J]. IEEE transactions on image processing, 2009, 18(11): 2385-2401.
[12] WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. Pacific Grove: IEEE, 2003: 1398-1402.
[13] XUE Wufeng, ZHANG Lei, MOU Xuanqin, et al. Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[J]. IEEE transactions on image processing, 2014, 23(2): 684-695.
[14] ZHANG Lin, SHEN Ying, LI Hongyu. VSI: a visual saliency-induced index for perceptual image quality assessment[J]. IEEE transactions on image processing, 2014, 23(10): 4270-4281.
[15] KIM J, LEE S. Deep learning of human visual sensitivity in image quality assessment framework[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1969-1977.
[16] PRASHNANI E, CAI Hong, MOSTOFI Y, et al. PieAPP: perceptual image-error assessment through pairwise preference[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1808-1817.
[17] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595.
[18] DING Keyan, MA Kede, WANG Shiqi, et al. Image quality assessment: unifying structure and texture similarity[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(5): 2567-2581.
[19] SEO S, KI S, KIM M. A novel just-noticeable-difference-based saliency-channel attention residual network for full-reference image quality predictions[J]. IEEE transactions on circuits and systems for video technology, 2021, 31(7): 2602-2616.
[20] GAO Fei, WANG Yi, LI Panpeng, et al. DeepSim: Deep similarity for image quality assessment[J]. Neurocomputing, 2017, 257: 104-114.
[21] WU Jinjian, MA Jupo, LIANG Fuhu, et al. End-to-end blind image quality prediction with cascaded deep neural network[J]. IEEE transactions on image processing, 2020, 29: 7414-7426.
[22] BOSSE S, MANIRY D, MüLLER K R, et al. Deep neural networks for No-reference and full-reference image quality assessment[J]. IEEE transactions on image processing, 2018, 27(1): 206-219.
[23] CHEON M, YOON S J, KANG B, et al. Perceptual image quality assessment with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2021: 433-442.
[24] VARGA D. No-reference image quality assessment using the statistics of global and local image features[J]. Electronics, 2023, 12(7): 1615.
[25] VARGA D. No-reference quality assessment of authentically distorted images based on local and global features[J]. Journal of imaging, 2022, 8(6): 173.
[26] LAO Shanshan, GONG Yuan, SHI Shuwei, et al. Attentions help CNNs see better: attention-based hybrid image quality assessment network[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans: IEEE, 2022: 1139-1148.
[27] MA Kede, LIU Wentao, ZHANG Kai, et al. End-to-end blind image quality assessment using deep neural networks[J]. IEEE transactions on image processing, 2018, 27(3): 1202-1213.
[28] YUAN Li, CHEN Yunpeng, WANG Tao, et al. Tokens-to-token ViT: training vision transformers from scratch on ImageNet[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 538-547.
[29] 毛明毅, 吴晨, 钟义信, 等. 加入自注意力机制的BERT命名实体识别模型[J]. 智能系统学报, 2020, 15(4): 772-779.
MAO Mingyi, WU Chen, ZHONG Yixin, et al. BERT named entity recognition model with self-attention mechanism[J]. CAAI transactions on intelligent systems, 2020, 15(4): 772-779.
[30] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×6 words: Transformers for image recognition at scale[EB/OL]. (2020-10-22)[2024-07-24].https://arxiv.org/abs/2010.11929.
[31] WANG Hao, ZHANG Yue, LIU Chao, et al. sEMG based hand gesture recognition with deformable convolutional network[J]. International journal of machine learning and cybernetics, 2022, 13(6): 1729-1738.
[32] SHI Shuwei, BAI Qingyan, CAO Mingdeng, et al. Region-adaptive deformable network for image quality assessment[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2021: 324-333.
[33] ZHANG Haochen, LIU Dong, XIONG Zhiwei. Convolutional neural network-based video super-resolution for action recognition[C]//2018 13th IEEE International Conference on Automatic Face & Gesture Recognition. Xi’an: IEEE, 2018: 746-750.
[34] ZHANG Weixia, MA Kede, YAN Jia, et al. Blind image quality assessment using a deep bilinear convolutional neural network[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(1): 36-47.
[35] 刘扬, 王立虎, 杨礼波, 等. 改进EEMD-GRU混合模型在径流预报中的应用[J]. 智能系统学报, 2022, 17(3): 480-487.
LIU Yang, WANG Lihu, YANG Libo, et al. Application of improved EMD-GRU hybrid model in runoff forecasting[J]. CAAI transactions on intelligent systems, 2022, 17(3): 480-487.
[36] GU Jinjin, CAI Haoming, CHEN Haoyu, et al. PIPAL: a large-scale image quality assessment dataset for perceptual image restoration[M]//Computer Vision-ECCV 2020. Cham: Springer International Publishing, 2020: 633-651.
[37] LAPARRA V, BALLé J, BERARDINO A, et al. Perceptual image quality assessment using a normalized Laplacian pyramid[J]. Electronic imaging, 2016, 28(16): 1-6.
[38] CHANDLER D M. Most apparent distortion: full-reference image quality assessment and the role of strategy[J]. Journal of electronic imaging, 2010, 19(1): 011006.
[39] SHEIKH H R, BOVIK A C. Image information and visual quality[J]. IEEE transactions on image processing, 2006, 15(2): 430-444.
[40] ZHANG Lin, ZHANG Lei, MOU Xuanqin, et al. FSIM: a feature similarity index for image quality assessment[J]. IEEE transactions on image processing, 2011, 20(8): 2378-2386.
[41] CHEN Chaofeng, MO Jiadi, HOU Jingwen, et al. TOPIQ: a top-down approach from semantics to distortions for image quality assessment[J]. IEEE transactions on image processing, 2024, 33: 2404-2418.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems