[1]ZHAO Wenqing,XU Lijiao,CHEN Haoyang,et al.Blind image quality assessment based on multi-level feature fusion and semantic enhancement[J].CAAI Transactions on Intelligent Systems,2024,19(1):132-141.[doi:10.11992/tis.202301007]
Copy

Blind image quality assessment based on multi-level feature fusion and semantic enhancement

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 1 Page number: 132-141 Column: 学术论文—智能系统 Public date: 2024-01-05
Title:
Blind image quality assessment based on multi-level feature fusion and semantic enhancement
Author(s):
ZHAO Wenqing1; 2;  XU Lijiao1;  CHEN Haoyang1;  LI Mengwei1
1. School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, China;
2. Engineering Research Center of the Ministry of Education for Intelligent Computing of Complex Energy System Department, Baoding 071003, China
Keywords:
deep learning; image quality; convolution neural network; feature extraction; channel attention structure; multi-level feature fusion; dilated convolution; triplet loss function
CLC:
TP391.41
DOI:
10.11992/tis.202301007
Abstract:
Aiming at the low performance of the existing blind image quality assessment algorithm when facing the real distorted images, the paper proposes a new no-reference image quality assessment algorithm, namely multi-level feature fusion and semantic enhancement for NR (MFFSE-NR), which combines multi-level feature fusion and semantic information enhancement. The local and global distortion features of an image are extracted, then a feature fusion module is used to fuse the features in layers. The multi-layer dilated convolution is employed to enhance semantic information and further direct the mapping process from distorted image to quality fraction. Finally, a novel loss function called Lmix is created by combining the triplet ranking loss function and the L1 loss function, taking account of the relative ranking relationship between the predicted score and the subjective score. Validation and comparison experiments carried out on LIVEC dataset show that both the SROCC and PLCC index are improved respectively by 2.3% than the original algorithm; cross-dataset validation on the KonIQ-10k dataset and LIVEC dataset confirm that the proposed algorithm has good generalization ability when dealing with the real distorted images.
References:
[1] 方玉明, 眭相杰, 鄢杰斌, 等. 无参考图像质量评价研究进展[J]. 中国图象图形学报, 2021, 26(2): 265–286
FANG Yuming, SUI Xiangjie, YAN Jiebin, et al. Progress in no-reference image quality assessment[J]. Journal of image and graphics, 2021, 26(2): 265–286
[2] 曹玉东, 刘海燕, 贾旭, 等. 基于深度学习的图像质量评价方法综述[J]. 计算机工程与应用, 2021, 57(23): 27–36
CAO Yudong, LIU Haiyan, JIA Xu, et al. Overview of image quality assessment method based on deep learning[J]. Computer engineering and applications, 2021, 57(23): 27–36
[3] 毕晓君, 潘梦迪. 基于生成对抗网络的机载遥感图像超分辨率重建[J]. 智能系统学报, 2020, 15(1): 74–83
BI Xiaojun, PAN Mengdi. Super-resolution reconstruction of airborne remote sensing images based on the generative adversarial networks[J]. CAAI transactions on intelligent systems, 2020, 15(1): 74–83
[4] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing:a publication of the IEEE signal processing society, 2004, 13(4): 600–612.
[5] GAO Xinbo, LU Wen, TAO Dacheng, et al. Image quality assessment based on multiscale geometric analysis[J]. IEEE transactions on image processing, 2009, 18(7): 1409–1423.
[6] 王志明. 无参考图像质量评价综述[J]. 自动化学报, 2015, 41(6): 1062–1079
WANG Zhiming. Review of no-reference image quality assessment[J]. Acta automatica sinica, 2015, 41(6): 1062–1079
[7] KANG Le, YE Peng, LI Yi, et al. Convolutional neural networks for no-reference image quality assessment[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 1733?1740.
[8] XU Jiangtao, YE Peng, LI Qiaohong, et al. Blind image quality assessment based on high order statistics aggregation[J]. IEEE transactions on image processing:a publication of the IEEE signal processing society, 2016, 25(9): 4444–4457.
[9] 严浙平, 曲思瑜, 邢文. 水下图像增强方法研究综述[J]. 智能系统学报, 2022, 17(5): 860–873
YAN Zheping, QU Siyu, XING Wen. An overview of underwater image enhancement methods[J]. CAAI transactions on intelligent systems, 2022, 17(5): 860–873
[10] BOSSE S, MANIRY D, MüLLER K R, et al. Deep neural networks for no-reference and full-reference image quality assessment[J]. IEEE transactions on image processing, 2017, 27(1): 206–219.
[11] WU Jinjian, MA Jupo, LIANG Fuhu, et al. End-to-end blind image quality prediction with cascaded deep neural network[J]. IEEE transactions on image processing, 2020, 29: 7414–7426.
[12] ZHANG Weixia, MA Kede, YAN Jia, et al. Blind image quality assessment using a deep bilinear convolutional neural network[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(1): 36–47.
[13] 鄢杰斌, 方玉明, 刘学林. 图像质量评价研究综述: 从失真的角度[J]. 中国图象图形学报, 2022, 27(5): 1430–1466
YAN Jiebin, FANG Yuming, LIU Xuelin. The review of distortion-related image quality assessment[J]. Journal of image and graphics, 2022, 27(5): 1430–1466
[14] SU Shaolin, YAN Qingsen, ZHU Yu, et al. Blindly assess image quality in the wild guided by a self-adaptive hyper network[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 3664?3673.
[15] 杨春玲, 杨雅静. 基于多尺度特征逐层融合深度神经网络的无参考图像质量评价方法[J]. 华南理工大学学报(自然科学版), 2022, 50(4): 81–89,141
YANG Chunling, YANG Yajing. A deep neural network based on layer-by-layer fusion of multi-scale features for no-reference image quality assessment[J]. Journal of South China University of Technology (natural science edition), 2022, 50(4): 81–89,141
[16] YING Zhenqiang, NIU Haoran, GUPTA P, et al. From patches to pictures (PaQ-2-PiQ): mapping the perceptual space of picture quality[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 3572?3582.
[17] GHADIYARAM D, BOVIK A C. Massive online crowdsourced study of subjective and objective picture quality[J]. IEEE transactions on image processing, 2015, 25(1): 372–387.
[18] HOSU V, LIN Hanhe, SZIRANYI T, et al. KonIQ-10k: an ecologically valid database for deep learning of blind image quality assessment[J]. IEEE transactions on image processing, 2020, 29: 4041–4056.
[19] 刘玉珍, 刘美怡, 林森等. 多尺度特征融合注意力网络的水下图像增强[J]. 计算机辅助设计与图形学学报, 2023, 35(5): 685–695
LIU Yuzhen, LIU Meiyi, LIN Sen, et al. Underwater image enhancement based on multi-scale feature fusion and attention network[J]. Journal of computer-aided design & computer graphics, 2023, 35(5): 685–695
[20] GOLESTANEH S A, DADSETAN S, KITANI K M. No-reference image quality assessment via transformers, relative ranking, and self-consistency[C]//2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE, 2022: 3989?3999.
[21] GAO Shanghua, CHENG Mingming, ZHAO Kai, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(2): 652–662.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2016: 770?778.
[23] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2015: 815?823.
[24] LI Dingquan, JIANG Tingting, JIANG Ming. Exploiting high-level semantics for no-reference image quality assessment of realistic blur images[C]//Proceedings of the 25th ACM international conference on Multimedia. New York: ACM, 2017: 378?386.
[25] 高涛, 杨朝晨, 陈婷, 等. 深度多尺度融合注意力残差人脸表情识别网络[J]. 智能系统学报, 2022, 17(2): 393–401
GAO Tao, YANG Zhaochen, CHEN Ting, et al. Deep multiscale fusion attention residual network for facial expression recognition[J]. CAAI transactions on intelligent systems, 2022, 17(2): 393–401
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems