Current Issue Cover
融合注意力机制的模糊图像多尺度复原

陈紫柠1, 张宏怡1, 曾念寅2, 李寒2(1.厦门理工学院光电与通信工程学院, 厦门 361024;2.厦门大学航天航空学院, 厦门 361102)

摘 要
目的 去模糊任务通常难以进行对图像纹理细节的学习,所复原图像的细节信息不丰富,图像边缘不够清晰,并且需要耗费大量时间。本文通过对图像去模糊方法进行分析,同时结合深度学习和对抗学习的方法,提出一种新型的基于生成对抗网络(generative adversarial network, GAN)的模糊图像多尺度复原方法。方法 使用多尺度级联网络结构,采用由粗到细的策略对模糊图像进行复原,增强去模糊图像的纹理细节;同时采用改进的残差卷积结构,在不增加计算量的同时,加入并行空洞卷积模块,增加了感受野,获得更大范围的特征信息;并且加入通道注意力模块,通过对通道之间的相关性进行建模,加强有效特征权重,并抑制无效特征;在损失函数方面,结合感知损失(perceptual loss)以及最小均方差(mean squared error, MSE)损失,保证生成图像和清晰图像内容一致性。结果 通过全参考图像质量评价指标峰值信噪比(peak signal to noise ratio, PSNR)、结构相似性(structural similarity, SSIM)以及复原时间来评价算法优劣。与其他方法的对比结果表明,本文方法生成的去模糊图像PSNR指标提升至少3.8%,复原图像的边缘也更加清晰。将去模糊后的图像应用于YOLO-v4(you only look once)目标检测网络,发现去模糊后的图像可以检测到更小的物体,识别物体的数量有所增加,所识别物体的置信度也有一定的提升。结论 采用由粗到细的策略对模糊图像进行复原,在残差网络中注入通道注意力模块以及并行空洞卷积模块改进网络的性能,并进一步简化网络结构,有效提升了复原速度。同时,复原图像有着更清晰的边缘和更丰富的细节信息。
关键词
Attention mechanism embedded multi-scale restoration method for blurred image

Chen Zining1, Zhang Hongyi1, Zeng Nianyin2, Li Han2(1.School of Opto-electronic and Communication Engineering, Xiamen University of Technology, Xiamen 361024, China;2.School of Aeronautics and Astronautics, Xiamen University, Xiamen 361102, China)

Abstract
Objective Image de-blurring task aims at a qualified image derived from the low-quality blurred one. Traditional fuzzy kernels based de-blurring techniques are challenging to sort out ideal fuzzy kernel for each pixel. Current restoration methods are based on manpowered prior knowledge of the images. Simultaneously, generalization capability is constrained to be extended. To harness image de-blurring processing, convolutional neural network (CNN)model has its priority in computer vision in the context of deep learning techniques. Nevertheless, poor CNN structure adaptive issues like over-fitting are extremely constrained of parameters and topology. A challenged image de-blurring tasks is to capture detailed texture. Tiny feature constrained images restoration have their deficiencies like inadequate detail information and indistinct image edge. To facilitate image in-painting and super-resolution restoration, generative adversarial networks (GAN) has its priority to preserve texture details.an adversarial network based end-to-end framework for image de-blurring is demonstrated based on GAN-based image-to-image translation, which can speed up multifaceted image restoration process. Method First, a variety of modified residual blocks are cascaded to build a multi-scale architecture, which facilitates extracting features from coarse to fine so that more texture details in a blurred image could be well restored. Next, extensive convolution module extended the receptive fields in parallel with no computational burden. Thirdly, channel attention mechanism is also applied to strengthen weights of useful features and suppress the invalid ones simultaneously via inter-channel modeling interdependencies. Finally, network-based perceptual loss is integrated with conventional mean squared error (MSE) to serve as the total loss function in order to maintain the fidelity of the image content. Consequently, our restored images quality can be guaranteed on the aspect of qualified semantic and fine texture details both. The application of minimum MSE loss between pixels also makes the generated de-blurred image have smoother edge information. Result GoPro database is adopted for our model training and testing, including 3 214 pairs of samples among which 2 013 pairs are used as training set and the remaining 1 111 pairs serve as testing data. To enhance the generalization capability of the network, data augmentation techniques like flipping and random angle rotation are conducted. Each training sample is randomly cropped into 256×256 pixels resolution images, and pixel values of clear and blurred images are all normalized to the range of[-1, 1]. In order to comprehensively evaluate the our method, several indicators like peak signal to noise ratio (PSNR), structural similarity (SSIM) and restoration time are used for evaluation. Our experimental results have demonstrated that overall performance of proposed method is satisfactory, which can effectively eliminate the blurred region in images. Compared with some other existing works, our method increases PSNR by 3.8% in less running time, which indicates the feasibility and superiority of our proposal. Restored images obtained by proposed method have clarified edges, and our method is efficient to restore the blurry images with different sizes of blur kernels to a certain extent. It is also found that while applying the restored images to the YOLO-v4(you only look once) object detection task, the results have been significantly improved regarding the identification accuracy and the confidence coefficient both, which reflects that designed strategies in proposed method. Conclusion Our image de-blurring method aims at blurred images and extracts features sufficiently from coarse to fine. Specifically, multi-scale improved residual blocks are cascaded for learned subtle texture details. Parallel enlarged convolution and channel attention mechanism are also intervened in our model to improve their adopted residual blocks capability. Moreover, trained loss function is modified via perceptual loss introducing to traditional mean square errors. Consequently, quality of restored images can be guaranteed to some extent like sharper edges and more abundant detail information. Our analyzed results have demonstrated their qualified and efficient priorities.
Keywords

订阅号|日报