Current Issue Cover
结合浓度划分与图像融合的多分支非均质图像去雾

金鑫乐, 刘春晓, 叶爽爽, 王成骅, 周子翔(浙江工商大学)

摘 要
目的 目前的去雾算法已能够较好地处理均质的薄雾图像,但针对雾霾浓度不同的非均质雾霾图像往往具有较低的去雾性能。本文将单幅非均质雾霾图像视为由多个具有均质薄雾或者均质浓雾的局部区域组成,通过分别解决单幅非均质雾图中的不同均质雾霾区域来进行整幅非均质图像去雾,为此提出了一种基于浓度划分与图像融合的多分支非均质图像去雾算法。方法 本文首先在不同均质雾霾浓度的去雾数据集上训练了多个图像增强网络,以得到针对不同均质雾霾浓度的图像增强模型,它们对于相应雾霾浓度的图像区域具有较好的增强性能。由于单个图像增强模型只能较好地增强一张非均质雾霾图像中具有对应雾霾浓度的图像区域,但对其他不同雾霾浓度的图像区域可能存在去雾力度不足或者过度增强的现象,因此本文又设计了一个图像融合网络,将多个初始图像增强结果中的优势区域进行融合,得到最终的图像去雾结果。结果 大量的实验测试结果显示:在合成雾霾数据集FiveK-Haze上,本文算法与排名第二的SCAN方法相比在PSNR和SSIM有参考指标上分别提高了5.2886dB和0.1138。在真实雾霾数据集Real-World上,本文算法与排名第二的DEAN方法相比在FADE和HazDes无参考指标上分别降低了0.0793和0.0512。在室内合成测试数据集SOTS-indoor上,本文算法的PSNR和SSIM指标比排名第2的DeFormer方法分别提高了2.5182dB和0.0123。在室外合成测试数据集SOTS-outdoor上,本文算法在PSNR指标上比排名第2的SGID-PFF方法提高了2.832dB,在SSIM指标上比排名第2的DeFormer方法提高了0.0238。结论 与已有的单幅图像去雾方法相比,本文提出的算法能够有效增强非均质雾霾图像,具有更高的鲁棒性,展现了较好的性能指标。
关键词
Multi-branch non-homogeneous image dehazing based on concentration partitioning and image fusion

JIN Xinle, LIU CHUN Xiao, YE Shuangshuang, WANG Chenghua, ZHOU Zixiang(Zhejiang Gongshang University)

Abstract
Objective When using a camera to capture images, the captured images may be affected by atmospheric floating particles such as smoke, dust, and fog, resulting in a decrease in image quality. Such images not only easily cause human visual misjudgment but also hinder the development of visual tasks, such as remote sensing monitoring and autonomous driving. Current dehazing methods are able to handle homogeneous thin hazy images well, but they often perform poorly on the non-homogeneous hazy images. Therefore, we propose a multi-branch non-homogeneous image dehazing method combined with concentration partitioning and image fusion. We consider a single non-homogeneous hazy image as a combination of multiple local regions with homogeneous thin or dense haze. By separately addressing different homogeneous hazy regions in a single non-homogeneous hazy image, the entire non-homogeneous image is dehazed. Method We the design Concentration Partitioning and Image Fusion based Multi-branch Image Dehazing Neural Network (CPIFNet), a two-stage network framework for image enhancement and image fusion. Through experiments, we found that training models based on homogeneous haze image datasets with different haze concentrations can result in image enhancement models with varying enhancement intensities. In order to get varying enhancement models, we need homogeneous hazed image datasets with different haze concentrations. FiveK-Haze is a synthesized dehazing dataset based on the atmospheric physical scattering model, containing nine kind of different homogeneous hazed images with varying haze concentrations. We re-partitioned the hazy images in the FiveK-Haze dataset based on haze concentration, dividing the dataset into 1 to 5 different haze concentration levels, and excluded the hazy image samples with excessive haze concentration. Then, we train image enhancement network on those new homogeneous dehazing datasets, and then we can obtain image enhancement models for different haze concentrations. In the image enhancement networks, the deep image features of hazy images are continuously extracted to obtain the stretching coefficient of the image enhancement model. This stretching coefficient is multiplied with the hazy image to produce the image enhancement result. To avoid losing feature information as the network deepens, the image enhancement network replaces network layers with residual modules to extract deep feature information. To accelerate the convergence speed of network training and avoid the transmission of negative values in the feature layers, we used the ReLU activation function after each convolutional layer. Each enhancement network performs well for the corresponding haze concentration image region. However, due to the fact that a single enhancement network can only effectively enhance image regions with corresponding haze concentrations, there may be insufficient or excessive dehazing in other regions. Therefore, we design an image fusion network to combine the advantageous regions in the multiple initial enhancement results, producing the final dehazed result. In the image fusion network, deep image features of different image enhancement results are continuously extracted, and the dehazed result is obtained by stacking and merging these deep image features. In addition to reconstruction loss, perceptual loss , and structural loss, the image enhancement network and image fusion network also utilize color loss to constrain the image dehazing results of the network modules. This is because the severe loss of pixel information in dense hazy images, color restoration is extremely difficult and color loss function can guide the color of the image dehazing result closer to the reference image. Result Theoretically, a dehazing dataset with more finely divided haze concentration levels could enable network models to learn more information of hazed images. However, experiments reveal that this is not the case. We find that when the number of haze datasets is 3, which means the number of image enhancement models is 3, CPIFNet achieves the best dehazing performance. Large-scale experiments are carried out in comparison with more than ten latest image dehazing algorithms, and our method achieves the best in both performance indicators and dehazing effects. Compared with the second-ranked SCAN method, our method improves the reference indicators as PSNR and SSIM by 5.2886dB and 0.1138 respectively over the synthetic hazy image dataset FiveK-Haze. Compared with the second-ranked DEAN method, our method reduces the non-reference metrics as FADE and HazDes by 0.0793 and 0.0512 respectively over the real-world hazy image dataset. Meanwhile, in order to get more comparative experiments and indicator evaluations, we conduct more tests on some publicly available datasets. On the SOTS-indoor dataset, our method improves PSNR and SSIM respectively by 2.5182dB and 0.0123 compared to the second-ranked DeFormer method. On the SOTS-outdoor dataset, our method improves PSNR by 2.832dB compared to the second-ranked SGID-PFF method, and improves SSIM by 0.0238 compared to the second-ranked DeFormer method. Conclusion We design a two-stage, multi-branch deep neural network to remove haze from a single non-homogeneous hazy image by separately addressing different homogeneous hazy regions. Compared to existing methods, our method can enhance the structural contrast of dense hazy regions and slightly enhance thin hazy regions while restoring color information.
Keywords

订阅号|日报