Current Issue Cover
  • 发布时间: 2024-09-04
  • 摘要点击次数:  18
  • 全文下载次数: 12
  • DOI:
  •  | Volume  | Number
基于门控注意力聚合网络的多源遥感图像分类方法(ChinaMM2024会议推荐论文)

金学鹏, 高峰, 石晓晨, 董军宇(中国海洋大学)

摘 要
目的 近年来,为了突破单一传感器的技术限制并弥补单一数据源应用的局限性,多源遥感数据融合成为了遥感应用领域的研究热点。当前的高光谱图像与LiDAR/SAR数据融合分类方法未能充分利用高光谱图像的光谱特征以及LiDAR/SAR数据的地物结构信息。由于不同成像模态的图像在数据特性上存在本质差异,这些差异为多源图像特征的关联带来了重大挑战。尽管一些基于深度学习的方法在高光谱和LiDAR/SAR数据融合分类任务上取得了较好的效果,但它们在融合过程中未能充分利用多源数据中的纹理信息和几何信息。方法 为了应对这一关键问题,本文提出了一种基于门控注意力聚合网络的多源遥感图像分类方法,可以更加全面地挖掘多源数据中的互补信息。首先,设计了一个门控跨模态聚合模块,利用交叉注意力特征融合将LiDAR/SAR数据中的地物精细结构信息与高光谱图像特征有机融合。然后,使用精细化的门控模块将有价值的LiDAR/SAR特征集成到高光谱图像特征中,从而增强多源数据的融合效果。结果 实验在Houston2013和Augsburg数据集上与主流的7种方法进行比较,在总体精度(Overall Accuracy,OA)、平均精度(Average Accuracy,AA)和卡帕系数(Kappa Coefficient,Kappa)指标上都取得了最优表现。特别是在Augsburg数据集中,本文方法在大多数类别上均取得了最佳指标。在分类的可视化结果中可以明显看出,本文所提出的分类模型在性能上具有显著优势。结论 在Houston2013和Augsburg数据集上的实验结果表明,本文所提出的GCA-Net具有优异的性能,显著优于HCT、MACN等当前主流方法。该方法能够根据不同模态的特点充分融合不同模态的信息进行分类,为多源遥感数据的融合分类提供了理论支持。
关键词
Gated cross-modal aggregation network for multi-source remote sensing data classification

Jin Xuepeng, Gao Feng, Shi Xiaochen, Dong Junyu(Ocean University of China)

Abstract
Objective In recent years, multi-source remote sensing data fusion has emerged as a research hotspot in the field of remote sensing applications. This trend aims to overcome the technical limitations of single sensors and address the constraints associated with relying on a single data source. Traditional remote sensing methods, which often depend on a single type of sensor, face significant challenges in providing comprehensive and accurate information due to the inherent limitations of the sensors. For instance, hyperspectral sensors capture detailed spectral information but may lack spatial resolution, while LiDAR and SAR sensors excel in capturing structural information but may not provide sufficient spectral details. The integration of hyperspectral images and LiDAR/SAR data holds great promise for enhancing remote sensing applications. However, current methods for fusion classification have not fully utilized the rich spectral features of hyperspectral images and the structural information of ground objects provided by LiDAR/SAR data. These two types of data have fundamentally different characteristics, which pose significant challenges for effective feature correlation. Hyperspectral images contain abundant spectral information that can identify different materials, while LiDAR provides 3D structural information, and SAR offers high-resolution imaging under various weather conditions. The differences in data characteristics among these imaging modalities create substantial challenges in correlating multi-source image features. Although some deep learning-based methods have shown promising results in the fusion classification tasks of hyperspectral and LiDAR/SAR data, they often fall short in fully exploiting the texture and geometric information embedded within the multi-source data during the fusion process. These methods may perform well in specific scenarios but often lack the robustness and versatility needed for broader applications. Consequently, there is a pressing need for more sophisticated approaches that can leverage the complementary strengths of different data sources to improve classification accuracy and reliability. Method To address this critical issue, this paper proposes a novel multi-source remote sensing image classification method based on a gated attention aggregation network (GCA-Net). This approach aims to more comprehensively exploit the complementary information available in multi-source data. The core innovation of the proposed method lies in its ability to effectively integrate the unique advantages of different sensor data types through a series of advanced neural network modules. Firstly, a gated cross-modal aggregation module is designed to facilitate the organic integration of detailed structural information from LiDAR/SAR data with the spectral features of hyperspectral images. This module leverages cross-attention mechanisms to enhance the feature fusion process, ensuring that the distinctive information from each data source is effectively utilized. The cross-attention feature fusion mechanism allows the model to focus on the most relevant features from each modality, enhancing the overall representation of the fused data. Secondly, a refined gating module is employed to integrate valuable LiDAR/SAR features into the hyperspectral image features. This gating mechanism selectively incorporates the most informative features, thereby enhancing the fusion effect of multi-source data. The gating module acts as a filter that prioritizes the integration of features that contribute the most to the classification task, ensuring that the fused data retains the critical information from both sources. This approach not only improves the accuracy of the classification but also enhances the robustness of the model across different datasets and scenarios. Results The proposed method was rigorously tested through experiments conducted on two widely recognized datasets: Houston2013 and Augsburg. These datasets provide a diverse range of scenes and conditions, making them ideal for evaluating the effectiveness of the proposed method. The performance of the GCA-Net was compared with seven mainstream methods to assess its efficacy. The experimental results demonstrated that the proposed method achieved the best performance in terms of Overall Accuracy (OA), Average Accuracy (AA), and Kappa Coefficient (Kappa). Specifically, the GCA-Net outperformed the other methods by a significant margin, highlighting its superior ability to fuse and classify multi-source remote sensing data. On the Augsburg dataset, in particular, the proposed method achieved the best indicators across most categories. This dataset includes a variety of urban and rural scenes, challenging the classification models with diverse textures and structures. The GCA-Net's ability to handle such diversity showcases its robustness and versatility. Moreover, the classification visualization results clearly show that the proposed classification model has a significant performance advantage. The visualized results provide a clear and intuitive representation of the classification accuracy, with the GCA-Net producing more precise and consistent classifications compared to other methods. This visual evidence underscores the practical benefits of the proposed method, demonstrating its potential for real-world applications in remote sensing. Conclusion The experimental results on the Houston2013 and Augsburg datasets provide strong evidence that the proposed GCA-Net has excellent performance, significantly surpassing current mainstream methods such as HCT and MACN. The GCA-Net's ability to fully integrate information from different modalities based on their unique characteristics is a key factor in its success. This integration allows the model to leverage the strengths of each data source, resulting in a more accurate and reliable classification. The proposed method's superior performance can be attributed to its innovative use of gated attention mechanisms and cross-modal feature fusion. These techniques enable the model to selectively focus on the most relevant features from each modality, enhancing the overall quality of the fused data. By effectively combining the spectral richness of hyperspectral images with the structural details of LiDAR/SAR data, the GCA-Net sets a new benchmark for multi-source remote sensing data fusion. In conclusion, the GCA-Net provides a robust theoretical foundation for the fusion classification of multi-source remote sensing data. Its ability to address the limitations of single-sensor approaches and leverage the complementary strengths of different data types makes it a valuable tool for advancing remote sensing applications. The proposed method not only improves classification accuracy but also enhances the practical applicability of remote sensing technologies, paving the way for more sophisticated and effective solutions in various domains such as environmental monitoring, urban planning, and disaster management.
Keywords

订阅号|日报