目的 电力设备的状态检测和故障维护是保障电力系统正常运行的重要基础。针对目前多数变电站存在电力设备缺陷类型复杂且现有的单分类缺陷检测方法无法满足电力设备的多标签分类缺陷检测需求的问题，提出一种面向电力设备缺陷检测的多模态层次化分类方法。方法 首先采集来自多个变电站的电力设备缺陷图像并进行人工标注、数据增强及归一化等预处理，构建了一个具有层次标签结构的电力设备缺陷图像数据集。然后提出一种基于多模态特征融合的层次化分类模型，采用 ResNet50 网络对图像进行特征提取，利用区域生成网络对目标进行定位以及前景、背景预测；为避免对区域生成网络生成的位置坐标进行量化时引入误差，进一步采用 ROI Align（region of interest align）方法连续操作，生成位置坐标。最后采用层次化分类，将父类别标签嵌入到当前层目标特征表示进行逐层缺陷分类，最后一层得到最终的缺陷检测结果。结果 在电力设备缺陷数据集和基准数据集上，与多标签分类电力设备缺陷检测方法和流行的常用目标检测算法进行对比实验。实验结果表明，模型对绝大部分设备缺陷类别的检测准确率最高，平均检测准确率达到 86. 4%，相比性能第 2 的模型，准确率提升了 5. 1%，并且在基准数据集上的平均检测准确率也提高了 1. 1%～3%。结论 提出的电力设备缺陷检测方法充分利用设备缺陷标签的语义信息、层次结构和设备缺陷数据的图像特征，通过多模态层次化分类模型，能够提升电力设备缺陷检测的准确率。

Objective Safety state detection of power equipment is a fundamental task to ensure the safe operation of power systems. The state detection and fault maintenance of power equipment are the basic prerequisites for ensuring the normal operation of the power system. With the growing diversities and complexity of defects in substations, the current defect recognition and power detection has increasingly been required to handle multi-label classification tasks based on a large number of closely related defect labels. However, due to the complex types of power equipment defects in most substations, most existing approaches for power equipment defect detection are inefficient at multi-label defect detection because the defect category labels often have different granularities in their semantic concepts and are often closely related with each other. All these problems cause existing defect detection methods to have difficulty meeting the requirements of multi-label classification-based defect detection tasks of power equipment. To address these problems, this paper proposes a multi-modal hierarchical classification for power equipment defect detection, which is suitable for defect detection in complex power equipment environments.Method We propose a multi-modal hierarchical classification method, which fuses the feature information of defect images, hierarchical structure information, and the semantic information of category labels. First, defect images of power equipment from multiple substations are collected and preprocessed with manual annotation, data enhancement, and normalization to construct a power equipment defect image dataset with a hierarchical label structure. Then, a hierarchical classification model based on multi-modal feature fusion and hierarchical fine-tuning techniques is proposed, which uses the ResNet50 network to extract features from images, and a region proposal network to locate object and predict the foreground and background. The region of interest align（ROI Align） method is further used to continuously generate the position coordinates to avoid introducing errors in quantifying the position coordinates generated by the region proposal network. Finally, the hierarchical structure of power equipment to be detected is used to embed the parent category labels into the current layer’s object feature representation for layer-by-layer defect classification. The final defect detection result is obtained in the final layer.Result Comparative experiments are conducted on the real-world power equipment defect dataset and the PASCAL VOC2012 benchmark dataset against the current multi-label classification-based power equipment defect detection methods and the popularly used object detection algorithms. Experimental results show that the proposed method achieved the best detection accuracy for most equipment defect categories, with a mean average precision of 86.4%. Compared with the second-best performing model, the accuracy improved by 5.1%, and the mean average precision on the benchmark dataset increased by 1.1% to 3%. The proposed method can be executed in a relevantly shorter time than the compared methods.Conclusion Our method achieves superior detection accuracy performance against the compared methods while maintaining a lower computational cost. It can improve the accuracy of power equipment defect detection through a hierarchical classification model based on multi-modal feature fusion by fully utilizing the semantic relationship between equipment defect labels.