基于视觉的非完全标注表面缺陷检测综述
叶标华, 康丹青, 谢晓华, 赖剑煌(中山大学计算机学院) 摘 要
在现代制造业中,基于机器视觉的表面缺陷检测是保证产品质量的关键,在工业智能化发展中发挥着重要作用。然而,获取缺陷数据的标注需要花费大量人力和时间成本。随着深度学习、 大数据和传感器等技术的发展,如何在非完全标注的情况下实现准确、快速和鲁棒的缺陷识别成为当前的研究热点。该文对非完全标注场景下的表面缺陷检测技术的研究进展进行了全面的梳理回顾。首先简要介绍了缺陷检测领域的研究背景、基础概念的定义、常用数据集和相关技术。在此基础上,从标签策略以及任务策略两个角度详细介绍了多种非完全标注场景下的缺陷检测技术。在标签策略中,本文比较了基于无监督、半监督、弱监督学习下的不同缺陷检测算法的研究思路。在任务策略中,本文总结了领域自适应、小样本以及大模型的表面缺陷检测算法的最新进展。接着,本文在多个数据集上横向对比了不同标签策略以及任务策略中前沿算法的性能。最后,对该任务中的弱小目标检测、伪标签质量评估以及大模型的知识迁移等问题进行总结和展望。总体而言,非完全标注的表面缺陷检测是一个充满挑战且技术性极强的问题。同时,如何进一步推动表面缺陷检测技术进一步利用非完全标注的数据,并切实在工业制造场景中落地应用还需要更深入的研究。
关键词
Review of Surface Defect Inspection with Incomplete Annotations
Ye Biaohua, Kang Danqing, Xie Xiaohua, Lai Jianhuang(School of Computer Science and Engineering) Abstract
Surface defect inspection is an industrial automation technology primarily used to identify and evaluate defects and anomalies in products or materials. Its purpose is to ensure product quality, reduce production costs, and improve production efficiency by obtaining relevant information such as the coordinates, categories, sizes, and contours of defects. In industrial contexts, defects are generally understood as the presence of missing parts, flaws, damages, errors, and anomalies compared to normal samples. Depending on the granularity of dataset annotations, defect information annotation can be categorized into three types: image-level annotation, instance-level annotation, and pixel-level annotation, corresponding to three computer vision tasks: classification, detection, and segmentation. In recent years, due to the rapid development of technologies such as machine vision, big data, and sensors, replacing human labor with machines has become possible. Using machine vision technology to replace manual inspection for surface defect inspection in industrial products has increasingly become a trend and mainstream practice. However, in industrial settings, detailed annotation of defect data requires significant time and effort, and there is an abundance of historical unannotated data in these settings. The research problem of surface defect inspection with incomplete annotations investigates how to utilize the rich unannotated data or leverage a small amount of annotated data to improve the effectiveness of surface defect inspection. Automated visual inspection systems are a crucial step in industrial manufacturing processes, replacing manual quality inspection and enabling smart manufacturing. Researchers have conducted studies on surface defect inspection technologies based on deep learning, involving the latest methods, applications, and challenges. However, there is still a lack of literature reviews on surface defect inspection methods with incomplete annotations in the field of industrial product surface defect inspection. This paper systematically summarizes the research background, definitions of basic concepts, commonly used datasets, and related technologies in the field of surface defect inspection with incomplete annotations. According to different industrial contexts, we have collected and organized commonly used datasets in fields such as metal surfaces, fabrics, building materials, and 3C electronic products. The paper meticulously categorizes various defect inspection techniques from the perspectives of label strategies and task strategies, pointing out the characteristics, advantages, and disadvantages of each method. Based on different label types, surface defect inspection tasks with incomplete annotations can be divided into unsupervised learning, semi-supervised learning, and weakly supervised learning methods. Unsupervised learning methods mainly include strategies such as clustering, positive sample modeling, and template matching. Semi-supervised learning methods primarily include techniques such as image reconstruction, pseudo-label generation, and data augmentation. Weakly supervised learning methods mainly involve the use of class activation maps and similar techniques. Based on different task strategies, surface defect inspection tasks with incomplete annotations can be categorized into domain adaptation-based methods, few-shot learning-based methods, and large model-based methods. Domain adaptation methods mainly include approaches such as distribution alignment, learning domain-invariant features, and dynamically adjusting hyperparameters. Few-shot learning methods primarily include techniques such as meta-learning, metric learning, and graph neural networks. Large model-based methods mainly involve the use of large models like SAM, CLIP, LLM, and VLM. Subsequently, this paper compares the performance of cutting-edge unsupervised, semi-supervised, and weakly supervised algorithms on multiple datasets. Finally, the paper discusses and forecasts future research trends in surface defect inspection with incomplete annotations, including the detection of weak small targets, efficient utilization of unannotated defect images, and the application of language models and vision-language models in defect inspection. Surface defect inspection tasks with incomplete annotations are commonly encountered in the industry, holding significant research and application value, deserving more attention and promotion from both the industrial and academic communities.
Keywords
surface defect inspection incomplete annotations unsupervised learning weak-supervised learning semi-supervised learning domain adaptation few-shot
|