Current Issue Cover
图像重定向质量评价的研究进展

胡波1,2, 谢国庆1,2, 李雷达3, 李静4, 杨嘉琛5, 路文6, 高新波1,2(1.重庆邮电大学图像认知重庆市重点实验室, 重庆 400065;2.广阳湾实验室重庆脑与智能科学中心, 重庆 400064;3.西安电子科技大学人工智能学院, 西安 710071;4.阿里巴巴集团, 北京 100020;5.天津大学电气自动化与信息工程学院, 天津 300072;6.西安电子科技大学电子工程学院, 西安 710071)

摘 要
随着移动显示设备的普及,人们可以使用不同的终端设备来获取图像信息。为了适配不同尺寸的设备,图像重定向技术应运而生,并成为当前研究的热点。尽管该技术在近年来取得了长足进步,但没有算法能够保证在不降低视觉内容质量的前提下满足多种显示设备的要求。换句话说,在图像重定向的过程中,不可避免地会引入失真,降低用户的视觉体验。因此,如何客观、准确地评价重定向图像质量对图像重定向算法的选择、优化及发展至关重要。当前,研究人员已针对图像重定向质量评价开展了一系列研究,并取得了一定的研究成果。但是缺乏对图像重定向质量评价的综述,因此本文对当前图像重定向质量评价的研究进展进行了较为全面的回顾和总结。首先简单介绍了图像重定向以及传统图像质量评价;然后分别介绍图像重定向质量评价的数据集和客观质量评价方法,将现有的客观质量评价分为基于传统特征相似性的方法和基于图像配准的方法两类,并分析了两类方法的优缺点;接着在3个数据集上对具有代表性的算法的性能进行了对比分析;最后,总结了当前图像重定向质量评价领域面临的问题和挑战,并指明了未来可能的发展方向。
关键词
Progress of image retargeting quality evaluation:a survey

Hu Bo1,2, Xie Guoqing1,2, Li Leida3, Li Jing4, Yang Jiachen5, Lu Wen6, Gao Xinbo1,2(1.Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;2.Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China;3.School of Artificial Intelligence, Xidian University, Xi'an 710071, China;4.Alibaba Group, Beijing 100020, China;5.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China;6.School of Electronic Engineering, Xidian University, Xi'an 710071, China)

Abstract
With the popularization of the Internet,intelligent technology,and various sensing devices,images and videos are playing an increasingly important role in video surveillance,health care,and distance education. All kinds of information can be obtained through images or videos using different terminal devices(such as mobile phones,laptops,and tablets)anytime and anywhere. Because different terminal devices have dissimilarsizes and resolutions,how to display the same image with high visual quality on these devices has become a common concern of the academic and industrial community. To solve this problem,image retargeting technology has emerged and become a popular,cutting-edge research direction in the field of computer vision and image processing. Image retargeting technology aims to adjust the image resolution without destroying the visual content to adapt to the information acquisition of various terminals. Traditional image retargeting algorithms achieve this goal through some simple operations(such as scaling and clipping),but such operations usually cause serious distortions of visual content and the loss of important information;thus,obtaining a visual satisfying retargeted image is difficult. To compensate for the performance disadvantages of traditional algorithms,a series of more advanced content-aware image retargeting algorithms have been proposed in recent years. Generally,these algorithms adopt a two-stage framework. The first step is to calculate the importance map of the input image and assign an importance weight to each pixel. The higher the weight is,the higher the probability that it should be retained. The second step is to implement the corresponding image resizing method to preserve the important content of the image as much as possible to meet the geometric constraints. According to different technologies,content-based image retargeting algorithms can be divided into four categories:discrete methods,continuous methods,multioperator methods,and deep learning methods. Although great progress has been made in this field,no algorithm can be guaranteed to meet the requirements of multiple display devices without reducing the visual quality. Distortions are inevitably introduced in image retargeting. Therefore, how to evaluate the quality of retargeted images objectively and accurately is very important for the selection,optimization, and development of image retargeting algorithms. Image quality assessment(IQA)is a basic problem in image processing and computer vision. In general,it can be divided into subjective assessment and objective assessment. Subjective assessment,which is the most direct,effective way,is completed through the judgment of subjects. Objective assessment is to predict and evaluate image quality automatically by constructing models. Compared with subjective assessment,objective assessment has the advantages of low cost,reusability,and easy deployment,so it is the focus of the existing research. According to whether reference image is used or not,it can be further divided into full-reference quality metrics,reducedreference quality metrics,and no-reference quality metrics. According to different problems,it can also be divided into objective assessment of natural images,objective assessment of screen content images,objective assessment of cartoon images,and objective assessment of image retargeting. In recent years,a massive effort has been made in objective image retargeting quality assessment(IRQA);thus,encouraging research progress has been achieved. However,up to now,no review paper has studied image retargeting quality evaluation,which could hinder further development of this field. To this end,this review paper analyzes the challenges facing the field,reviews and summarizes the existing methods,examines their advantages and disadvantages,and states the possible development directions of the field to help researchers quickly understand and master the basic situation and promote the development of this field. First,the related work,namely, image retargeting and traditional IQA,is briefly introduced according to the classification principle. Specifically,image retargeting is divided into traditional algorithms and content-aware algorithms,and traditional IQA is described and summarized according to full-reference metric,reduced-reference metric,and no-reference metric. Second,the dataset and objective evaluation of image retargeting are introduced emphatically. For the datasets,subjective quality assessment is used as the performance upper bound of objective quality assessment and provides performance comparison for objective quality algorithms. From these datasets,the distortion characteristics of image retargeting are analyzed,and the important steps in the construction of different databases are compared and summarized. Two types of representative IRQA models,namely, the traditional feature similarity-based IRQA model and the image registration-based IRQA model,are introduced and summarized in detail. Traditional feature similarity-based IRQA models extract perception-sensitive features from reference images and retargeted images,respectively and quantify the distortion degree of retargeted images by calculating their feature similarity. To cater to the characteristics of the human visual system,saliency maps are often incorporated into the design of these models. Before feature extraction,image registration-based IRQA models first conduct sparse or dense image registration of two images to extract features effectively and improve the outcome of the algorithms. Next,their main ideas,advantages,and disadvantages are thoroughly analyzed and compared. Third,the performances of representative IRQA metrics are compared and analyzed on three public datasets in terms of prediction accuracy and monotonicity. Experimental results show the current models are only partially consistent with human perception and can still be improved. More energy and efforts are needed in this field. Finally,the current problems and challenges in the field of IRQA are summarized,and the possible development directions in the future are identified.
Keywords

订阅号|日报