Current Issue Cover
用户多维感知的3D图像体验质量评价

董天阳, 杨丽锦, 张鑫鹏(浙江工业大学计算机科学与技术学院, 杭州 310023)

摘 要
目的 符合用户视觉特性的3维图像体验质量评价方法有助于准确、客观地体现用户观看3D图像或视频时的视觉感知体验,从而给优化3维内容提供一定的思路。现有的评价方法仅从图像失真、深度感知和视觉舒适度中的一个维度或两个维度出发对立体图像进行评价,评价结果的准确性有待进一步提升。为了更加全面和准确地评价3D图像的视觉感知体验,提出了一种用户多维感知的3D图像体验质量评价算法。方法 首先对左右图像的差异图像和融合图像提取自然场景统计参数表示失真特征;然后对深度图像提取敏感区域,对敏感区域绘制失真前后深度变换直方图,统计深度变化情况以及利用尺度不变特征变换(SIFT)关键点匹配算法计算匹配点数目,两者共同表示深度感知特征;接下来对视觉显著区域提取视差均值、幅值表示舒适度特征;最后综合考虑图像失真、深度感知和视觉舒适度3个维度特征,将3个维度特征归一化后联合成体验质量特征向量,采用支持向量回归(SVR)训练评价模型,并得到最终的体验质量得分。结果 在LIVE和Waterloo IVC数据库上的实验结果表明,所提出的方法与人们的主观感知的相关性达到了0.942和0.858。结论 该方法充分利用了立体图像的特性,评价结果优于比较的几种经典算法,所构建模型的评价结果与用户的主观体验有更好的一致性。
关键词
3D image experience quality evaluation method for users' multi-dimensional perception

Dong Tianyang, Yang Lijin, Zhang Xinpeng(College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China)

Abstract
Objective Although 3D technology is increasingly used in film and television, the development of 3D display technology has been stagnant in recent years. The main reason is that image degradation occurs in the process of 3D information transmission, along with the decline of the depth perception of stereo images to a certain extent. These conditions affect users' immersion experience. Furthermore, the discomfort caused by 3D display limits the development of 3D technology. On the one hand, an effective stereo image quality evaluation technology can provide new ideas for image compression standards. On the other hand, the technology provides a reference for the rational improvement of the quality of 3D videos, thereby accelerating the development of 3D multimedia application technology. Providing content that conforms to users' viewing experience is of paramount importance for the further promotion of 3D multimedia technology. The stereoscopic image quality evaluation method that conforms to users' visual characteristics helps to accurately and objectively reflect the visual perception experience when users watch 3D images or videos. On the basis of different dimensions that affect the quality of stereoscopic image experience, four categories of image quality assessment algorithms are used:image quality evaluation based on distortion, quality of experience based on depth perception, quality of experience based on comfort, and comprehensive dimensions. The quality of experience (QoE) represents the quality of the stereoscopic visual experience of users. QoE is an objective result that takes users as the core and considers the multi-dimensional perception factors that comprehensively affect it. The stereo image quality is the result of the three perceptions of distortion, depth, and comfort of 3D images. Distortion quality indicates the degree of image degradation caused by image distortion. Depth quality indicates the depth and immersion feeling experienced when viewing 3D content. Visual comfort indicates the degree of visual fatigue experienced when viewing stereoscopic images. The existing research on the objective evaluation of 3D QoE only evaluates results beginning from one or two dimensions of image distortion, stereoscopic perception, and visual comfort. However, in an actual subjective experiment, we found that any change in dimensions leads to changes in the quality of the stereo image experience and that existing methods do not comprehensively consider the three factors of image distortion, depth perception, and comfort. To evaluate the visual perception experience of 3D images comprehensively and accurately, this study proposes a stereoscopic image experience quality evaluation method that is based on users multi-dimensional perception. Method A distortion-free natural scene image has a certain regularity in distribution, and image distortion causes its distribution law to change; thus, image quality can be estimated from the extracted feature parameters. The left and right eye images are subtracted and added to obtain the difference image and fused image. Then, the difference image and fused image are fitted by the generalized Gaussian distribution function, and the fitting parameters are obtained as the distortion quality features. Distortion reduces the depth perception quality of stereo images. It exerts two main effects on depth perception. First, the relative depth information between objects is lost, and the position of the object consequently becomes blurred, thereby affecting depth perception. Second, distortion reduces the feature points at which the left and right viewpoint images are matched; thus, the binocular depth perception information is reduced, thereby diminishing the sense of depth. Then, the distortion-sensitive pixel map is obtained. For each distortion-sensitive pixel, the neighborhood brightness distribution is calculated. SIFT (scale invariant feature transform) key point matching is performed on the left and right views. The statistical result of the neighborhood brightness distribution and the key point matching quantity are used as the depth quality feature. When the parallax of the stereoscopic image exceeds a certain range, the human eye may generate a convergence conflict, thereby resulting in visual fatigue. The human eye is only sensitive to the comfort/discomfort characteristics of the significant area. Thus, we adopt the comfort evaluation model based on the visual important regions and extract the mean parallax value of the significant area. Finally, the three-dimensional features are combined as the experience quality feature vector, and the objective prediction model is constructed by support vector regression. Result Experimental results on the LIVE database and Waterloo IVC database show that the proposed method correlates with people's subjective perception at values of 0.942 and 0.858, which are better than those of other methods. Conclusion The method fully uses the characteristics of the stereo image, and the evaluation result is better than that of several classical algorithms. Therefore, the evaluation result of the constructed model shows improved consistency with the subjective experience of the user. In the future, we will combine the evaluation process with the stereo image quality optimization process and guide quality optimization to stereo images from various dimensions.
Keywords

订阅号|日报