A no-reference objective image quality metric based on perceptually weighted local noise
© Zhu and Karam; licensee Springer. 2014
Received: 15 April 2013
Accepted: 23 December 2013
Published: 16 January 2014
This work proposes a perceptual based no-reference objective image quality metric by integrating perceptually weighted local noise into a probability summation model. Unlike existing objective metrics, the proposed no-reference metric is able to predict the relative amount of noise perceived in images with different content, without a reference. Results are reported on both the LIVE and TID2008 databases. The proposed no-reference metric achieves consistently a good performance across noise types and across databases as compared to many of the best very recent no-reference quality metrics. The proposed metric is able to predict with high accuracy the relative amount of perceived noise in images of different content.
Reliable assessment of image quality plays an important role in meeting the promised quality of service (QoS) and in improving the end user’s quality of experience (QoE). There is a growing interest to develop objective quality assessment algorithms that can predict perceived image quality automatically. These methods are highly useful in various image processing applications, such as image compression, transmission, restoration, enhancement, and display. For example, the quality metric can be used to evaluate and control the performance of individual system components in image/video processing and transmission systems.
One direct way to evaluate video quality is through subjective tests. In these tests, a group of human subjects are asked to judge the quality under a predefined viewing condition. The scores given by observers are averaged to produce the mean opinion score (MOS). However, subjective tests are time-consuming, laborious, and expensive. Objective image quality (IQA) assessment methods can be categorized as full reference (FR), reduced reference (RR), and no reference (NR) depending on whether a reference, partial information about a reference, or no reference is used for calculation. Quality assessment without a reference is challenging. A no-reference metric is not relative to a reference image, but rather an absolute value is computed based on some characteristics of the test image.
Of particular interest to this work is the no-reference noisiness objective metric. Noisiness and blurriness are two key distortions in multiple applications, and typically there is a tradeoff to balance between noisiness and blurriness. For example, in soft-thresholding for image denoising , the image could be blurry when the threshold is high, while the image could remain noisy when the threshold is low. Also, in Wiener-based super-resolution , too much regularization will result in less noise at the expense of more blur. The reconstructed image could be blurry when the auto-correlation function is modeled to be too flat, while the reconstructed image could be noisy when the auto-correlation function is modeled to be too sharp. No-reference image sharpness/blur metrics have been widely discussed [3, 4]. However, these image sharpness/blur metrics typically fail in the presence of noise. The sharpness metric may increase when noise increases. A no-reference noise-immune image sharpness metric was also proposed . Furthermore, all the edge-based sharpness metrics can be easily applied in the wavelet domain as described in  to provide resilience to noise. Still, it lacks the ability to assess the impairment due to noise. For visual quality assessment of noisiness, many full-reference metrics are presented in , such as peak signal-to-noise ratio (PSNR), multi-scale structural similarity (MS-SSIM) , noise quality measure (NQM) , and information fidelity criterion (IFC) . However, these full-reference metrics require the reference image for calculation. There is a need to develop a no-reference noisiness quality metric. Furthermore, such noisiness metric could be further combined with the no-reference blur metrics [3, 4] to provide a better prediction of image quality for several applications including super-resolution, image restoration, and other multiply distorted images. A global estimate of image noise variance was used as a no-reference noisiness metric in . The histogram of the local noise variances is used to derive the global estimate. However, the locally perceived visibility of noise is not considered. Similarly in , noisiness is expressed by the sum of estimated noise amplitudes and the ratio of noise pixels. Both the metrics of [10, 11] do not account for the effects of locally varying noise on the perceived noise impairment and they do not exploit the characteristics of the human visual system (HVS).
To tackle this issue, this paper firstly presents a full-reference image noisiness metric which integrates perceptually weighted local noise into a probability summation model. This proposed metric can predict the perceptual noisiness in images with high accuracy. In addition, a no-reference objective noisiness metric is derived based on local noise standard deviation, local perceptual weighting, and probability summation. The experimental results show that the proposed FR and NR metrics show better and more consistent performance across databases and distortion types, when compared with several very recent FR and NR metrics.
The remainder of this paper is organized as follows. A perceived noisiness model based on probability summation is presented first followed by details on the contrast sensitivity thresholds computation. A full-reference perceptually weighted noise (FR-PWN) metric is proposed next based on perceptual weighting using the computed contrast sensitivity thresholds and probability summation. After that, a no-reference perceptually weighted noise (NR-PWN) metric is further derived. Performance results and comparison with existing metrics are presented followed by a conclusion.
Perceptual noisiness model based on probability summation
where JND(i,j) is the JND value at (i,j) and it depends on the mean intensity in a local neighborhood region surrounding pixel (i,j). β is a parameter whose value is chosen to maximize the correspondence of (2) with the experimentally determined psychometric function for noise detection. In psychophysical experiments that examine summation over space, a value of about 4 has been observed to correspond well to probability summation .
From (4), it can be seen that Pnoise(R) increases if D R increases and vice versa. So D R can be used as a noisiness metric over region R. However, the probability of noise detection does not directly translate to noise annoyance level. In this work, the β parameter in (4) and (5) is replaced with α=β × s, which has the effect of steering the slope of the psychometric function in order to translate noise detection levels into noise annoyance levels. The factor s was found experimentally to be 1/16 resulting in a value of 0.25 for α. More details about how JND(i,j) is computed is given in the Section ‘Perceptual contrast sensitivity threshold model and JND computation’.
Perceptual contrast sensitivity threshold model and JND computation
where is the intensity level at pixel location (n1,n2) in a N × N region surrounding pixel (i,j). It should be noted that the indices (n1,n2) are used to denote the location with respect to the top left corner of the N × N region, while the indices (i,j) are used to denote the location with respect to the top left corner of the whole image. is the mean value over the considered N × N region surrounding pixel (i,j). α T is a correction exponent that controls the degree to which luminance masking occurs and is set to α T = 0.649, as given in . JND(i,j) in (5) is computed using (14). In our implementation, N = 8 was used for the N × N region.
Full-reference noisiness metric
The resulting distortion measure, D, normalized by the number of blocks, is adopted as the proposed full-reference metric FR-PWN. This full-reference metric not only works for noisiness, but could also work for other additive distortions.
No-reference noisiness metric
The resulting noise measure D, normalized by the number of blocks, is adopted as the proposed no-reference NR-PWN metric.
In (24), the noise variance is estimated directly from the test image, without the reference image. Multiple methods are available to estimate the noise variance, such as fast noise variance estimation (FNV)  and generalized cross validation (GCV)-based method [20, 21]. In our implementation, the GCV method was used for computing the local noise variance. Similar results were also obtained using the FNV  noise estimation method.
The performance of the proposed FR-PWN and NR-PWN metrics is assessed using the LIVE  and TID2008  databases.The LIVE database  consists of 29 RGB color image. The images are distorted using different distortion types: JPEG2000, JPEG, Gaussian blur, white noise, and bit errors. The difference mean opinion score (DMOS) for each image is provided. The white noise part of the LIVE database includes 174 images with a noise standard deviation ranging from 0 to 2. White noise was added to the RGB components of images after scaling between 0 and 1. All of the white noise images (174 images) from the LIVE database are used in our experiments. The TID2008 database  consists of 25 reference images (512 × 384) and 1,700 distorted images. The images are distorted using 17 types of distortions, including additive Gaussian noise, high-frequency noise, JPEG2000, and Gaussian blur. The MOS was obtained using a total of 838 observers with 256,428 comparisons of the visual quality of distorted images. All of the additive Gaussian noise image (100 images) and high-frequency noise images (100 images) from the TID2008 database are used in our experiments. As mentioned in , additive zero-mean noise is often present in images and it is commonly modeled as a white Gaussian noise. This type of distortion is included in most studies of quality metric effectiveness. High-frequency noise is an additive non-white noise which can be used for analyzing spatial frequency sensitivity of the HVS . High-frequency noise is typical in lossy image compression and watermarking.
Performance evaluation for the LIVE database
Fuzzy S7 
BSDM (S4) 
Estimated noise standard deviation
Performance evaluation using SROCC for the TID2008 database
BLINDS-II (SVM) 
BLINDS-II (Prob.) 
Li et al. 
From Table 1, it can be observed that the proposed FR-PWN metric outperforms the existing FR metrics for the LIVE database while achieving a similar performance as the NQM  metric. Table 2 shows that the proposed FR-PWN metric outperforms the existing FR metrics for the TID2008 database, on both Gaussian noise and high-frequency noise. The proposed NR-PWN metric comes close in performance to the proposed FR-PWN metric for both the LIVE and the TID2008 databases. In particular, Table 1 shows that the proposed NR-PWN metric performs better than existing NR metrics except for the Blinds-II and BRISQUE metrics in terms of PLCC. The proposed NR-PWN metric outperforms all the considered NR metrics in terms of SROCC and even existing FR metrics except the full-reference NQM  for the LIVE database. Table 2 shows that the proposed NR-PWN metric surpasses existing NR metrics except BRISQUE  for additive Gaussian noise, and that it significantly outperforms existing FR and NR metrics for high-frequency noise. Particularly, it should be noted that the performance of BRISQUE  drops dramatically on high-frequency noise and is significantly lower than the proposed metric. In addition, many of the shown state-of-the-art metrics including BLINDS-II , NIQE , and BRISQUE  use 80% of the data for training [30, 32, 33]. Consequently, these may not perform well on new distortions outside the training set, such as high-frequency noise (Table 2). In contrast, the proposed NR-PWN does not require training and still performs well on this new distortion.
Furthermore, it is worth indicating that as shown in Tables 1 and 2, the existing metrics exhibit differences in performance across different databases and types of distortions. It is noted in  that the performance of many image quality metrics could be quite different across databases. The difference in performance can be attributed to the differences in quality range, distortions, and contents across databases. Despite this, the results obtained show that the proposed FR-PWN and NR-PWN metrics achieve consistently a good performance across noise types (white noise and high-frequency noise) and across databases as compared to the existing quality metrics. For example, the proposed FR-PWN metric exhibits a performance similar to NQM  for the LIVE database, while it significantly outperforms NQM  for white noise images from TID2008. Also, the existing BLINDS-II  performs fairly well for the LIVE database, but its performance significantly decreases when applied to TID2008. It is also interesting to note that although the mathematical derivations for the proposed NR-PWN is based on white noise, the proposed NR-PWN metric performs consistently well for high-frequency noise, a non-white noise.
The performance results presented in Tables 1 and 2 for the proposed NR-PWN metric are obtained using the GCV method [20, 21] for local variance estimation. If the local variance is estimated using the FNV method , the resulting SROCC values are 0.9627 for the LIVE database additive Gaussian noise, 0.7850 for the TID2008 database additive Gaussian noise, and 0.9210 for the TID2008 database high-frequency noise, respectively.
SROCC of the proposed metrics using different L max
This paper proposed both a full-reference and a no-reference noisiness metrics. The no-reference noisiness metric is derived from the proposed full-reference metric and integrates noise variance estimation and perceptual contrast sensitivity thresholds into a probability summation model. The proposed metrics can predict the relative noisiness in images based on the probability of noise detection. Results show that the proposed metrics achieve a consistently good performance across noise types and across databases as compared to the existing quality metrics. Further work can be performed to develop a no-reference quality metric for multiply distorted images.
- Donoho D: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 1995, 41(3):613-627. 10.1109/18.382009MathSciNetView ArticleMATHGoogle Scholar
- Hardie R: A fast image super-resolution algorithm using an adaptive Wiener Filter. IEEE Trans. Image Process 2007, 16(12):2953-2964.MathSciNetView ArticleGoogle Scholar
- Ferzli R, Karam L: A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process 2009, 18(4):717-728.MathSciNetView ArticleGoogle Scholar
- Narvekar N, Karam L: A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans. Image Process 2011, 20(9):2678-2683.MathSciNetView ArticleGoogle Scholar
- Ferzli R, Karam L: No-reference objective wavelet based noise immune image sharpness metric. In IEEE International Conference on Image Processing, ICIP 2005. IEEE, Piscataway; 2005:I-405–8.View ArticleGoogle Scholar
- Sheikh H, Sabir M, Bovik A: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process 2006, 15(11):3440-3451.View ArticleGoogle Scholar
- Wang Z, Simoncelli E, Bovik A: Multiscale structural similarity for image quality assessment. In Proceedings of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, 2004. Pacific Grove; 9–12 November 2003:1398-1402.Google Scholar
- Damera-Venkata N, Kite T, Geisler W, Evans B, Bovik A: Image quality assessment based on a degradation model. IEEE Trans. Image Process 2000, 9(4):636-650. 10.1109/83.841940View ArticleGoogle Scholar
- Sheikh H, Bovik A, De Veciana G: An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process 2005, 14(12):2117-2128.View ArticleGoogle Scholar
- Farias MCQ, Mitra S: No-reference video quality metric based on artifact measurements. In IEEE International Conference on Image Processing, ICIP 2005. IEEE, Piscataway; 2005:III-141–144.View ArticleGoogle Scholar
- Choi MG, Jung JH, Jeon JW: No-reference image quality assessment using blur and noise. Int. J. Electrical Electron. Eng 2009, 3: 318-322.Google Scholar
- Hontsch I, Karam L: Locally adaptive perceptual image coding. IEEE Trans. Image Process 2000, 9(9):1472-1483. 10.1109/83.862622MathSciNetView ArticleMATHGoogle Scholar
- Robson JG, Graham N: Probability summation and regional variation in contrast sensitivity across the visual field. Vis. Res 1981, 21: 409-418. 10.1016/0042-6989(81)90169-3View ArticleGoogle Scholar
- Karam L, Sadaka N, Ferzli R, Ivanovski Z: An efficient selective perceptual-based super-resolution estimator. IEEE Trans. Image Process 2011, 20(12):3470-3482.MathSciNetView ArticleGoogle Scholar
- Ahumada A, Peterson H: Luminance-model-based DCT quantization for color image compression. Human Vision, Visual Processing, and Digital Display III, SPIE Proc 1992, 1666: 365-374. 10.1117/12.135982View ArticleGoogle Scholar
- Watson AB: DCT quantization matrices visually optimized for individual images. Human Vision, Visual Processing, and Digital Display IV, SPIE Proc 1913: 202-216. (SPIE, San Jose, 1993)Google Scholar
- Liu Z, Karam L, Watson A: JPEG2000 encoding with perceptual distortion control. IEEE Trans. Image Process 2006, 15(7):1763-1778.View ArticleGoogle Scholar
- Wilkelbauer A: Moments and absolute moments of the normal distribution. (2012), . (Accessed 15 April 2013) http://arxiv.org/abs/1209.4340 (2012), . (Accessed 15 April 2013)
- Immerkær J: Fast noise variance estimation. Comput. Vis. Image Underst 1996, 64(2):300-302. . http://www.sciencedirect.com/science/article/pii/S1077314296900600 10.1006/cviu.1996.0060View ArticleGoogle Scholar
- Wahba G: Estimating the smoothing parameter. In Spline Models for Observational Data. Society for Industrial Mathematics, Philadelphia; 1990:45-65.View ArticleGoogle Scholar
- Garcia D: Robust smoothing of gridded data in one and higher dimensions with missing values. Comput. Stat. Data Anal 2010, 54(4):1167-1178. 10.1016/j.csda.2009.09.020View ArticleMathSciNetMATHGoogle Scholar
- Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, Battisti F: TID2008-A database for evaluation of full-reference visual quality assessment metrics. Adv. Mod. Radioelectronics 2009, 10: 30-45.Google Scholar
- Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V: On between-coefficient contrast masking of DCT basis functions. In Proceedings of the Third International Workshop on Video Processing and Quality Metrics. Scottsdale; January 2007.Google Scholar
- VQEG: Final report from the Video Quality Experts Group on the validation of objective models of video quality assessment. 2000.http://www.vqeg.org/ . (Accessed 15 April 2013)Google Scholar
- Watson AB: DCTune: A technique for visual optimization of DCT quantization matrices for individual images. Soc. Inf. Display Dig. Tech. Papers 1993, 24: 946-949.Google Scholar
- Miyahara M, Kotani K, Algazi V: Objective picture quality scale (PQS) for image coding. IEEE Trans. Commun 1998, 46(9):1215-1226. 10.1109/26.718563View ArticleGoogle Scholar
- Weken DV, Nachtegael M, Kerre EE: Using similarity measures and homogeneity for the comparison of images. Image Vis. Comput 2004, 22(9):695-702. 10.1016/j.imavis.2004.03.002View ArticleGoogle Scholar
- Avcibas I, Avcıbaş I, Sankur B, Sayood K: Statistical evaluation of image quality measures. J. Electron. Imaging 2002, 11: 206-223. 10.1117/1.1455011View ArticleGoogle Scholar
- Wang Z, Wu G, Sheikh H, Simoncelli E, Yang EH, Bovik A: Quality-aware images. IEEE Trans. Image Process 2006, 15(6):1680-1689.View ArticleGoogle Scholar
- Saad M, Bovik A, Charrier C: Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans. Image Process 2012, 21(8):3339-3352.MathSciNetView ArticleGoogle Scholar
- Shen J, Li Q, Erlebacher G: Hybrid no-reference natural image quality assessment of noisy, blurry, JPEG2000, and JPEG images. IEEE Trans. Image Process 2011, 20(8):2089-2098.MathSciNetView ArticleGoogle Scholar
- Mittal A, Moorthy A, Bovik A: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process 2012, 21(12):4695-4708.MathSciNetView ArticleGoogle Scholar
- Mittal A, Soundararajan R, Bovik A: Making a completely blind image quality analyzer. IEEE Signal Process. Lett 2013, 20(3):209-212.View ArticleGoogle Scholar
- Moorthy A, Bovik A: A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett 2010, 17(5):513-516.View ArticleGoogle Scholar
- Tang H, Joshi N, Kapoor A: Learning a blind measure of perceptual image quality. In 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway; 2011:305-312.Google Scholar
- Li C, Bovik A, Wu X: Blind image quality assessment using a general regression neural network. IEEE Trans. Neural Netw 2011, 22(5):793-799.View ArticleGoogle Scholar
- Li C, Tang G, Wu X, Ju Y: No-reference image quality assessment with learning phase congruency feature. J. Electrical Inf. Technol 2012, 35(2):484-488.View ArticleGoogle Scholar
- Tourancheau S, Autrusseau F, Sazzad Z, Horita Y: Impact of subjective dataset on the performance of image quality metrics. In 15th IEEE International Conference on Image Processing. IEEE Piscataway; 2008:365-368.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.