No-reference image blur assessment using multiscale gradient
© Chen and Bovik; licensee Springer. 2011
Received: 15 September 2010
Accepted: 19 July 2011
Published: 19 July 2011
The increasing number of demanding consumer video applications, as exemplified by cell phone and other low-cost digital cameras, has boosted interest in no-reference objective image and video quality assessment (QA) algorithms. In this paper, we focus on no-reference image and video blur assessment. We consider natural scenes statistics models combined with multi-resolution decomposition methods to extract reliable features for QA. The algorithm is composed of three steps. First, a probabilistic support vector machine (SVM) is applied as a rough image quality evaluator. Then the detail image is used to refine the blur measurements. Finally, the blur information is pooled to predict the blur quality of images. The algorithm is tested on the LIVE Image Quality Database and the Real Blur Image Database; the results show that the algorithm has high correlation with human judgments when assessing blur distortion of images.
KeywordsNo-reference blur metric Gradient histogram Multi-resolution analysis Information pooling
With the rapid and massive dissemination of digital images and videos, people live in an era replete with digitized visual information. Since many of these images are of low quality, effective systems for automatic image quality differentiation are needed. Although there are a variety of effective full-reference (FR) quality assessment (QA) models, such as the PSNR, the structural similarity (SSIM) index [1, 2], the visual information fidelity index , and the visual signal-to-noise ratio (VSNR) , models for no-reference (NR) QA have not yet achieved performance that is competitive with top performing FR QA models. As such, research in the area of blind or NR QA remains quite vital.
There are many artifacts that may occur in a distorted image, such as blocking, ringing, noise, and blur. Unlike FR QA, where a reference is available to test against any distortion, NR QA approaches generally seek to capture one or a few distortions. Here we are mainly concerned with NR blur assessment, which remains an important problem in many applications. Generally, humans tend to conclude that images with more detail are of higher quality. Of course, the question is not so simple, since blur can be space-variant, may depend on depth-of-field (hence effect foreground and background objects differently), and may depend on what is being blurred in the image.
A number of NR blur indices have been developed, the majority of which are based on the analyzing luminance edges. For example, the sharpness measurement index proposed by Caviedes and Gurbuz  is based on local edge kurtosis. The blur measurement metric proposed by Marziliano et al.  is based on analyzing of the width or spread of edges in an image, while their other work is based on an analysis of edges and adjacent regions in an image . Chuang et al.  evaluate blur by fitting the image gradient magnitude to a normal distribution, while Karam et al. develop a series of blur metrics based on the different types of analysis applied to edges [9–13].
Other researchers have studied blur assessment by frequency domain analysis of local DCT coefficients , and of image wavelet coefficients [15–17]. These methods generally rely on a single feature to accomplish blur assessment. While some of these algorithms deploy simple perceptual models in their design [7, 9, 11, 12, 17], a theme that we extend in our approach. Specifically, we use a model of neural pooling of the responses of correlated neuronal populations in the primary visual cortex . The use of multiple features combined using machine learning methods has also been studied [19, 20].
We are also inspired by recent progress on utilizing natural scene statistics (NSS) to improve image processing algorithms. Natural images obey specific statistical laws that, in principle, might be used to distinguish natural images from artificially distorted images . In this regard, images that are blurred beyond a norm (e.g., more than the band limit provided by the normal human lens at optical center) may measurably depart from statistical "naturalness." By this philosophy, we may anticipate that NR blur indices can be designed that analysis image statistics. Indeed, Sheikh et al. successfully used NSS for NR QA of JPEG-2000 distorted images . In their work, specific NSS features drawn from the gradient histogram were used.
Here we develop a new blur assessment index that operates in a coarse-to-fine manner. First, a coarse blur measurement using gradient histogram features is deployed that relies on training a probabilistic support vector machine (SVM). A multi-resolution analysis is then used to improve the blur assessment, deploying a model of neural pooling in cortical area V1 . The overall algorithm is shown to agree well with human subjectivity.
The rest of the paper is organized as follows: Section 2 describes the way in which NSS are used. Section 3 describes the coarse-scale NR blur index. Section 4 extends the metric using multi-resolution analysis. Section 5 explains the use of the neural pooling model. The overall NR blur index is evaluated in Section 6, and concluding remarks are given in Section 7.
2. Natural image statistics
Recent research on natural image statistics have shown that natural scenes belong to a small set in the space of all possible image signals . One example of a natural scene property is the greater prevalence of strong image gradients along the cardinal (horizontal and vertical) orientations, in images projected from both indoor and outdoor scenes. A number of researchers have developed statistical models that describe generic natural images  (including images of man-made scenes).
Liu et al.  and Levin  have demonstrated that measurements on the heavy tailed distributions of gradients can be used for blur detection. Liu et al. used the gradient histogram span as a feature in their classification model. Levin fits the observed gradient histogram using a mixture model.
3. Probabilistic SVM For blur assessment
Based on our discussion of NSS, we seek to evaluate the distance between the gradient statistics of an (ostensibly distorted) image and a statistical model of natural scenes. This distance can then be used for image QA.
A classification method is used to measure the distance. We classify the images into two groups. One is tagged as "sharp" and the other as "blurred." Using the probabilistic SVM classification model, confidence values are computed that represent the distance between the test image and the training set. A higher confidence value implies a higher certainty of the classification result. In this case, this means that the test sample is closer to the assigned class center, i.e., the statistic of the test image is closer to that of "sharp" or "blurred" images.
We chose to use a SVM  as our classification model. The main reason for using SVM is that it works well for classifying a few classes with few training samples. This is highly suitable for our application having only two classes. Moreover, SVM allows substitution of kernels to achieve better classification results. Although here we only use the default kernel, the possibility of modifying the kernel leaves room for performance improvement.
Due to the limited scope of the coarse evaluation of the image, we use the entire gradient histogram as a feature, rather than simple measured parameter such as the mean or the slope of the histogram [25, 26]. While this implies a fairly large number of features, it is not very large, and the small number of classes ensures reasonable computation. We describe the training procedure and the dataset used in Section 6.
4. Multi-resolution NR QA of blur
As in most other areas of image processing and analysis, multi-resolution methods have afforded improved performance relative to single-resolution methods for image QA [2, 4]. In the following, we modify QS-SVM using information derived from a multi-resolution decomposition.
where W i and H i are the pixel dimensions of the subband image that DS i is defined on, and is the gradient magnitude value of the subband image at coordinate (m, n).
Blur quality score is the final blur evaluation result, which is the weighted (by exponents) product of the full-resolution score QS-SVM and the values of DS from each layer. The parameters r i are normalized exponents: .
5. Decoding the neural responses
Perceptual models have played an important role in the development of image QA algorithms. These have included image masking models , cortical decompositions , extra-cortical motion processing , and foveation [29, 32–34], among others.
Since a block size of 2 n along each dimension facilitates optimization and allows better memory management (aligned memory allocation), the choice of a 64 × 64 block size is a decent approximation. We then apply the blur QA method described in Section 4 on each of these blocks.
When a human observer studies an image, thus arriving at a sense of its quality, she/he engages in a process of eye movement, where visual saccades place fixations at discrete points on the image. Image quality is largely decided by information that is collected from these foveal regions, with perhaps, additional information drawn from extra-foveal information. The overall perception of quality drawn from these fixated regions might be described as "attentional pooling," by analogy with the aggregation of information from spatially distributed neurons. We utilize the results of a study conducted by Chen et al.  to formulate such an attentional pooling strategy. In this study, the authors examined the efficacy of different patch pooling strategies in a primate undergoing a visual (Gabor) target detection task.
where w i is the weight applied to the neuronal amplitude response x i .
Maximum average amplitude: w i ≠ 0 only for the patch having maximum average neuronal response amplitude.
Maximum d': w i ≠ 0 only for the patch having maximum d'
Maximum amplitude: w i ≠ 0 only for the site with maximum amplitude in a given trial
Mean amplitude: w i = 1/n
Weighted average amplitude: w i is proportional to the average amplitude response of x i
Weighted d': w i is proportional to d'
where DS ki is the detail response of block k from layer i, and w ki = 1/p if the detail responses of block k in layer i belong to the largest 10% of detail responses of all activated blocks in the layer; otherwise w ki = 0. Here, p is nominally set to 10. The blocking analysis and pooling are only applied on the multi-resolution part, since the NSS mentioned in Section 2 are based on the statistics of whole images.
6. Experiments and results
The LIVE image quality database  and the real blur image Database  were used to evaluate the performance of our algorithm. The experiments in Sections 6.1-6.3 were conducted on the LIVE database to gain insights into the performance of algorithms that combine different blur assessment factors. The performances are also compared to the performance of multi-scale SSIM (or MS-SSIM, a popular and effect FR QA method).
6.1. Performance of SVM Classification
To train the coarse SVM classifier, we used 240 training samples which were marked as "sharp" or "blurred." The training samples were randomly chosen and some of them are out-of-focus images. Due to the unbalanced quality of the natural training samples (there were more sharp images than naturally blurred images), we applied a gaussian blur to some of the sharp samples to generate additional blurred samples. The final training set included 125 sharp samples and 115 blurred samples. The training and test sets do not share content.
When tagging samples, if an original image's quality was mediocre, the image was duplicated; one copy marked as "blurred" and the other marked as "sharp," with both images used for training. This procedure prevents misclassifications arising from marking mediocre image as "sharp" or "blurred." This duplication was applied to lower the confidence when classifying mediocre samples.
Note that DMOS scores of these images we are not required to train the SVM. Images were simply tagged as "blurred" or "sharp" to train the SVM. Likewise, the output of the probabilistic SVM model is a class type ("blurred" or "sharp") and a confidence level. The class type and confidence level are used to predict the image quality score.
Comparison of the performance of VQA algorithms
In Table 1, QS-SVM means blind blur QA using probabilistic SVM, PSNR means peak signal to noise ratio, and MS-SSIM means multi-scale structure similarity index. To obtain an objective evaluation result, we compared our method to FR methods tested on the same database as in [4, 38].
As can be seen, the coarse algorithm QS-SVM delivered lower SROCC scores than the FR indices, although the results are promising. Of course, QS-SVM is not trained on DMOS scores, hence does not fully capture the perceptual elements of blur assessment.
6.2. Performance with multi-resolution decomposition
QA performance using different layers
In Table 2, DS0 is the detail score computed from the original image. The experiment shows the SROCC score of DS1 to be significantly higher than for the other layers. The detail map at this middle scale appears to deliver a high correlation with human impression of image quality.
QA performance using different combinations of layers
Table 3 shows that, except for combination with QS-SVM, all other combinations with DS1 did not achieve higher performance than using only DS1. This result is consistent with our other work in FR QA, where we have found that mid-band QA scores tend to score higher than low-band or high-band scores. Adding more layers did not improve performance here. The highest performance occurs by combining DS1 with QS-SVM (r0 = 0.610, r1 = 0.390), yielding an impressive SROCC score of 0.9105. Combination QS-SVM with DS2 (r0 = 0.683, r2 = 0.317) also improved performance relative to DS2, suggesting that QS-SVM and the DS scores offer complementary measurements.
6.3. Performance with pooling strategy
QA performance numbers by tenfold cross-validation
Summary of QA performance of different algorithms on the blurred image portion of the LIVE Image Quality Database
6.4. Challenging blur database
Our foregoing experiments on the LIVE database were by way of algorithm design and tuning, and not performance verification. To verify the performance of our algorithm, we conducted an experiment on a real blurred image database. The database contains 585 images with resolutions ranging from 1280 × 960 to 2272 × 1704 pixels.
Blur QA performance of applying different pooling rules on real blur database
By examining the experimental results from the LIVE Image Quality Database and the Real Blur Image Database, we found that there is a significant performance difference of the models on these two databases. The LIVE database includes synthetically and globally blurred sample images. The task of performing QA on a globally blurred image is less complex and harder to relate to perceptual models. On LIVE, our proposed method of pooling showed significant improvement (from 0.9 to 0.925). However, on the Real Blur Database, where the blurs are more complex, possibly nonlinear, and spatially variant, blur perception is more complex and probably more correlated with content (e.g., what is blurred in the image?). By example, in the partially blurred image shown in Figure 10 (bottom right), the rating is likely highly affected by image content, object positioning, probable viewer fixation, and so on.
Blur QA performance of different algorithms on real blur database
Frequency domain metric*
HVS based metric*
Local phase coherence metric*
The main contributions of this work are as follows. First, we found that the statistics of the image gradient histogram and a detail map from the image wavelet decomposition can be combined to yield good NR blur QA performance. Second, our results discuss that a perceptually motivated pooling strategy can be used to improve the NR blur index on assessing the blur images.
Performance was demonstrated on the LIVE Image Quality Database and the Real Blur Image Database. As compared with other NR blur metrics, our method yields competitive performance with reasonable complexity.
support vector machine
visual signal-to-noise ratio
natural scene statistics
Spearman rank order correlation coefficient
human visual system
multi-features neural network classifier.
- Wang Z, Bovik AC, Sheikh HR, Simoncelli EP: Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004, 13(4):600-612. 10.1109/TIP.2003.819861View ArticleGoogle Scholar
- Wang Z, Simoncelli EP, Bovik AC: Multi-scale structural similarity for image quality assessment. IEEE Asilomar Conf. Signals, Systems, and Computers 2003, 1398-1402.Google Scholar
- Sheikh HR, Bovik AC: Image information and visual quality. IEEE Trans Image Process 2006, 15(2):430-444.View ArticleGoogle Scholar
- Chandler DM, Hemami SS: VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 2007, 16(9):2284-2298.View ArticleMathSciNetGoogle Scholar
- Caviedes JE, Gurbuz S: No-reference sharpness metric based on local edge kurtosis. IEEE International Conference on Image Processing, Rochester, NY 2002.Google Scholar
- Marziliano P, Dufaux F, Winkler S, Ebrahimi T: A no-reference perceptual blur metric. International Conference on Image Processing, Rochester, NY 2002.Google Scholar
- Marziliano P, Dufaux F, Winkler S, Ebrahimi T: Perceptual blur and ringing metrics: application to JPEG2000. Signal Process Image Commun 2004, 19(2):163-172. 10.1016/j.image.2003.08.003View ArticleGoogle Scholar
- Chung Y, Wang J, Bailey R, Chen S, Chang S: A nonparametric blur measure based on edge analysis for image processing applications. IEEE Conf Cybern Intell Syst 2004, 1: 356-360.Google Scholar
- Narvekar ND, Karam LJ: A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection. International Workshop on Quality of Multimedia Experience 2009.Google Scholar
- Narvekar ND, Karam LJ: A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans Image Process 2011, in press.Google Scholar
- Ferzli R, Karam LJ: A human visual system based no-reference objective image sharpness metric. IEEE International Conference on Image Processing, Atlanta, GA 2006.Google Scholar
- Ferzli R, Karam LJ: A no-reference objective mage sharpness metric based on just-noticeable blur and probability summation. IEEE International Conference on Image Processing, San Antonio, TX 2007.Google Scholar
- Varadarajan S, Karam LJ: An improved perception-based no-reference objective image sharpness metric using iterative edge refinement. IEEE International Conference on Image Processing, Chicago, IL 1998.Google Scholar
- Marichal X, Ma WY, Zhang H: Blur determination in the compressed domain using DCT information. IEEE International Conference on Image Processing, Kobe, Japan 1999.Google Scholar
- Tong H, Li M, Zhang H, Zhang C: Blur detection for digital images using wavelet transform. IEEE International Conference on Multimedia and EXPO 2004, 1: 17-20.Google Scholar
- Ciancio A, Targino AN, da Silva EAB, Said A, Obrador P, Samadani R: Objective no-reference image quality metric based on local phase coherence. IET Electron Lett 2009, 45(23):1162-1163. 10.1049/el.2009.1800View ArticleGoogle Scholar
- Wang Z, Simoncelli EP: Local phase coherence and the perception of blur. In Advances in Neural Information Processing Systems. MIT Press, Cambridge; 2004:786-792.Google Scholar
- Chen Y, Geisler WS, Seidemann E: Optimal decoding of correlated neural population responses in the primate visual cortex. Nat. Neurosci 2006, 9: 1412-1420.Google Scholar
- Narwaria M, Lin W: Objective image quality assessment based on support vector regression. IEEE Trans Neural Netw 2010, 21(3):515-519.View ArticleGoogle Scholar
- Ciancio A, Targino da Costa ALN, da Silva EAB, Said A, Samadani R, Obrador P: No-reference blur assessment of digital pictures based on multi-feature classifiers. IEEE Trans Image Process 2011, 20(1):64-75.View ArticleMathSciNetGoogle Scholar
- Simoncelli EP: Statistical models for images: compression, restoration and synthesis. Proceeding of the IEEE Asilomar Conference on Signals, Systems, and Computers 1997.Google Scholar
- Sheikh HR, Bovik AC, Cormack LK: No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans Image Process 2005, 14(11):1918-1927.View ArticleGoogle Scholar
- Ruderman DL: The statistics of natural images. Netw Comput Neural Syst 1994, 5(4):517-548. 10.1088/0954-898X/5/4/006View ArticleMATHGoogle Scholar
- Field D: What is the goal of sensory coding? Neural Comput 1994, 6: 559-601. 10.1162/neco.19126.96.36.1999View ArticleGoogle Scholar
- Liu R, Li Z, Jia J: Image partial blur detection and classification. IEEE International Conference on Computer Vision Pattern Recognition 2008, 1-8.Google Scholar
- Levin A: Blind motion deblurring using image statistics. Neural Information Processing Systems (NIPS) 2006, 841-848.Google Scholar
- Abdelnour AF, Selesnick IW: Nearly symmetric orthogonal wavelet bases. IEEE International Conference on Acoustic, Speech, Signal Processing (ICASSP) 2001.Google Scholar
- Teo PC, Heeger DJ: Perceptual image distortion. IEEE International Conference on Image Processing, Austin, TX 1994.Google Scholar
- Taylor CC, Pizlo Z, Allebach JP, Bouman CA: Image quality assessment with a Gabor pyramid model of the human visual system. SPIE Conference on Human Vision and Electronic Imaging, San Jose, CA 1997.Google Scholar
- Seshadrinathan K, Bovik AC: Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 2010, 19(2):335-350.View ArticleMathSciNetGoogle Scholar
- Lee S, Pattichis MS, Bovik AC: Foveated video quality assessment. IEEE Trans Multimedia 2002, 4(1):129-132. 10.1109/6046.985561View ArticleGoogle Scholar
- Wang Z, Bovik AC, Lu L, Kouloheris J: Foveated wavelet image quality index. SPIE's 46th Annual Meeting, Proceedings of SPIE, Application of digital image processing 2001, XXIV: 4472.Google Scholar
- Cormack LK: Computational models of early human vision. In The Handbook of Image and Video Processing. Edited by: Bovik AC. Academic Press; 2000.Google Scholar
- Mark F: Color Appearance Models. Addison-Wesley, Boston; 1998:7.Google Scholar
- Sheikh HR, Wang Z, Cormack LK, Bovik AC: LIVE image quality assessment database. Release 2 [http://live.ece.utexas.edu/research/quality/subjective.htm]
- BID--blurred image database[http://www.lps.ufrj.br/profs/eduardo/ImageDatabase.htm]
- Wang Z, Wu G, Sheikh HR, Simoncelli EP, Yang EH, Bovik AC: Quality-aware images. IEEE Trans Image Process 2006, 15(5):1680-1689.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.