An adaptive bandwidth nonlocal means image denoising in wavelet domain
© You and Cho; licensee Springer. 2013
Received: 20 February 2013
Accepted: 4 October 2013
Published: 5 November 2013
This paper proposes a new wavelet domain denoising algorithm. In the results of conventional wavelet domain denoising methods, ringing artifacts or wavelet-shaped noises are sometimes observed due to thresholding of small but important coefficients or due to generation of large coefficients in flat areas. In this paper, nonlocal means filtering is applied to each subband of wavelet decomposition, which can keep small coefficients and does not generate unwanted large coefficients. Since the performance of nonlocal means filtering depends on the appropriate kernel bandwidth, we also propose a method to find global and local kernel bandwidth for each subband. In comparison with conventional methods, the proposed method shows lower PSNR than BM3D when pseudo white Gaussian noise is added, but higher PSNR than the spatial nonlocal means filtering and wavelet thresholding methods. For the mixture noise or Poisson noise, which may better explain the real noise from camera sensors, the proposed method shows better or comparable results than the state-of-the-art methods. Also, it is believed that the proposed method shows better subjective quality for the noisy images captured in the low-illumination conditions.
Denoising is one of the fundamental image processing problems and thus has been studied for a long time. To name a few of the existing methods that are related with our work and the state-of-the-art methods, there are wavelet shrinkage methods [1, 2], a total variation minimization , a prior probability modeling , nonlocal means filtering , and BM3D . Among these, the BM3D generally shows the highest PSNR when the noise is additive white Gaussian.
In the case of wavelet domain thresholding methods [1, 2], an image is transformed into the wavelet domain, and the coefficients in each subband are suppressed by hard or soft thresholding. The advantage of wavelet shrinkage methods is that they require not much computations while providing pleasing results. The probabilistic wavelet coefficient modeling method  fits the neighborhoods of coefficients as Gaussian scale mixture (GSM) model and applies the Bayesian least squares (BLS) technique to adjust the coefficients. Although wavelet shrinkage methods and BLS-GSM provide relatively high PSNR improvement, shrinking or modifying wavelet coefficients sometimes bring ringing or wavelet-shaped artifacts. For example, wavelet transformation of a step edge generates small coefficients up to the highest subbands. Hence, when the small coefficients are removed by thresholding and are inverse transformed, then ringing artifacts arise due to the loss of high frequencies. In the case of probabilistic wavelet coefficient modeling, unwanted coefficients can be generated in the homogeneous region, which result in wavelet-shaped artifacts in the spatial domain. Another popular denoising method is the nonlocal means filtering , which substitutes a noisy pixel by the weighted sum of neighborhood pixels. The weights are determined based on the kernel density estimation, which can be regarded as a Nadaraya-Watson estimator, i.e., a kind of local constant regression . In other words, the smooth kernel estimate in the nonlocal means approach is a sum of bumps placed on the data points. The kernel function determines the shape of the bumps, and the ‘smoothing parameter’ or ‘bandwidth’ controls the degree of smoothness. In , an automatic bandwidth selection method was proposed based on the reduction of entropy of image patterns, and the global bandwidth was applied to the overall area of image. However, it is noted that narrower kernels are suitable for the complex regions, whereas larger kernels would be better for more sparse areas. Hence, it is important to find an appropriate bandwidth according to the local characteristics, which is not an easy task. One of the main factors that strongly influence the local properties of the image is the noise statistics in the neighborhood, and thus, the bandwidth needs to be adaptively determined according to the local noise variance. In summary, we need to estimate the local noise statistics for finding an appropriate bandwidth for the given region. There are many methods for estimating the variance of white additive noise in images, but they cannot be used for the images with non-uniform noise variance. In this consideration, the estimation of local noise statistics is necessary to find the appropriate bandwidth for the given area.
In this paper, inspired by the performance of nonlocal means filtering method in keeping the structures of the image while suppressing the noise, we attempt to apply the nonlocal means filter to the wavelet coefficients. The wavelet coefficients contain the information on the structures of the image, which have different but related characteristics depending on the subbands. This property has been extensively and effectively exploited in many image processing applications, including denoising. However, as stated previously, manipulation of wavelet coefficients sometimes brings ringing artifacts and wavelet-shaped noise. Hence, instead of thresholding or generating the wavelet coefficients, we filter the coefficients based on the nonlocal means approach. This approach keeps small but important wavelet coefficients which would have been thresholded in the conventional schemes and also does not generate large coefficients in homogeneous regions while effectively suppressing noisy ones. In applying the nonlocal means filter, determining the bandwidth is also an important factor for successful filtering. Hence, we also propose a method that gives different bandwidths to each subband and region, depending on its properties and noise statistics.
The experiments are conducted with various types of pseudo noises and also with real noise that is observed in the images taken in low-illumination conditions. It is shown that the proposed method gives lower PSNR than the state-of-the-art methods such as BM3D  and BLS-GSM  when the white Gaussian noise is added. However, it gives higher PSNR than the conventional wavelet shrinkage methods and the spatial nonlocal means filtering method. Also, it gives higher PSNR than BM3D when the noise is a mixture of Gaussian and impulsive noises and when the noise model is Poisson which better explains the real noise from CCD/CMOS sensors . For the experiments with real noise, images taken under low-illumination conditions and film images are denoised by various denoising methods. Subjective comparison shows that the wavelet domain nonlocal means filtering provides competitive results for real noises, which supports the simulation results with non-Gaussian noises.
The rest of this paper is organized as follows. In the second section, we review the nonlocal means filter and its bandwidth parameter estimation. In the third section, we propose the extension of nonlocal means filter to the wavelet domain denoising with the bandwidth selection method. Then, we show some experimental results on the images degraded by various pseudo noise and the images with real noise. The last section concludes this paper.
Nonlocal means filter
where Y i represents the vector of pixel intensities in N i , is a normalizing factor, and h is the smoothing kernel width which controls the degree of averaging. The denoised pixel is obtained by locally weighted averaging, which corresponds to the Nadaraya-Watson estimator . From Equation (1), it can be seen that a small h shrinks the area of averaging, and thus, the noise is not suppressed enough. Conversely, if h is too large, the weights at the boundary of S i are also very large, which results in a blurry output. In the conventional work , h is set between 10σ and 15σ, where the noise standard deviation σ is estimated from the image statistics.
where p KDE(x) is the kernel density estimate of true density p(x), at a point x. This shows that there is a tradeoff between the bias and variance, which also means that a large bandwidth reduces the variance of the estimator but increases the bias and vice versa.
There have been several approaches to bandwidth estimation , which include subjective choice based on the asymptotic mean squared error, cross-validation methods using pseudo-likelihood maximization by the leave-one-out criterion, and plug-in estimator based on the asymptotically best choice of h. Also, the existing methods can be categorized as global or local bandwidth adjustment, where the local adaptivity gives better performance but requires heavy computational burden.
Thus, instead of thresholding or making probabilistic decision, bilateral filtering of wavelet coefficients is shown to provide competitive results . In our previous work , we have also shown that the wavelet domain nonlocal means filter provides higher PSNR than the spatial domain nonlocal means filtering. In this paper, we improve the performance by finding the locally adaptive bandwidth for each subband and region, whereas the previous work applied global bandwidth. In addition, we test the algorithm for various kinds of noise model such as mixture noise or Poisson noise, which better explains the real camera noise. Also, it is tested on real noises that arise in the low-illumination conditions and film grain noise.
Wavelet domain nonlocal means filter with adaptive bandwidth
The main idea of our work is to apply the nonlocal means filtering to scaling and wavelet coefficients of an image to keep small but important coefficients which might have been shrunk in conventional wavelet denoising. Another contribution is the derivation of global and local bandwidth for the wavelet-domain nonlocal means filtering, according to the subband’s statistics. It is noted that each subband has different noise statistics which may also vary depending on the location in each band. Hence, for each subband, we first find a global bandwidth that can be applied to the overall subband, based on the plug-in method . Then, from Abramson’s rule  using the statistics derived in this process, we also find the locally adaptive bandwidth in each subband.
where i is the index for the pixel position (m,n), and X m,n is the wavelet coefficient at that position. The residual can be considered a prediction error of X(m,n) by its neighboring data, and the median operation over the residuals as Equation (7) gives approximated standard deviation of data. In summary, we estimate the noise standard deviation by Equation (7) for each subband and then the nonlocal means filter with the bandwidth in Equation (6) can be applied to the given subband.
where α l (i) denotes a noise-free coefficient, and ε l (i) denotes a random variable assumed to be . When there seems to be no confusion, we will drop the indexes and subscripts of the above notation in the rest of the paper.
which is a simple shrinkage rule for the given wavelet coefficient under the hypothesis test. We use as a measure to decide the noisy coefficients.
where is the incomplete gamma function.
Algorithm 1 Summary of wavelet domain denoising
Summarizing the experimental results in advance, the proposed method yields higher PSNR than the spatial nonlocal means filtering and conventional wavelet shrinkage methods. However, it shows lower PSNR than the BM3D for the denoising of images corrupted by additive pseudo white Gaussian noise. A recent wavelet domain approach, BLS-GSM, also gives quite high PSNR, slightly less than BM3D. However, for more complex noise models such as mixture of impulsive and Gaussian, uniform noise, or Poisson noise that better explains the real camera noise, the proposed method yields higher PSNR and better subjective quality than the above referenced methods. Also, it is believed that the proposed method yields subjectively better output for the real noise, especially the noises that are often observed in the images taken under low-illumination conditions.
Denoising performances (dB) of various methods for the pseudo white Gaussian noise
Denoising performances (dB) of various methods for the mixed noise, additive non-Gaussian noise, and Poisson noise
20% impulse noise
20% impulse noise + Gaussian (σ = 10)
10% impulse noise + Gaussian (σ = 10)
10% impulse noise + 10% uniform noise
Poisson noise, σ = 2
Poisson noise, σ = 2
Poisson noise, σ = 2
Finally, we have also performed experiments changing the decomposition levels and kind of wavelet filters. About the decomposition level, the maximum PSNR was attained for the 1-level decomposition as in our experiment. Further decomposition does not improve the gain because the lower- and mid-band images become small, and thus, there are not enough patches to be used for smoothing the data. In the case of experiments on different kinds of wavelet filters, there was not much difference in the PSNR gain. It is believed that the denoising effect is not much affected by the shape of the coefficients which differs depending on the given wavelet filters as long as the coefficients are effectively denoised.
We have proposed a new image denoising algorithm based on nonlocal means filtering in the wavelet domain. By the nonlocal means filtering, small wavelet coefficients that constitute an important image structure are well preserved, while noisy coefficients are suppressed. Since the local adaptation of the kernel bandwidth gives better results, we have also proposed a method to find the appropriate kernel bandwidth to each region for the effective nonlocal means filtering. As a result, the proposed method provides comparable or sometimes higher PSNR than the state-of-the-art algorithms. Also, subjective comparisons show that the proposed method keeps the structures of the images very well and gives less ringing artifacts.
We appreciate anonymous reviewers for their valuable comments and helpful suggestions. This research was supported by Samsung Electronics and supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and Future Planning (2009-0083495).
- Donoho DL, Johnstone IM: Ideal spatial adaptation by wavelet shrinkage. Biometrika 1994, 81: 425-445. 10.1093/biomet/81.3.425MathSciNetView ArticleMATHGoogle Scholar
- Chang SG, Yu B, Vetterli M: Adaptive wavelet thresholding for image denoising and compression. IEEE Trans. Image Process 2000, 9(9):1532-1546. 10.1109/83.862633MathSciNetView ArticleMATHGoogle Scholar
- Rudin LI, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Phys. D 1992, 60: 259-268. 10.1016/0167-2789(92)90242-FView ArticleMathSciNetMATHGoogle Scholar
- Portilla J, Strela V, Wainwright M, Simoncelli E: Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans. Image Process 2003, 12(11):1338-1351. 10.1109/TIP.2003.818640MathSciNetView ArticleMATHGoogle Scholar
- Buades A, Coll B, Morel JM: A non-local algorithm for image denoising. CVPR 2005, 2: 60-65.MATHGoogle Scholar
- Dabov K, Foi A, Katkovnik V: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Proc 2007, 16(8):2080-2095.MathSciNetView ArticleGoogle Scholar
- Wand M, Jones M: Kernel Smoothing. Chapman and Hall, London, UK; 1995.View ArticleMATHGoogle Scholar
- Awate SP, Whitakerl RT: Unsupervised, information-theoretic, adaptive image filtering for image restoration. IEEE Trans. Pattern Anal. Mach. Intell 2006, 28(3):364-376.View ArticleGoogle Scholar
- Alter F, Matsushita Y, Tang X: An intensity similarity measure in low-light conditions. In Proceeding of European Conference on Computer Vision. Edited by: Leonardis A, Bischof H, Pinz A. Springer, Berlin; 2006:267-280.Google Scholar
- Chu CK, Marron JS: Choosing a kernel regression estimator. Stat. Sci 1992, 6: 404-436.MathSciNetView ArticleMATHGoogle Scholar
- Jones MC, Marron JS, Sheather SJ: A brief survey of bandwidth selection for density estimation. J. Am. Stat. Assoc 1996, 91(3):401-407.MathSciNetView ArticleMATHGoogle Scholar
- Zhang M, Gunturk BK: Multiresolution bilateral filtering for image denoising. IEEE Trans. Image Process 2008, 17(12):2324-2333.MathSciNetView ArticleGoogle Scholar
- You SJ, Cho NI: A new image denoising method based on the wavelet domain nonlocal means filtering. In Proceeding of the International Conference on Acoustics, Speech, and Signal Processing. Czech Republic, Prague; 22–27 May 2011.Google Scholar
- Sheather SJ, Jones MC: A reliable data-based bandwidth selection method for kernel density estimation. J. R. Stat. Soc. Series B 1991, 53(3):683-690.MathSciNetMATHGoogle Scholar
- Abramson I: On bandwidth variation in kernel estimates—a square root law. Ann. Stat 1982, 10: 1217-1223. 10.1214/aos/1176345986MathSciNetView ArticleMATHGoogle Scholar
- Silverman BW: Monographs on Statistics and Applied Probability: Density Estimation for Statistics and Data Analysis. Chapman and Hall, London);Google Scholar
- Black MJ, Sapiro G: Edges as outliers: anisotropic smoothing using local image statistics. In Scale-Space Theories in Computer Vision, Second Int. Conf.. Edited by: Nielsen M, Johansen P, Olsen OF, Weickert J. (Springer, Berlin; 1999:259-270.View ArticleGoogle Scholar
- Pizurica A, Philips W: Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising. IEEE Trans. Image Process 2006, 15(3):654-665.View ArticleGoogle Scholar
- Simoncelli EP, Adelson EH: Noise removal via Bayesian wavelet coring. Int. Con. Image Proc 1996, 1: 379-382.Google Scholar
- Hirakawa K, Parks TW: Joint demosaicing and denoising. IEEE Trans. Image Proc 2006, 15(8):2146-2157.View ArticleGoogle Scholar
- Deledalle C, Tupin F, Denis L: Poisson NL means: unsupervised non local means for Poisson noise. In Proceeding of the International Conference on Image Processing. Hong Kong; 26–29 Sept 2010.Google Scholar
- Luisier F, Blue T, Parks TW: Image denoising in mixed poisson-Gaussian noise. IEEE Trans. Image Proc 2011, 20(3):696-708.View ArticleMathSciNetGoogle Scholar
- Liu C: The image database. . Accessed Jan 2010 http://people.csail.mit.edu/celiu/
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.