An adaptive bandwidth nonlocal means image denoising in wavelet domain

Abstract

This paper proposes a new wavelet domain denoising algorithm. In the results of conventional wavelet domain denoising methods, ringing artifacts or wavelet-shaped noises are sometimes observed due to thresholding of small but important coefficients or due to generation of large coefficients in flat areas. In this paper, nonlocal means filtering is applied to each subband of wavelet decomposition, which can keep small coefficients and does not generate unwanted large coefficients. Since the performance of nonlocal means filtering depends on the appropriate kernel bandwidth, we also propose a method to find global and local kernel bandwidth for each subband. In comparison with conventional methods, the proposed method shows lower PSNR than BM3D when pseudo white Gaussian noise is added, but higher PSNR than the spatial nonlocal means filtering and wavelet thresholding methods. For the mixture noise or Poisson noise, which may better explain the real noise from camera sensors, the proposed method shows better or comparable results than the state-of-the-art methods. Also, it is believed that the proposed method shows better subjective quality for the noisy images captured in the low-illumination conditions.

Introduction

Denoising is one of the fundamental image processing problems and thus has been studied for a long time. To name a few of the existing methods that are related with our work and the state-of-the-art methods, there are wavelet shrinkage methods [1, 2], a total variation minimization [3], a prior probability modeling [4], nonlocal means filtering [5], and BM3D [6]. Among these, the BM3D generally shows the highest PSNR when the noise is additive white Gaussian.

In the case of wavelet domain thresholding methods [1, 2], an image is transformed into the wavelet domain, and the coefficients in each subband are suppressed by hard or soft thresholding. The advantage of wavelet shrinkage methods is that they require not much computations while providing pleasing results. The probabilistic wavelet coefficient modeling method [4] fits the neighborhoods of coefficients as Gaussian scale mixture (GSM) model and applies the Bayesian least squares (BLS) technique to adjust the coefficients. Although wavelet shrinkage methods and BLS-GSM provide relatively high PSNR improvement, shrinking or modifying wavelet coefficients sometimes bring ringing or wavelet-shaped artifacts. For example, wavelet transformation of a step edge generates small coefficients up to the highest subbands. Hence, when the small coefficients are removed by thresholding and are inverse transformed, then ringing artifacts arise due to the loss of high frequencies. In the case of probabilistic wavelet coefficient modeling, unwanted coefficients can be generated in the homogeneous region, which result in wavelet-shaped artifacts in the spatial domain. Another popular denoising method is the nonlocal means filtering [5], which substitutes a noisy pixel by the weighted sum of neighborhood pixels. The weights are determined based on the kernel density estimation, which can be regarded as a Nadaraya-Watson estimator, i.e., a kind of local constant regression [7]. In other words, the smooth kernel estimate in the nonlocal means approach is a sum of bumps placed on the data points. The kernel function determines the shape of the bumps, and the ‘smoothing parameter’ or ‘bandwidth’ controls the degree of smoothness. In [8], an automatic bandwidth selection method was proposed based on the reduction of entropy of image patterns, and the global bandwidth was applied to the overall area of image. However, it is noted that narrower kernels are suitable for the complex regions, whereas larger kernels would be better for more sparse areas. Hence, it is important to find an appropriate bandwidth according to the local characteristics, which is not an easy task. One of the main factors that strongly influence the local properties of the image is the noise statistics in the neighborhood, and thus, the bandwidth needs to be adaptively determined according to the local noise variance. In summary, we need to estimate the local noise statistics for finding an appropriate bandwidth for the given region. There are many methods for estimating the variance of white additive noise in images, but they cannot be used for the images with non-uniform noise variance. In this consideration, the estimation of local noise statistics is necessary to find the appropriate bandwidth for the given area.

In this paper, inspired by the performance of nonlocal means filtering method in keeping the structures of the image while suppressing the noise, we attempt to apply the nonlocal means filter to the wavelet coefficients. The wavelet coefficients contain the information on the structures of the image, which have different but related characteristics depending on the subbands. This property has been extensively and effectively exploited in many image processing applications, including denoising. However, as stated previously, manipulation of wavelet coefficients sometimes brings ringing artifacts and wavelet-shaped noise. Hence, instead of thresholding or generating the wavelet coefficients, we filter the coefficients based on the nonlocal means approach. This approach keeps small but important wavelet coefficients which would have been thresholded in the conventional schemes and also does not generate large coefficients in homogeneous regions while effectively suppressing noisy ones. In applying the nonlocal means filter, determining the bandwidth is also an important factor for successful filtering. Hence, we also propose a method that gives different bandwidths to each subband and region, depending on its properties and noise statistics.

The experiments are conducted with various types of pseudo noises and also with real noise that is observed in the images taken in low-illumination conditions. It is shown that the proposed method gives lower PSNR than the state-of-the-art methods such as BM3D [6] and BLS-GSM [4] when the white Gaussian noise is added. However, it gives higher PSNR than the conventional wavelet shrinkage methods and the spatial nonlocal means filtering method. Also, it gives higher PSNR than BM3D when the noise is a mixture of Gaussian and impulsive noises and when the noise model is Poisson which better explains the real noise from CCD/CMOS sensors [9]. For the experiments with real noise, images taken under low-illumination conditions and film images are denoised by various denoising methods. Subjective comparison shows that the wavelet domain nonlocal means filtering provides competitive results for real noises, which supports the simulation results with non-Gaussian noises.

The rest of this paper is organized as follows. In the second section, we review the nonlocal means filter and its bandwidth parameter estimation. In the third section, we propose the extension of nonlocal means filter to the wavelet domain denoising with the bandwidth selection method. Then, we show some experimental results on the images degraded by various pseudo noise and the images with real noise. The last section concludes this paper.

Related works

Nonlocal means filter

Let us denote a noisy observation of an image as y(i) = u(i) + n(i), where y(i), u(i), and n(i) are the noisy observation, original image, and the noise, respectively, at the i th pixel. Also, we define N _i  and S _i  as a square neighborhood and a square search window centered at the pixel i, respectively. Then, the nonlocal means filter can be described as [5]

$$û\left(i\right)=\sum _{j\in S_i}\frac{1}{Z\left(i\right)}{e}^{-\frac{\parallel {\mathbf{Y}}_i-{\mathbf{Y}}_j{\parallel }^2}{h^2}}y\left(j\right),$$

(1)

where Y _i  represents the vector of pixel intensities in N _i , $Z\left(i\right)=\sum _{j\in S_i}{e}^{-\frac{\parallel {\mathbf{Y}}_i-{\mathbf{Y}}_j{\parallel }^2}{h^2}}$ is a normalizing factor, and h is the smoothing kernel width which controls the degree of averaging. The denoised pixel $û\left(i\right)$ is obtained by locally weighted averaging, which corresponds to the Nadaraya-Watson estimator [10]. From Equation (1), it can be seen that a small h shrinks the area of averaging, and thus, the noise is not suppressed enough. Conversely, if h is too large, the weights at the boundary of S _i  are also very large, which results in a blurry output. In the conventional work [5], h is set between 10σ and 15σ, where the noise standard deviation σ is estimated from the image statistics.

Bandwidth selection

Choosing an appropriate bandwidth is thus very important for the balanced nonlocal means filtering. Traditionally, the bandwidth h is selected to minimize the error between the estimate and true density. For this purpose, the mean square error (MSE) at a point x is defined as [7]

$$\begin{array}{ll}{\text{MSE}}_x\left(p_{\text{KDE}}\right)& =E\left[{\left(p_{\text{KDE}}\right(x)-p(x\left)\right)}^2\right]\\ ={E\left[p_{\text{KDE}}\right(x)-p(x\left)\right]}^2+\mathit{\text{var}}\left(p_{\text{KDE}}\right(x\left)\right),\end{array}$$

(2)

where p _KDE(x) is the kernel density estimate of true density p(x), at a point x. This shows that there is a tradeoff between the bias and variance, which also means that a large bandwidth reduces the variance of the estimator but increases the bias and vice versa.

There have been several approaches to bandwidth estimation [11], which include subjective choice based on the asymptotic mean squared error, cross-validation methods using pseudo-likelihood maximization by the leave-one-out criterion, and plug-in estimator based on the asymptotically best choice of h. Also, the existing methods can be categorized as global or local bandwidth adjustment, where the local adaptivity gives better performance but requires heavy computational burden.

Wavelet denoising

The wavelet transform has an excellent localization property and thus shown to be effective in many image processing applications. Since the work of Donoho and Johnstone [1], there have been a lot of researches on wavelet shrinkage method. These wavelet denoising methods suppress the noisy coefficient magnitudes while keeping the local structures. Ideally, only the wavelet coefficients that correspond to the noise component should be removed, whereas the coefficients containing a significant structure component should be less reduced. Figure 1 shows the comparison of wavelet coefficients of noisy and noiseless images, where it can be seen that the small coefficients appear in the subbands of noisy image. Hence, one of the popular wavelet domain denoising methods is to shrink the coefficients by thresholding [1, 2], i.e., the coefficients under a certain magnitude are treated as nonsignificant and are set to zero, while the remaining significant ones are kept unmodified (hard-thresholding) or their magnitudes are reduced (soft-thresholding). Unfortunately, the edges often generate small wavelet coefficients up to the highest bands, along with the significant ones. Hence, the suppression of the small coefficients around the significant ones results in ringing artifacts, i.e., Gibb’s phenomena.

Figure 1

Noise in the wavelet domain. Wavelet coefficients of the vertical subband in a noisy image and a noise-free image.

Thus, instead of thresholding or making probabilistic decision, bilateral filtering of wavelet coefficients is shown to provide competitive results [12]. In our previous work [13], we have also shown that the wavelet domain nonlocal means filter provides higher PSNR than the spatial domain nonlocal means filtering. In this paper, we improve the performance by finding the locally adaptive bandwidth for each subband and region, whereas the previous work applied global bandwidth. In addition, we test the algorithm for various kinds of noise model such as mixture noise or Poisson noise, which better explains the real camera noise. Also, it is tested on real noises that arise in the low-illumination conditions and film grain noise.

Wavelet domain nonlocal means filter with adaptive bandwidth

The main idea of our work is to apply the nonlocal means filtering to scaling and wavelet coefficients of an image to keep small but important coefficients which might have been shrunk in conventional wavelet denoising. Another contribution is the derivation of global and local bandwidth for the wavelet-domain nonlocal means filtering, according to the subband’s statistics. It is noted that each subband has different noise statistics which may also vary depending on the location in each band. Hence, for each subband, we first find a global bandwidth that can be applied to the overall subband, based on the plug-in method [14]. Then, from Abramson’s rule [15] using the statistics derived in this process, we also find the locally adaptive bandwidth in each subband.

In kernel density estimation, the estimated density at any point x is formulated as

$${\widehat{f}}_h\left(x\right)=\frac{1}{n}\sum _{i=1}^{n}K\left(\frac{x-x\left(i\right)}{h}\right),$$

(3)

where x(i) is a neighboring point to x, n is the number of neighbors, K(·) is the kernel function, and h is its bandwidth. The kernel function can be considered a weighting factor that gives larger value when x(i) is close to x, and h is also called the smoothing constant. A typical shape of kernel function is Gaussian, and the bandwidth h determines its width and thus smoothing factor in estimating the kernel density. To obtain the globally optimal bandwidth for the given data, denoted as h _go, the conventional method is to find the h that minimizes the mean integrated squared error (MISE) between ${\widehat{f}}_h\left(x\right)$ defined above and the true but unknown density f(x) as [7]

$$h_{\text{go}}=\underset{h}{\text{argmin}}\text{MISE}\left(\widehat{f_h}\right)=\underset{h}{\text{argmin}}\int E{\left\{\widehat{f_h}\right(x)-f(x\left)\right\}}^2\mathrm{dx.}$$

(4)

Since the plug-in approach [14] is known to be one of the best data-driven bandwidth selection methods, we employ this approach to minimize the MISE, and we obtain the bandwidth

$$h_{\text{go}}={\left(\frac{\parallel K{\parallel }^2}{\parallel f^{\prime \prime }{\parallel }^2{\left\{{\mu }_2\right(K\left)\right\}}^{2}M}\right)}^{1/5},$$

(5)

where $\parallel K{\parallel }^2=\int K^2\left(x\right)\mathit{\text{dx}}$ and ${\mu }_2\left(K\right)=\int x^{2}K\left(x\right)\mathit{\text{dx}}$ are constants depending on the kernel function, and M is the number of sample data in the subband. Note that ∥f^′′∥² is the only unknown term in Equation (5), and the idea behind the plug-in approach is to replace f^′′ by an estimate from the data. Silverman’s rule of thumb [16] computes f^′′ as if f had the density of the normal distribution N(μ,σ²) and then the optimal global bandwidth for the subband can be approximated as

$$h_{\text{go}}={\left(\frac{4{\widehat{\sigma }}^5}{3M}\right)}^{1/5}\cong 1.06\widehat{\sigma }{M}^{-1/5},$$

(6)

where $\widehat{\sigma }$ is the standard deviation of the noise to be estimated. To obtain $\widehat{\sigma }$, we employ the empirical preliminary estimation from each subband’s wavelet coefficients as introduced in [17]:

$$\widehat{\sigma }=1.4826\mathbf{\text{med}}(\parallel \mathbf{r}-\mathbf{\text{med}}(\parallel \mathbf{r}\parallel )\parallel ),$$

(7)

where r = {r ₁,r ₂,…,r _∣X∣} is the set of residuals of the entire wavelet coefficients in the subband, and |X| is the total number of coefficients in the subband. The residual r _i  is defined as

$$r_i=\frac{2X_{m,n}-(X_{m+1,n}+X_{m,n+1})}{\sqrt{6}},$$

(8)

where i is the index for the pixel position (m,n), and X _m,n  is the wavelet coefficient at that position. The residual can be considered a prediction error of X(m,n) by its neighboring data, and the median operation over the residuals as Equation (7) gives approximated standard deviation of data. In summary, we estimate the noise standard deviation by Equation (7) for each subband and then the nonlocal means filter with the bandwidth in Equation (6) can be applied to the given subband.

In addition to the above global characteristics, the consideration of local statistics brings better denoising results. That is, applying locally adaptive bandwidth yields better result than applying the above h _go to the overall subband. For the derivation of locally adaptive bandwidth, let us denote the i th wavelet coefficient in the l th subband as

$$X_l\left(i\right)={\alpha }_l\left(i\right)+{\epsilon }_l\left(i\right),$$

(9)

where α _l (i) denotes a noise-free coefficient, and ε _l (i) denotes a random variable assumed to be $N(0,{\sigma }_{{\epsilon }_l}^2(i\left)\right)$. When there seems to be no confusion, we will drop the indexes and subscripts of the above notation in the rest of the paper.

The main step for the locally adaptive filtering is to estimate the local noise statistics ${\widehat{\sigma }}_{\text{lo}}$ from a set of L × L coefficients centered at a pixel of interest. This requires a hypothesis test for determining whether a coefficient X is a noise coefficient or not and then we compute the variance of the noise coefficients within the L × L window. Here, the hypothesis test follows the algorithm in probabilistic wavelet shrinkage [18]. To be precise, we first model the wavelet coefficient as a sample of generalized Gaussian random variable with the probability density function

$$f_G(\alpha )=\frac{\lambda ·\nu }{2\Gamma \left(\frac{1}{\nu }\right)}\text{exp}(-\lambda |\alpha {|}^{\nu }),$$

(10)

where $\Gamma \left(z\right)=\underset{0}{\overset{\infty }{\int }}{t}^{z-1}{e}^{-t}\mathit{\text{dt}},z>0$ is the Gamma function, λ > 0 is the scale parameter, and ν is the shape parameter. To test whether a given X is a noise coefficient or not, we test its significance by binary hypothesis test: H ₀ is the hypothesis that X is a noise coefficient, and H ₁ is a significant one. To assess the hypothesis, we use the Bayes’ rule, where it is assumed that a prior is known, and its parameters are random variables. The Bayes’ rule produces the conditional probability P(H ₁|X) = μ η/(1 + μ η), where μ = P(H ₁)/P(H ₀), η = f _G (X|H ₁)/f _G (X|H ₀), and the product μ η defines the generalized likelihood ratio. From the assumption of generalized Gaussian prior and the Bayes’ rule, the estimate of the true coefficient is represented as

$$\widehat{\beta }=P\left(H_1\right|X)X=\frac{\mu \eta }{1+\mu \eta }X$$

(11)

which is a simple shrinkage rule for the given wavelet coefficient under the hypothesis test. We use $P\left(H_1\right|X)=\frac{\mu \eta }{1+\mu \eta }$ as a measure to decide the noisy coefficients.

For details of calculating the parameters above, the conditional densities of the noisy coefficients f _G (X|H ₀) and the noise-free coefficients f _G (X|H ₁) are defined as the following convolutions [18]:

$$\begin{array}{l}{f}_G\left(X\right|H_0)=\underset{-\infty }{\overset{\infty }{\int }}\varphi (X-\alpha ;\sigma \left)f_G\right(\alpha \left|H_0\right)d\alpha \\ f_G\left(X\right|H_1)=\underset{-\infty }{\overset{\infty }{\int }}\varphi (X-\alpha ;\sigma \left)f_G\right(\alpha \left|H_1\right)\mathrm{d\alpha },\end{array}$$

(12)

where ϕ(X;σ) is the zero mean Gaussian density, and the standard deviation σ is computed as Equation (7). The hypothesis test is to define an element as a significant one when it is larger than a threshold and vice versa, and thus the conditional densities are defined as

$$f_G(\alpha |H_0)=\left\{\begin{array}{l}{B}_0\text{exp}(-\lambda |\alpha {|}^{\nu }),\text{if}|\alpha |\le T_{\alpha }\\ 0,\text{if}|\alpha |>T_{\alpha }\end{array}\right.$$

(13)

and

$$f_G(\alpha |H_1)=\left\{\begin{array}{l}0,\text{if}|\alpha |\le T_{\alpha }\\ B_1\text{exp}(-\lambda |\alpha {|}^{\nu }),\text{if}|\alpha |>T_{\alpha },\end{array}\right.$$

(14)

where $T_{\alpha }=\widehat{\sigma }$ is the threshold, and B ₀ and B ₁ are the normalizing constants as

$$\begin{array}{l}{B}_0={\left(\underset{-T}{\overset{T}{\int }}\text{exp}(-|\lambda \alpha {|}^{\nu })d\alpha \right)}^{-1}\text{and}\\ B_1={\left(2\underset{T}{\overset{\infty }{\int }}\text{exp}(-|\lambda \alpha {|}^{\nu })d\alpha \right)}^{-1}.\end{array}$$

The following steps are also from wavelet shrinkage method [18], which is repeated here for convenience. The parameters λ and ν are determined by the noise statistics in each band. To be precise, let σ _X  be the variance of the overall coefficients in the subband, m _4,X  be their fourth moment, and σ _α  be the standard deviation of noise-free coefficients in the band. Then, from [19], ν and λ are found from the equations as follows:

$$\begin{array}{l}\frac{\Gamma \left(\frac{1}{\nu }\right)\Gamma \left(\frac{5}{\nu }\right)}{{\Gamma }^2\left(\frac{3}{\nu }\right)}=\frac{m_{4,X}+3{\sigma }_{\alpha }^4-6{\sigma }_{\alpha }^2{\sigma }_X^2}{{\sigma }_X^2-{\sigma }_{\alpha }^2},\text{and}\\ \lambda ={\left(({\sigma }_X^2-{\sigma }_{\alpha }^2)\frac{\Gamma \left(\frac{1}{\nu }\right)}{\Gamma \left(\frac{3}{\nu }\right)}\right)}^{-\frac{1}{2}}.\end{array}$$

(15)

From the left equation above, ν can be derived numerically, and it is used for computing λ from the right equation above. The next step is to compute P(H ₀) and P(H ₁) as [18]:

$$P\left(H_0\right)={\Gamma }_{\text{inc}}\left({(\lambda T)}^{\nu },\frac{1}{\nu }\right)\text{and}P\left(H_1\right)=1-P\left(H_0\right),$$

(16)

where ${\Gamma }_{\text{inc}}(x,a)=\frac{1}{\Gamma \left(a\right)}\underset{0}{\overset{x}{\int }}{t}^{a-1}\stackrel{-t}{\text{exp}}\mathit{\text{dt}}$ is the incomplete gamma function.

From the parameters derived above, we can compute $P\left(H_1\right|X)=\frac{\mu \eta }{1+\mu \eta }$ in Equation (11) and test whether this is above a certain threshold T _α . If P(H ₁|X) < T _α , then the corresponding coefficient is considered a noisy one. For all the coefficients that are determined to be noisy in the L×L window, the local noise variance ${\widehat{\sigma }}_{\text{lo}}$ is estimated, and in the same manner as h _go in Equation (6) is derived, the local bandwidth is determined as

$$h_{\text{lo}}\left(X_i\right)\cong 1.06{\widehat{\sigma }}_{\text{lo}}{L}_{\text{lo}}^{-1/5},$$

(17)

where L _lo is the number of noise coefficients in the L × L window centered at X _i . In summary, Figure 2 shows the block diagram of overall process of the proposed method, and the algorithm is summarized as follows:

Figure 2

Block diagram of the proposed method.

Algorithm 1 Summary of wavelet domain denoising

Finally, it is worth mentioning that these procedures are applied regardless of noise statistics in our experiments, i.e., the same algorithm is applied to the images corrupted by Gaussian noise, Poisson noise, mixture noise, and real images without any modification. The hypothesis test that we adopt is based on the generalized Gaussian distribution of coefficients, whereas the Poisson noise or real noise may not meet this assumption. However, we found that almost all noises that we test show Gaussian statistics in the wavelet domain, especially in local patches that we process. Figure 3 shows some evidence for this, which shows the histogram of noise over the local patches. It can be seen that the histograms show generalized Gaussian shape unlike their histograms in the spatial domain. More precisely, Figure 3a is the histogram of Poisson noise, which shows some peaky distribution, whereas Figure 3b shows the distribution closer to Gaussian, although long tail remains. In [20], they also show some experiments that the camera noise has Gaussian shape in local areas even though the noise is actually signal dependent.

Figure 3

Histogram of noise signal in the spatial domain local patches of wavelet domain. Histograms for (a) Poisson noise in the spatial domain. (b) Poisson noise in the local patches of wavelet domain. (c) Real camera noise in the spatial domain. (d) Real camera noise in the local patches of wavelet domain.

Experimental results

Summarizing the experimental results in advance, the proposed method yields higher PSNR than the spatial nonlocal means filtering and conventional wavelet shrinkage methods. However, it shows lower PSNR than the BM3D for the denoising of images corrupted by additive pseudo white Gaussian noise. A recent wavelet domain approach, BLS-GSM, also gives quite high PSNR, slightly less than BM3D. However, for more complex noise models such as mixture of impulsive and Gaussian, uniform noise, or Poisson noise that better explains the real camera noise, the proposed method yields higher PSNR and better subjective quality than the above referenced methods. Also, it is believed that the proposed method yields subjectively better output for the real noise, especially the noises that are often observed in the images taken under low-illumination conditions.

In the implementation of the proposed method, Daubechie’s orthogonal wavelet is used for the subband decomposition, specifically db 8 filters in MATLAB is used for one-level multiresolution analysis. The window size centered at a wavelet coefficient is 21×21, and the threshold T _α  for classifying a noise coefficient is set as 0.3. In the experiments with the pseudo Gaussian noise, the images are artificially corrupted by the addition of noise with standard deviation, σ = 20 and 30. The proposed method is compared with several state-of-the-art algorithms, and the results are summarized in Tableف1. It can be seen that the BM3D provides the best PSNR for the white Gaussian noise, the proposed method gives the second best PSNR in Barbara, and the BLS-GSM shows the second best PSNR for the rest.

Table 1 Denoising performances (dB) of various methods for the pseudo white Gaussian noise

For the subjective comparison, Figure 4 shows cropped parts of Lena image, processed by several methods reference in this paper and the proposed method. It can be observed that the proposed method gives the least blurry output. Figure 5 shows another result that the proposed method yields less blurry output than the other methods that show higher PSNR. Figures 6 and 7 show that the result of proposed method has less artifacts that are commonly found in the conventional wavelet domain denoising. Specifically, Figure 6a shows the result of BLS-GSM, where many wavelet-like noise are observed (in the circles, and magnified in Figure 6b) due to the generation of large wavelet coefficients, whereas there is no such artifacts in Figure 6h. Figure 7d shows the ringing artifacts due to the shrinkage of wavelet coefficients in the conventional method, where the proposed method shows less artifacts.

Figure 4

Denoising results with Lena, σ = 30. Comparison of various methods for the Lena image. (a) Cropped image from Lena, σ = 30. (b) Nonlinear TV. (c) Nonlocal means. (d) ProbShrink. (e) Multiresolution bilateral. (f) BLS-GSM. (g) BM3D. (h) Proposed method.

Figure 5

Denoising results with Barbara, σ = 20. Comparison of various methods for the Barbara image. (a) Cropped image from Barbara, σ = 20. (b) Nonlinear TV. (c) Nonlocal means. (d) ProbShrink. (e) Multiresolution bilateral. (f) BLS-GSM. (g) BM3D. (h) Proposed method.

Figure 6

Denoising results with Lena, σ = 20. Comparison of ringing artifacts. (a) Denoising result by BLS-GSM. (b) Magnified region around the circles near the lip in (a). (c) Nonlinear TV. (d) Nonlocal means. (e) ProbShrink. (f) Multiresolution bilateral. (g) BM3D. (h) Proposed method.

Figure 7

Denoising results for Lena corrupted by additive Gaussian noise, σ = 30. Denoising result comparison of the cropped part of Lena (a) noisy image, σ = 30, (b) nonlinear TV [3], (c) nonlocal means filter [5], (d) ProbShrink [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], (g) BM3D [6], and (h) the proposed method.

Experiments for the mixture noise model and Poisson noise model, which might explain the real noise in the low-illumination condition [21], are also conducted. The results are summarized in Table 2, where it can be seen that the proposed method provides higher PSNR than the state-of-the-art methods in many cases, except for the Poisson-specific NL method in [21] in the case of Poisson noise corruption. Figures 8, 9, 10, 11, and 12 show the outputs by several methods for subjective comparison. Finally, the subjective comparisons for the real noisy images are shown in Figures 13, 14, 15, 16, and 17. From these results, it is believed that the proposed method effectively reduces the real noise that arises due to low-light conditions. Original-sized images of all the results in this paper and additional ones can be found in http://ispl.snu.ac.kr/sjyou21/project/denoising.

Table 2 Denoising performances (dB) of various methods for the mixed noise, additive non-Gaussian noise, and Poisson noise

Figure 8

Denoising results for Lena corrupted by mixture noise with 20% impulse noise and Gaussian noise, with σ = 10. Denoising result comparison of the mixture noise for Lena. (a) Lena corrupted by mixed noise (PSNR = 17.98 dB), (b) denoising by nonlinear TV (PSNR = 19.36 dB) [3], (c) denoising by nonlocal means filter (PSNR = 21.91 dB) [5], (d) denoising by ProbShrink method (PSNR = 22.05 dB) [18], (e) denoising by multiresolution bilateral (PSNR = 18.09 dB)[12], (f) denoising by BLS-GSM (PSNR = 22.06 dB) [4], (g) denoising by BM3D (PSNR = 25.27 dB) [6], (h) denoising by the proposed method (PSNR = 28.92 dB).

Figure 9

Denoising results for Boat corrupted by mixture noise with 10% impulse noise and Gaussian noise, with σ = 10. Denoising result comparison of the mixture noise for Boat. (a) Boat corrupted by mixed noise (PSNR = 18.14 dB), (b) denoising by nonlinear TV (PSNR = 25.78 dB) [3], (c) denoising by nonlocal means filter (PSNR = 21.90 dB) [5], (d) denoising by ProbShrink method (PSNR = 21.23 dB) [18], (e) denoising by multiresolution bilateral (PSNR = 18.24 dB) [12], (f) denoising by BLS-GSM (PSNR = 22.15 dB) [4], (g) denoising by BM3D (PSNR = 24.85 dB) [6], (h) denoising by the proposed method (PSNR = 27.85 dB).

Figure 10

Denoising results for Barbara corrupted by 20% impulse noise only. Denoising result comparison of impulse noise for Barbara. (a) Barbara corrupted by impulse noise (PSNR = 18.42 dB), (b) denoising by nonlinear TV (PSNR = 24.84 dB) [3], (c) denoising by nonlocal means filter (PSNR = 22.36 dB) [5], (d) denoising by ProbShrink method (PSNR = 21.15 dB) [18], (e) denoising by multiresolution bilateral (PSNR = 18.52 dB) [12], (f) denoising by BLS-GSM (PSNR = 22.14 dB) [4], (g) denoising by BM3D (PSNR = 25.50 dB) [6], (h) denoising by the proposed method (PSNR = 28.23 dB).

Figure 11

Denoising results for Lena corrupted by corrupted by Poisson noise σ = 2. Comparison of Poisson noise removal. (a) Lena corrupted by Poisson noise (PSNR = 16.33 dB), (b) denoising by MA filter (PSNR = 25.09 dB), (c) denoising by NLM (PSNR = 27.30 dB) [5], (d) denoising by the Poisson NL (PSNR = 29.38 dB) [21], (e) denoising by the PURE LET (PSNR = 27.47 dB) [22], and (f) denoising by the proposed method (PSNR = 28.54 dB).

Figure 12

Denoising results for Boat corrupted by Poisson noise σ = 2. Comparison of denoising results for the Poisson noise. (a) Boat corrupted by Poisson noise (PSNR = 15.95 dB), (b) denoising by MA filter (PSNR = 22.80 dB), (c) denoising by NLM (PSNR = 24.76 dB) [5], (d) denoising by the Poisson NL (PSNR = 26.70 dB) [21], (e) denoising by the PURE LET (PSNR = 26.16 dB)[22], and (f) Denoising by the proposed method (PSNR = 26.56 dB).

Figure 13

Denoising results for a real image. The results of (a) real noisy image, (b) nonlinear TV [3], (c) NLM [5], (d) ProShrink method [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], and (g) BM3D [6] contain more ringing artifacts and worse visual quality than that of the (h) proposed method.

Figure 14

Denoising results for another real image. The results of (a) real noisy image, (b) nonlinear TV [3], (c) NLM [5], (d) ProShrink method [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], and (g) BM3D [6] contain more ringing artifacts and worse visual quality than that of the (h) proposed method.

Figure 15

Denoising results for real images taken under low-light condition. The results of (a) real noisy image, (b) nonlinear TV [3], (c) NLM [5], (d) ProShrink method [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], and (g) BM3D [6] contain more ringing artifacts and worse visual quality than that of the (h) proposed method.

Figure 16

Denoising results for a real image from image database by Liu [23]. The results of (a) real noisy image, (b) nonlinear TV [3], (c) NLM [5], (d) ProShrink method [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], and (g) BM3D [6] contain more ringing artifacts and worse visual quality than that of the (h) proposed method.

Figure 17

Denoising results for a grain noisy image. The results of (a) real noisy image, (b) nonlinear TV [3], (c) NLM [5], (d) ProShrink method [18], (e) multiresolution bilateral [12], (f) BLS-GSM [4], and (g) BM3D [6] contain more ringing artifacts and worse visual quality than that of the (h) proposed method.

Finally, we have also performed experiments changing the decomposition levels and kind of wavelet filters. About the decomposition level, the maximum PSNR was attained for the 1-level decomposition as in our experiment. Further decomposition does not improve the gain because the lower- and mid-band images become small, and thus, there are not enough patches to be used for smoothing the data. In the case of experiments on different kinds of wavelet filters, there was not much difference in the PSNR gain. It is believed that the denoising effect is not much affected by the shape of the coefficients which differs depending on the given wavelet filters as long as the coefficients are effectively denoised.

Conclusion

We have proposed a new image denoising algorithm based on nonlocal means filtering in the wavelet domain. By the nonlocal means filtering, small wavelet coefficients that constitute an important image structure are well preserved, while noisy coefficients are suppressed. Since the local adaptation of the kernel bandwidth gives better results, we have also proposed a method to find the appropriate kernel bandwidth to each region for the effective nonlocal means filtering. As a result, the proposed method provides comparable or sometimes higher PSNR than the state-of-the-art algorithms. Also, subjective comparisons show that the proposed method keeps the structures of the images very well and gives less ringing artifacts.