Sparse Bayesian blind image deconvolution with parameter estimation

Abstract

In this article, we propose a novel blind image deconvolution method developed within the Bayesian framework. We concentrate on the restoration of blurred photographs taken by commercial cameras to show its effectiveness. The proposed method is based on a non-convex l _p quasi norm with 0<p<1 that is used for the image, and a total variation (TV) based prior that is utilized for the blur. Bayesian inference is carried out by utilizing bounds for both the image and blur priors using a majorization-minimization principle. Maximum a posteriori estimates of the unknown image, blur and model parameters are calculated. Experimental results (i.e., restorations of more than 30 blurred photographs) are presented to demonstrate the advantage of the proposed method compared to existing ones.

1 Introduction

Blind image deconvolution (BID) refers to the process of estimating both the original image and the blur from the degraded noisy image observation by using partial information about the imaging system. Blind image deconvolution algorithms represent a valuable tool that can be used for improving image quality without requiring complicated calibrations of the real-time image acquisition and processing system (i.e., medical imaging, video-conferencing, space exploration, x-ray imaging, etc.).

The blind image deconvolution problem is encountered in many different technical areas, such as astronomical imaging, remote sensing, microscopy, medical imaging, optics, super-resolution applications, and motion tracking applications among others (see, for example, [1–9]). Astronomical imaging is one of the primary applications of blind image deconvolution algorithms [1, 2]. Ground based imaging systems are subject to blurring due to the rapidly changing index of refractions of the atmosphere. Extraterrestrial observations of the Earth and the planets are degraded by motion blur as a result of slow camera shutter speeds relative to the rapid spacecraft motion. Blind image deconvolution is used for improving the quality of the Poisson distributed film grain noise present in the blurred X-rays, mammograms, and digital angiographic images. In such applications, many times, the blurring is unavoidable because the medical imaging systems limit the intensity (e.g., low X-ray intensity) of the incident radiation in order to protect the patient’s health [10]. In optics, blind image deconvolution is used to restore the original image from the degradation introduced by a microscope or any other optical instrument [6, 7]. The Hubble Space Telescope main mirror imperfections have provided an inordinate amount of images for the digital image processing community [1]. As a final example, in tracking applications the object being tracked might be blurred due to its speed or the motion of the camera. As a result, the track is lost with conventional tracking approaches and the application of blind restoration approaches can improve tracking results [9].

The standard formulation of the gray-scale image degradation model is given in matrix-vector form by

$$\mathbf{y}=\mathbf{H}\mathbf{x}+\mathbf{n},$$

(1)

where the N×1 vectors x, y, and n represent, respectively, the original image, the available noisy and blurred image, and the observation noise, and H represents the blurring matrix created from the blur point spread function h. The images are assumed to be of size m×n = N, and they are lexicographically ordered into N×1 vectors. Given y, the BID problem calls for finding estimates of x and H using prior knowledge on them.

A number of methods have been proposed to address BID (a recent literature review can be found in [11]). The most recent methods are based on a Bayesian framework, and have addressed the removal of the camera motion from blurred photographs [12–19]. In [12] the unknown image and blur were estimated in a two step process. In the first step the blur is estimated from the blurred photograph by regularizing the image gradients with a mixture of Gaussian distributions and by regularizing the blur with a mixture of exponential distributions. In the second step the image is estimated from the blurred photograph and the estimated blur by utilizing the Richardson-Lucy (RL) algorithm. Finally, the restored image is obtained by performing the histogram equalization (to that of the observed image) on the output of RL. A multi-scale approach is utilized for the algorithm implementation. The multi-scale approach consists of down-sampling the blurred photograph to number of low resolution images, and by utilizing the proposed algorithm iteratively to obtain the blur estimate at each resolution.

Estimating camera motion has also been investigated in [13, 14], where the unknown image and blur were estimated in a simultaneous fashion. Additionally, [14] concentrated on synthetic experiments where the performance of the algorithm was evaluated by the improvement in signal to noise ratio. Also, in [13, 14] the regularization parameters are not automatically estimated but rather heuristically chosen by the user at each iteration in order to yield an unknown image estimate with good visual quality. The major disadvantage of the methods proposed in [13, 14] compared to the method proposed in [12] is the lack of parameter estimation.

In this article, we extend our study in [20] by providing (1) a multi-scale based implementation of the algorithm which improves the quality of the obtained restored images, and (2) a complete comparison with many existing state of the art blind deconvolution methods. The proposed Bayesian algorithm for BID utilizes a variant of the non-convex l _p quasi norm based prior as the unknown image prior and the TV prior as the unknown blur prior. Furthermore, we utilize the Bayesian framework to provide the estimates for all model parameters. Finally, we evaluate the performance of the proposed algorithm and provide comparisons with [12–14, 16, 19] by restoring blurred photographs taken by commercial cameras.

This article is organized as follows. In Section 2 we provide the proposed Bayesian modeling of the BID problem. The Bayesian inference is presented in Section 2. In Section 2, we describe implementation details of the proposed algorithm. Experimental results are provided in Section 2 and conclusions are drawn in Section 2.

2 Bayesian modeling

As already discussed in the previous section, the observation noise is modeled as a zero mean white Gaussian random vector. Therefore, the observation model is defined as

$$\mathrm{p}(\mathbf{y}|\beta ,\mathbf{x},\mathbf{h})\propto {\beta }^{N/2}\text{exp}\left[-\frac{\beta }{2}\parallel \mathbf{y}-\mathbf{H}\mathbf{x}{\parallel }^2\right],$$

(2)

where β is the precision of the multivariate Gaussian distribution.

As the image prior we utilize a variant of the generalized Gaussian distribution, given by

$$\mathrm{p}(\mathbf{x}|\mathit{\alpha })=\frac{1}{Z_{\text{GG}}(\mathit{\alpha })}\text{exp}\left[-\sum _{d\in \mathrm{D}}{\alpha }_d\sum _i|{\Delta }_i^d(\mathbf{x}){|}^p\right],$$

(3)

where Z_GG(α) is the partition function, 0<p<1, α denotes the set {α _d } and d∈D={h,v,hh,vv,hv}. ${\Delta }_i^h(\mathbf{x})$ and ${\Delta }_i^v(\mathbf{x})$ correspond to, respectively, the horizontal and vertical first order differences, at pixel i, that is, ${\Delta }_i^h(\mathbf{x})=x_i-x_{l(i)}$ and ${\Delta }_i^v(\mathbf{x})=x_i-x_{a(i)}$, where l(i) and a(i) denote the nearest neighbors of i, to the left and above, respectively. The operators ${\Delta }_i^{\mathit{\text{hh}}}(\mathbf{x})$, ${\Delta }_i^{\mathit{\text{vv}}}(\mathbf{x})$, ${\Delta }_i^{\mathit{\text{hv}}}(\mathbf{x})$ correspond to, respectively, horizontal, vertical and horizontal-vertical second order differences, at pixel i.

In this study, similarly to [15, 21], we utilize a non-convex l _p quasi norm with 0<p<1 since the derivatives of blurry photographs are expected to be sparse. The distributions of the image derivatives often have heavier tails that are better modeled with the non-convex l _p quasi norm prior with 0<p<1 compared to the convex priors modeled with p = 1,2.

For reducing the complexity of the problem we assume that α _h  = α _v  = α and α _hh  = α _vv  = α _hv  = α/2. Additionally, similarly to [22], the partition function is approximated as $Z_{\text{GG}}(\alpha )\propto {\alpha }^{-{\lambda }_{1}N/p}$, where λ₁ is a positive real number. We then simplify (3) accordingly to obtain the following image prior

$$\mathrm{p}(\mathbf{x}|\alpha )\propto {\alpha }^{{\lambda }_{1}N/p}\text{exp}\left[-\alpha \sum _{d\in \mathrm{D}}{2}^{1-o(d)}\sum _{i=1}^N|{\Delta }_i^d(\mathbf{x}){|}^p\right],$$

(4)

where o(d)∈{1,2} denotes the order of the difference operator ${\Delta }_i^d(\mathbf{x})$.

For the blur we utilize the total-variation prior given by (see [23] for more details)

$$\mathrm{p}(\mathbf{h}|\gamma )\propto {\gamma }^{{\lambda }_{2}N}\text{exp}\left[-\gamma \text{TV}(\mathbf{h})\right],$$

(5)

where λ₂ is a positive real number and TV(h) is defined as

$$\text{TV}(\mathbf{h})=\sum _i\sqrt{{({\Delta }_i^h(\mathbf{h}))}^2+{({\Delta }_i^v(\mathbf{h}))}^2}.$$

(6)

In this study, we use flat improper hyperpriors on α, β and γ, that is, we utilize

$$\mathrm{p}(\alpha )\propto \text{const},\mathrm{p}(\beta )\propto \text{const},\mathrm{p}(\gamma )\propto \text{const}.$$

(7)

Note that with this choice of the hyperpriors, the observed image y is made solely responsible for the estimation of the image, blur and hyperparameters.

3 Bayesian inference

Bayesian inference on the unknown components of the blind image deconvolution problem is based on the estimation of the unknown posterior distribution p(α,β,γ,x,h∣y), given by

$$\mathrm{p}(\alpha ,\beta ,\gamma ,\mathbf{x},\mathbf{h}\mid \mathbf{y})=\frac{\mathrm{p}(\alpha ,\beta ,\gamma ,\mathbf{x},\mathbf{h},\mathbf{y})}{\mathrm{p}(\mathbf{y})}.$$

(8)

Assuming that x and h are independent, the joint distribution p(α,β,γ,x,h,y) can be factorized in terms of the observation model p(y|β,x,h), the prior distributions p(x|α) and p(h|γ), and the hyperparameter distributions p(α), p(β) and p(γ), that is,

$$\mathrm{p}(\alpha ,\beta ,\gamma ,\mathbf{x},\mathbf{h},\mathbf{y})=\mathrm{p}(\mathbf{y}|\beta ,\mathbf{x},\mathbf{h})\mathrm{p}(\mathbf{x}|\alpha )\mathrm{p}(\mathbf{h}|\gamma )\mathrm{p}(\alpha )\mathrm{p}(\beta )\mathrm{p}(\gamma ).$$

(9)

In this study, we adopt the maximum a posteriori (MAP) approach to obtain a single point estimate, $\bar{\mathrm{\Theta }}=(\bar{\alpha },\bar{\beta },\bar{\gamma },\bar{\mathbf{x}},\bar{\mathbf{h}})$, that maximizes p(α,β,γ,x,h∣y) as follows,

$$\begin{array}{ll}\bar{\mathrm{\Theta }}& ={\mathit{\text{argmax}}}_{\mathrm{\Theta }}\mathrm{p}(\alpha ,\beta ,\gamma ,\mathbf{x},\mathbf{h}\mid \mathbf{y})\\ =\mathit{\text{mi}}{n}_{\mathrm{\Theta }}\left\{\frac{\beta }{2}\parallel \mathbf{y}-\mathbf{H}\mathbf{x}{\parallel }^2+\alpha \sum _{d\in \mathrm{D}}{2}^{1-o(d)}\sum _i|{\Delta }_i^d(\mathbf{x}){|}^p\right.\\ +\left.\gamma \text{TV}(\mathbf{h})-\frac{{\lambda }_{1}N}{p}\text{log}\alpha -\frac{N}{2}\text{log}\beta -{\lambda }_{2}N\text{log}\gamma \right\}.\end{array}$$

(10)

As can be seen from (10), obtaining the point estimate that maximizes the posterior distribution p(α β γ x h∣y) is not straightforward since it requires the minimization of a non-convex functional. Maximizing the posterior distribution p(α β γ x h∣y) by iteratively optimizing in one variable while fixing the others (the so called iterated conditional modes (ICM) method [24]) is equivalent to the variational Bayesian based maximization (see [25] for an example derivation) for the special case when all the posterior distributions are assumed to be degenerate.

In this article, we apply the majorization-minimization approach twice to bound the non-convex functional to be minimized. We start by bounding the non-convex image prior p(x|α) by the functional M₁(α,x,Z), that is

$$\mathrm{p}(\mathbf{x}|\alpha )\ge \text{const}·{\mathrm{M}}_1(\alpha ,\mathbf{x},\mathbf{Z}).$$

(11)

The majorization-minimization approach has been utilized in several approaches for image restoration [25, 26].

The functional M₁(α,x,Z) is derived by considering the relationship between the weighted geometric and arithmetic means, which is given by

$$t^{p/2}{z}^{1-p/2}\le \frac{p}{2}t+\left(1-\frac{p}{2}\right)z,$$

(12)

where t≥0, z>0, and 0<p<2. We first rewrite (12) as

$$t^{p/2}\le \frac{p}{2}\frac{t+\frac{2-p}{p}z}{z^{1-p/2}}.$$

(13)

Using (13) we obtain

$$|{\Delta }_i^d(\mathbf{x}){|}^p\le \frac{p}{2}\frac{{[{\Delta }_i^d(\mathbf{x})]}^2+\frac{2-p}{p}{z}_{d,i}}{z_{d,i}^{1-p/2}}.$$

(14)

Therefore, we have

$$\begin{array}{l}\mathrm{p}(\mathbf{x}|\alpha )=\text{const}·{\alpha }^{{\lambda }_{1}N/p}\text{exp}\left[-\alpha \sum _{d\in \mathrm{D}}{2}^{1-o(d)}\sum _i|{\Delta }_i^d(\mathbf{x}){|}^p\right]\\ \ge \text{const}·{\alpha }^{{\lambda }_{1}N/p}\text{exp}\left[-\frac{\mathrm{\alpha p}}{2}\sum _{d\in \mathrm{D}}{2}^{1-o(d)}\right.\\ \left.\sum _i\frac{{[{\Delta }_i^d(\mathbf{x})]}^2+\frac{2-p}{p}{z}_{d,i}}{z_{d,i}^{1-p/2}}\right].\end{array}$$

(15)

Then (11) holds by setting

$$\begin{array}{l}{\mathrm{M}}_1(\alpha ,\mathbf{x},\mathbf{Z})={\alpha }^{{\lambda }_{1}N/p}\text{exp}\left[-\frac{\mathrm{\alpha p}}{2}\sum _{d\in \mathrm{D}}{2}^{1-o(d)}\right.\\ \left.\sum _i\frac{{[{\Delta }_i^d(\mathbf{x})]}^2+\frac{2-p}{p}{z}_{d,i}}{z_{d,i}^{1-p/2}}\right],\end{array}$$

(16)

where Z is a matrix with elements z_d,i, with d∈{h,v,hh,vv,hv} and i=1,…,N.

Similarly, the majorization-minimization criterion is used to bound the blur prior p(h|γ) utilizing the functional M₂(γ,h,u). Let us define, for γ and any N-dimensional vector u∈(R⁺ )^N, with components u _i , i=1,…,N, the following functional

$${\mathrm{M}}_2(\gamma ,\mathbf{h},\mathbf{u})={\alpha }^{{\lambda }_{2}N}\text{exp}\left[-\frac{\gamma }{2}\sum _i\frac{{({\Delta }_i^h(\mathbf{h}))}^2+{({\Delta }_i^v(\mathbf{h}))}^2+u_i}{\sqrt{u_i}}\right].$$

(17)

Using the inequality in (13) with p=1, for t≥0 and z>0, that is,

$$\sqrt{t}\le \sqrt{z}+\frac{1}{2\sqrt{z}}(t-z),$$

(18)

we obtain

$$\mathrm{p}(\mathbf{h}|\gamma )\ge \text{const}·{\mathrm{M}}_2(\gamma ,\mathbf{h},\mathbf{u}).$$

(19)

The lower bounds of p(x|α) and p(h|γ) defined above lead to the following lower bound of the distribution p(α,β,γ,x,h,y)

$$\begin{array}{l}\mathrm{p}(\alpha ,\beta ,\gamma ,\mathbf{x},\mathbf{h},\mathbf{y})=\mathrm{p}(\alpha )\mathrm{p}(\beta )\mathrm{p}(\gamma )\mathrm{p}(\mathbf{x}|\alpha )\mathrm{p}(\mathbf{h}|\gamma )\mathrm{p}(\mathbf{y}|\beta ,\mathbf{x},\mathbf{h})\\ \ge \text{const}·\mathrm{p}(\alpha )\mathrm{p}(\beta )\mathrm{p}(\gamma ){\mathrm{M}}_1(\alpha ,\mathbf{x},\mathbf{V}){\mathrm{M}}_2\\ (\gamma ,\mathbf{h},\mathbf{u})\mathrm{p}(\mathbf{y}|\beta ,\mathbf{x},\mathbf{h}).\end{array}$$

Therefore, a single point estimate that maximizes the lower bound of the posterior distribution p(α,β,γ,x,h∣y) is found as follows

$$\begin{array}{l}\bar{\mathrm{\Theta }}={\mathit{\text{min}}}_{\mathrm{\Theta }}\left\{\frac{\beta }{2}\parallel \mathbf{y}-\mathbf{H}\mathbf{x}{\parallel }^2+\frac{\mathrm{\alpha p}}{2}\sum _{d\in \mathrm{D}}{2}^{1-o(d)}\right.\\ \sum _i\frac{{[{\Delta }_i^d(\mathbf{x})]}^2+\frac{2-p}{p}{z}_{d,i}}{z_{d,i}^{1-p/2}}+\\ +\frac{\gamma }{2}\sum _i\frac{{({\Delta }_i^h(\mathbf{h}))}^2+{({\Delta }_i^v(\mathbf{h}))}^2+u_i}{\sqrt{u_i}}\\ \left.-\frac{{\lambda }_{1}N}{p}\text{log}\alpha -\frac{N}{2}\text{log}\beta -{\lambda }_{2}N\text{log}\gamma \right\}.\end{array}$$

(20)

As shown in (20), we are effectively replacing the original non-convex minimization problem (10) by a series of convex ones by utilizing the majorization-minimization criteria and introducing the additional variational vectors z _d and u. By iteratively solving this convex optimization problem in an alternating fashion with respect to all unknowns, we obtain a sequence of point estimates and derive the proposed algorithm as shown next.

3.1 Algorithm

Given α¹,β¹,γ¹, h¹, $u_i^1={[{\Delta }_i^h({\mathbf{h}}^1)]}^2+{[{\Delta }_i^v({\mathbf{h}}^1)]}^2$, and $z_{d,i}^1$.

for k=1,2,… until a stopping criterion is met:

• 1. Calculate

  $$\begin{array}{ll}{\mathbf{x}}^k& =\left[{\beta }^k{({\mathbf{H}}^k)}^t({\mathbf{H}}^k)+{\alpha }^{k}p\right.\\ {\left.\sum _d{2}^{1-o(d)}{({\mathit{\Delta }}^d)}^t{\mathbf{W}}_d^k({\mathit{\Delta }}^d)\right]}^{-1}{\beta }^k{({\mathbf{H}}^k)}^t\mathbf{y},\end{array}$$

  (21)

where ${\mathbf{W}}_d^k$ is a diagonal matrix with entries ${\mathbf{W}}_d^k(i,i)={(z_{d,i}^k)}^{p/2-1}$.

• 2. Calculate

  $$\begin{array}{ll}{\mathbf{h}}^{k+1}& =\left[{\beta }^k{({\mathbf{X}}^k)}^t({\mathbf{X}}^k)+{\gamma }^k\right.\\ {\left.\sum _{d\in \{h,v\}}{({\mathit{\Delta }}^d)}^t{\mathbf{U}}^k({\mathit{\Delta }}^d)\right]}^{-1}{\beta }^k{({\mathbf{X}}^k)}^t\mathbf{y},\end{array}$$

  (22)

where U^k is a diagonal matrix with entries ${\mathbf{U}}^k(i,i)={(u_i^k)}^{-1/2}$.

• 3. For each d∈{h,v,hh,vv,hv} calculate

  $$\begin{array}{ll}{z}_{d,i}^{k+1}& ={[{\Delta }_i^d({\mathbf{x}}^k)]}^2,\end{array}$$

  (23)

• 4. Calculate

  $$\begin{array}{ll}{u}_i^{k+1}& ={[{\Delta }_i^h({\mathbf{h}}^{k+1})]}^2+{[{\Delta }_i^v({\mathbf{h}}^{k+1})]}^2,\end{array}$$

  (24)

• 5. Calculate

  $${\alpha }^{k+1}=\frac{{\lambda }_{1}N/p}{{\sum }_{d\in \mathrm{D}}{2}^{1-o(d)}{\sum }_i|{\Delta }_i^d({\mathbf{x}}^k){|}^p},$$

  (25)
  $${\beta }^{k+1}=\frac{N}{\parallel \mathbf{y}-{\mathbf{H}}^{k+1}{\mathbf{x}}^k{\parallel }^2},$$

  (26)
  $${\gamma }^{k+1}=\frac{{\lambda }_{2}N}{\text{TV}({\mathbf{h}}^{k+1})},$$

  (27)

In the line of study presented in [21] the parameter p is set to 4/5 (see [21] for a detailed discussion). Additionally, the parameters λ₁and λ₂are needed to approximate the partition functions for prior distributions p(x|α) and p(h|γ), respectively. Unfortunately, the approximations of partition functions for the distributions p(x|α) and p(h|γ) are necessary since its corresponding partition functions are analytically intractable. We follow the approaches proposed in [22, 23], as already described in the previous section, and determine the values of the parameters λ₁ and λ₂ experimentally. The values of the parameters p, λ₁, and λ₂ are therefore set throughout all the experiments that follow. The robustness of the proposed method will be tested and evaluated under various blurring and noisy conditions.

Note that if the blur h and the hyperparameters α β, and γ are assumed to be known, the proposed algorithm coincides with the iteratively re-weighted least squares (IRLS) algorithm presented in [21] (i.e., in this case for both algorithms the image estimate is calculated as shown in (21)). Note, that the l _p quasi norm based prior is also utilized in [15], and that this study simplifies the prior used in [21] by omitting the second order derivatives.

4 Multi-scale implementation

The restoration results presented in [12], and more recently in [27], showed the effectiveness of the multiscale approach in implementing blind image deconvolution algorithms. Furthermore, it is shown in [27] that the multi-scale approach prevents the algorithm from converging to the unit impulse. Alternatively, the authors in [13, 14] introduced heuristic re-weighting of the regularization parameters, at each iteration, to prevent the algorithms from converging to unrealistic blur estimates. In our study, no heuristic adjustment of each parameter is performed but instead we estimate the parameters automatically within the Bayesian framework.

To avoid unrealistic blur estimates we adopt here a multi-scale scheme similar in spirit to the one proposed in [12]. Additionally, the proposed multi-scale approach allows user to automatically initialize the proposed algorithm without visually inspecting the observed blurred image for determining the initial blur estimate. By analyzing the observed blurred image it is possible to come up with more informative initial blur estimates; however in this study our focus is to develop a completely automated algorithm once the blur support is provided.

The basic idea behind the multi-scale approach is to down-sample the observed blurred image to a number of low resolution images. At the lowest resolution the initial blur estimate (i.e., h¹) is set to the uniform blur and the lowest resolution of the down-sampled observed image is utilized as the initial image estimate (i.e., x¹). After convergence is achieved at each scale we up-sample the image and the blur estimates to the next higher resolution and re-run the proposed algorithm. This iterative process is repeated until the algorithm converges and image and blur estimates are obtained at their native resolutions. The detailed pseudocode used in the implementation of our multi-scale approach is shown in Appendix.

5 Experimental results

In this section, we present the experimental results obtained by the use of the proposed algorithm. As the performance metric, for the experiments in which the original image is known, we utilize the improvement in signal to noise ratio of the restored image (denoted as ${\text{ISNR}}_{\widehat{\mathbf{x}}}$), which is defined as $10{\text{log}}_{10}\left(\parallel \mathbf{x}-\mathbf{y}{\parallel }^2/\parallel \mathbf{x}-\widehat{\mathbf{x}}{\parallel }^2\right)$, where x, y and $\widehat{\mathbf{x}}$ are the original, observed and estimated images, respectively. Analogously, when the blur is known we utilize the improvement in signal to noise ratio of the restored blur (denoted as ${\text{ISNR}}_{\widehat{\mathbf{h}}}$), which is defined as $10{\text{log}}_{10}\left(\parallel \mathbf{h}-{\mathbf{h}}_{\delta }{\parallel }^2/\parallel \mathbf{h}-\widehat{\mathbf{h}}{\parallel }^2\right)$, where h, h _δ and $\widehat{\mathbf{h}}$ are the original blur, the unit impulse, and the estimated blur, respectively.

In addition, after the blur support, greater than the original one, is specified by the user, all unknown parameters and the estimates of the unknown image and blur are estimated automatically as described in Sections 2 and 2. Specifying the blur support for the unknown blur is common with the state-of-the-art approaches (see [12–14, 16, 19]).

Similarly to these approaches, the proposed algorithm is very robust when the support of the blur is largely overestimated by the user, as can be seen in all the experiments that follow. Also, for the experiments in which blurred colored images are considered, only the luminance component of the observed image is restored while the observed chroma components are used to obtain the restored colored image once the original luminance is estimated. Finally, the proposed algorithm is terminated when the criterion ∥x^k−x^k−1∥/∥x^k−1∥<10⁻³ is achieved or the number of iterations reaches 100. After each iteration, we enforce the following constraints on the blur estimates: the positivity (blur elements less than zero are set to zero), the support constraint (blur elements outside of the blur support estimate are set to zero), and the energy conservation (sum of the blur elements equals one).

In the first set of experiments, we evaluate the performance of the proposed method on four standard images (cameraman, satellite, shepp-logan phantom and airplane) which are widely used in image restoration experiments. The original images are then blurred with five different motion blurs, which are shown in Figure 1. Realizations of white Gaussian noise are added to the respective blurred images in order to obtain degraded images with the blurred signal to noise (BSNR) ratio of 40 dB. The blurred signal to noise ratio is defined as follows

$$\text{BSNR}=10{\text{log}}_{10}\frac{\text{Var}(\mathbf{H}\mathbf{x})}{\text{Var}(\mathbf{n})},$$

(28)

Figure 1

Five different synthetic non-parametric motion blurs included in the window of size 21×21 : (a) Blur 1: the support is 14×12 , (b) Blur 2: the support is 15×14 , (c) Blur 3: the support is 14×14 (d) Blur 4: the support is 18×19, (e) Blur 5: the support is 18×15 .

where Var(·) denotes the variance of the random sequence. Example restorations obtained by the proposed algorithm are shown in Figure 2. The restoration results in terms of the previously defined ${\text{ISNR}}_{\widehat{\mathbf{x}}}$ and ${\text{ISNR}}_{\widehat{\mathbf{h}}}$, metrics are shown in Table 1. It can be observed from Table 1 that the proposed algorithm is very robust and it is capable of restoring the blurred images very successfully under various non-parametric motion blurs. Example convergence curves obtained by the proposed algorithm are shown in Figure 3.

Figure 2

Example restorations from Table 1 : 1st column represents four different blurred observations, 2nd column represents their respective restorations obtained by the proposed algorithm, 3rd column represents their respective original images, 4th column represents their respective blurs obtained by the proposed algorithm, 5th column represents their respective original blurs.

Figure 3

Example convergence curves for the finest scale of the proposed algorithm of the airplane-color restoration for five different synthetic non-parametric motion blurs: (a) Blur 1. (b) Blur 2. (c) Blur 3. (d) Blur 4 and (e) Blur 5.

Table 1 ${\mathbf{\text{ISNR}}}_{\widehat{\mathbf{x}}}$ and ${\text{ISNR}}_{\widehat{\mathbf{h}}}$values, for the cameraman, satellite, shepp-logan, and airplane images degraded by five different motion blurs (BSNR = 40 dB)

In the second set of experiments, we evaluate the performance of the proposed method on a set of 32 blurred test images taken by a commercial camera. The test images are obtained from [27] and they are available online (http://www.wisdom.weizmann.ac.il/~levina/papers/LevinEtalCVPR09Data.zip). The set of 32 blurred images was obtained by taking the original images shown in Figure 4 and by putting them side by side in order to form a calibration image. Once the calibration image was formed, a commercial camera was mounted on a tripod and eight photos of the calibration image were obtained. Note that during the acquisition process, the Z-axis rotation handle was locked in while the X-axis and Y-axis handles were loosened up in order to simulate in-plane camera shake (see [27] for details; resulting blurs are shown in Figure 5). In this study, we consider a comparison with the following methods [12, 13, 16, 19]. For convenience, from now on, the methods proposed in [12, 13, 16], and the best method from [19] will be denoted, respectively, as Fergus et al., Shan et al., Cho et al., and Levin at al., while the proposed algorithm will be denoted as ALG.

Figure 4

Four different original images from[27]: (a) Image 1. (b) Image 2. (c) Image 3. (d) Image 4.

Figure 5

Eight different non-parametric motion blurs from[27]: (a) Blur 1. (b) Blur 2. (c) Blur 3. (d) Blur 4. (e) Blur 5. (f) Blur 6. (g) Blur 7 and (h) Blur 8.

The restoration results, for the second set of experiments where the original image and blur are both known, in terms of Sum of Squared Errors (i.e., ${\text{SSE}}_{\widehat{\mathbf{x}}}=\parallel \mathbf{x}-\widehat{\mathbf{x}}{\parallel }^2$) and the SSE ratio test (i.e., ${\text{ER}}_{\widehat{\mathbf{x}}}={\text{SSE}}_{\widehat{\mathbf{x}}}/{\text{SSE}}_{\tilde{\mathbf{x}}}$) defined in [27] are shown in Table 2. Note that $\tilde{\mathbf{x}}$ is an image estimation obtained from the non-blind method from [27]. It can be observed from Table 2 that the proposed algorithm is very robust and that it is capable of restoring blurred images taken by a commercial camera very successfully under various non-parametric motion blurs. In addition, the proposed algorithm is very competitive with the state-of-the-art methods. As can be noted in Table 2, the ${\text{SSE}}_{\widehat{\mathbf{x}}}$ metric for the proposed algorithm is respectively, on the average, 427, 89, 6, and 21 smaller than the ${\text{SSE}}_{\widehat{\mathbf{x}}}$ metric for the Fergus at al., Shan at al., Cho at al., and Levin et al. methods. Note that in Fergus’ method, the blur estimation is performed separately from the image estimation. In order to understand the differences in the restoration results we provide some additional information. In Figure 6 it can be seen that for 66 % of tested cases the proposed algorithm yields the smallest ${\text{SSE}}_{\widehat{\mathbf{x}}}$ values. In addition, Figure 7 shows that for a number of test cases the proposed algorithm is capable of achieving very small ${\text{ER}}_{\widehat{\mathbf{x}}}$ values. For example, there are 78 % of the cases for which the proposed algorithm has ${\text{ER}}_{\widehat{\mathbf{x}}}$ smaller than 1.5 while at the same time (as the second best) there are 53 % of the test cases for which Cho et al. method achieves such condition. Example restorations and comparison with Fergus at al., Cho at al., Shan at al, and Levin at al. are shown, respectively, in Figures 8, 9, 10, and 11.

Figure 6

Comparing the proposed algorithm with the methods proposed in[12, 13, 16, 19]based on the restoration results from Table2: Percentage of cases for which the restored image yields the smallest${\text{SSE}}_{\widehat{\mathbf{x}}}$values.

Figure 7

Comparing the cumulative histogram of the proposed algorithm with the cumulative histogram of the methods proposed in[12, 13, 16, 19]based on the restoration results from Table2in terms of the${\text{ER}}_{\widehat{\mathbf{x}}}$comparison metric.

Figure 8

Example restorations from Table2: 1st column represents eight different blurred observations, 2nd column represents their respective original undistorted versions, 3rd column represents their respective restorations obtained by the proposed algorithm, 4th column represents their respective restorations obtained by the method proposed in[12], 5th column represents their respective original blurs, 6th column represents their respective restored blurs obtained by the proposed algorithm, 7th column represents their respective restored blurs obtained by the method proposed in[12].

Figure 9

Example restorations from Table2: 1st column represents eight different blurred observations, 2nd column represents their respective original undistorted versions, 3rd column represents their respective restorations obtained by the proposed algorithm, 4th column represents their respective restorations obtained by the method proposed in[16], 5th column represents their respective original blurs, 6th column represents their respective restored blurs obtained by the proposed algorithm, 7th column represents their respective restored blurs obtained by the method proposed in[16].

Figure 10

Example restorations from Table2: 1st column represents eight different blurred observations, 2nd column represents their respective original undistorted versions, 3rd column represents their respective restorations obtained by the proposed algorithm, 4th column represents their respective restorations obtained by the method proposed in[13], 5th column represents their respective original blurs, 6th column represents their respective restored blurs obtained by the proposed algorithm, 7th column represents their respective restored blurs obtained by the method proposed in[13].

Figure 11

Example restorations from Table2: 1st column represents eight different blurred observations, 2nd column represents their respective original undistorted versions, 3rd column represents their respective restorations obtained by the proposed algorithm, 4th column represents their respective restorations obtained by the method proposed in[19], 5th column represents their respective original blurs, 6th column represents their respective restored blurs obtained by the proposed algorithm, 7th column represents their respective restored blurs obtained by the method proposed in[19].

Table 2 ${\mathbf{\text{SSE}}}_{\widehat{\mathbf{x}}}$ and ${\text{ER}}_{\widehat{\mathbf{x}}}$ values, for the images and blurs defined in Figures 4 and 5 , respectively

In the third set of experiments, we compare the performance of the proposed method with the method proposed in [14] by using the same set of blurred photographs as presented in [14]. Note that in the method proposed in [14] the parameters are not estimated but rather they are manually tuned which results in the sequence of numbers for each parameter. As can be seen in Figure 12 the restoration results obtained by the proposed algorithm are very competitive with the method proposed in [14]. It is clear from Figure 12 that the proposed algorithm produces much sharper restoration results with higher visual quality. Since we lack the true knowledge of the scene, the comparison metrics ${\text{SSE}}_{\widehat{\mathbf{x}}}$ and ${\text{ER}}_{\widehat{\mathbf{x}}}$ are undefined for this experiment.

Figure 12

Comparing the proposed method with the method proposed in[14]: 1st column represents three different blurred observations, 2nd column represents their respective restorations obtained by the proposed algorithm, 3rd column represents their respective restorations obtained by the method proposed in[14], 4th column represents their respective blurs obtained by the proposed algorithm, 5th column represents their respective blurs obtained by the method proposed in[14].

6 Conclusions

In this article, a novel blind image deconvolution algorithm is presented. The proposed algorithm was developed within a Bayesian framework utilizing a non-convex l _p quasi norm based sparse prior on the image, and a total-variation prior on the unknown blur. The proposed algorithm is completely automated once the blur support is provided. Experimental results demonstrate that using sparse priors and the proposed parameter estimation, both the unknown image and blur can be estimated with very high accuracy. Furthermore, numerous restorations of photographs taken by commercial cameras are provided to demonstrate the robustness and effectiveness of the proposed approach. Finally, it was shown that the performance of the proposed algorithm is competitive to existing state-of-the-art blind image deconvolution algorithms.

Appendix

Algorithm Appendix is shown below.