 Research
 Open Access
 Published:
Blockiterative RichardsonLucy methods for image deblurring
EURASIP Journal on Image and Video Processing volumeÂ 2015, ArticleÂ number:Â 14 (2015)
Abstract
In this paper, we extend the RichardsonLucy (RL) method to blockiterative versions, separated BIRL, and interlaced BIRL, for image deblurring applications. We propose combining algorithms for separated BIRL to form block artifactfree output images from separately deblurred block images. For interlaced BIRL to accelerate the iteration, we propose an interlaced blockiteration algorithm on downsampled blocks of the observed image. Simulation studies show that separated BIRL and interlaced BIRL achieve desired goals in Gaussian and diagonal deblurrings.
1 Introduction
The image deblurring problem has many applications in science and engineering fields, and many methods have been proposed for it [1]. Among them, the RichardsonLucy (RL) method, which was proposed independently by Richardson [2] and Lucy [3] in 1970s, has been one of the most widely used iterative deblurring methods. Applications of RL in microscopy, astronomy, or motion deblurring can be found in [4, 5] and references therein. Variants of RL with specific purposes such as adaptivity, parallel implementation, acceleration, suppression of ringing/boundary artifacts, or edgepreserving can be found in [6â€“12].
In the paper by Shepp and Vardi [13], the same algorithm was rederived from the maximization likelihood principle in the application to emission tomography and the expectation maximization (EM) method in [14]. In the paper by Hudson and Larkin [6], the ordered subsets expectation maximization (OSEM) method was proposed as an acceleration technique for EM. Since its introduction, OSEM has been successfully used for many applications in emission tomography [15, 16].
The use of RL in image deblurring is, however, computer intensive and often suffers from slow convergence. To deal with these obstacles, blockiterative RL (BIRL) methods have been proposed [17, 18]. BIRL decomposes the main problem into several subproblems by grouping pixels of the observed image into several subsets and applies RL to each subproblem. In this paper, image parts defined on those grouped pixel subsets will be called blocks. In [17], the observed image was decomposed to four rectangular blocks, RL was applied to each block separately, and the final image was obtained by combining four deblurred block images. In [18], an OSEMlike method was proposed to accelerate the iteration by using multiviews of the image. The term â€˜multiviews of the imageâ€™ means that the image to be recovered is observed several times under different imaging environments. The observed data at each imaging environment is often called a view of the image. The problem of deblurring images from multiviews is also called multiple image deconvolution [4].
In this paper, we extend methods in [17, 18] to separated BIRL and interlaced BIRL, for image deblurring applications.
The goal of separated BIRL is to combine separately deblurred subimage blocks into the resulting output image without block artifacts. Obviously, this is not a new approach. What is new in the proposed separated BIRL is that the proposed method can suppress block artifacts efficiently for arbitrarily shaped blocks. This flexibility is important for the success of separated BIRL; simulation results of this paper will show that diagonally shaped blocks produce better results than rectangularly shaped blocks for separated BIRL in the deblurring problem modeled by a diagonal point spread function (PSF).
The goal of interlaced BIRL is the accelerated iteration by using an OSEMlike method as in [18]. Image deblurring problems of this paper, however, are assumed to have only one view of the image, unlike in [18]. To overcome this â€˜one viewâ€™ limitation, the proposed interlaced BIRL decomposes the observed image into several downsampled blocks and treats those downsampled blocks as multiviews of the image. Interlaced BIRL accelerates the iteration by using the onestep RL block iterates as the starting image in the next block iteration, as OSEM does in emission tomography.
The performance of proposed methods depends on how blocks are formed. There are many possibilities for the block partition. It is, however, not clear how blocks should be formed for a specific PSF.
The work in this paper will be restricted to the test of rectangular or diagonal subimage blocks for separated BIRL and rectangularly or diagonallydownsampled blocks for interlaced BIRL in Gaussian and diagonal deblurrings. While explaining test results, possible extensions of proposed methods to general deblurring problems will be presented briefly.
2 Definition and background
2.1 Image deblurring problem
We assume that the true image f is defined on an index set Î© and its observed version g on an index set Î›. We also assume that g is blurred from f by a linear operator \({\mathcal T}\colon \ell ^{2}(\Omega) \to \ell ^{2}(\Lambda)\) and further corrupted by pixelwisely independent Poisson noise:
Here, \(g_{i_{1},i_{2}}\) represents the intensity of the image g at the pixel (i _{1},i _{2}). We will use the same convention, the use of boldface alphabet for the image and the normal alphabet with the pixel subscript for the intensity, throughout this paper. Notations â„“ ^{2}(Î©) and â„“ ^{2}(Î›) are used to denote image spaces defined on Î© and Î›, respectively. Here, inner products of â„“ ^{2}(Î©) and â„“ ^{2}(Î›) are the usual dot product of two images (sums of pixelbypixel multiplications).
In (1), we assume that the blurring by \({\mathcal T}\) represents a truncated convolution with a known PSF \(\mathbf {k} = (k_{i_{1},i_{2}})\): For any image pâˆˆâ„“ ^{2}(Î©),
Here \({\mathcal S}_{\mathbf {k}}\), the support of k, is \(\phantom {\dot {i}\!}\{(j_{1},j_{2})\mid k_{j_{1},j_{2}}>0\}\). We also assume that the PSF k is nonnegative, its components have sum 1, and the point \((0,0)\in {\mathcal S}_{\mathbf {k}}\); \({\mathcal T}\mathbf {p}\) is defined on Î›, where (i _{1},i _{2})âˆˆÎ› if and only if \((i_{1},i_{2}){\mathcal S}_{\mathbf {k}}\subset \Omega \). Thus, Î›âŠ‚Î©.
2.2 Computation of \({\mathcal T}\) and \({\mathcal T}^{\ast }\)
Let \({\mathcal T}^{\ast }\) be the adjoint operator of \({\mathcal T}\) in (2), i.e., \({\mathcal T}^{\ast }\) is defined by the relation
for qâˆˆâ„“ ^{2}(Î›) and pâˆˆâ„“ ^{2}(Î©), and hence \({\mathcal T}^{\ast }\colon \ell ^{2}(\Lambda) \to \ell ^{2}(\Omega)\). Here, notations Â· and âŠ™ were used to denote dot products of images defined on Î› and Î©, respectively.
The computation of \({\mathcal T}\textbf {p}\) can be carried out by the pixelwise definition (2) or by using the fast Fourier transform (FFT) with a zero padding. In the case when the pixelwise definition is used for \({\mathcal T}\textbf {p}\), then
since \({\mathcal S}_{\mathbf {k}}\) operations (one operation = one multiplication + one addition) are required for the computation of \(({\mathcal T}\textbf {p})_{i_{1},i_{2}}\) for each (i _{1},i _{2})âˆˆÎ›. Here, OC means operation counts. On the other hand, with the assumption \({\mathcal S}_{\mathbf {k}} < \Omega \),
It is not difficult to show that \({\mathcal T}^{\ast }\) can be computed by
for qâˆˆâ„“ ^{2}(Î›).
Thus,
It is also possible that the computation of \({\mathcal T}^{\ast }\textbf {q}\) can be carried out by using FFT with a zero padding. In that case, with the assumption \({\mathcal S}_{\mathbf {k}} < \Omega \),
2.3 Richardsonâ€“Lucy iteration
For the image deblurring problem (1), one iteration of RL takes
Here, the notation I _{ A } means the allone image on the pixel subset A and.âˆ— is the pixelbypixel multiplication, and the division between two images, \({\frac {\mathbf {g}}{{\mathcal T}\mathbf {f}^{n}}}\), is the image resulted from the pixelbypixel division of g and \({\mathcal T}\mathbf {f}^{n}\).
For future use, onestep iteration of RL (8) will be denoted by
Here, the PSF k was used instead of \({\mathcal T}\) and \({\mathcal T}^{\ast }\).
Performing RL as described in (8) often results in a very slow convergence. For the acceleration of RL, several methods have been suggested [7, 19â€“21]. Among them, the technique in [7] has been noted for its success. For example, the function â€˜deconvlucy,â€™ the RL implementation of the software MATLABÂ®; (The MathWorks, Natick, MA, USA), uses the technique described in Section 2.3 and 2.4 in [7]. Interlaced BIRL, the proposed method for the acceleration of RL in this paper, will be compared with the technique in [7] in Section 4.
2.4 Boundary artifacts
Boundary artifacts are one of key obstacles in the development of BIRL; block boundaries introduced by the block partition may cause artifacts in combining deblurred block images. It is also true that boundary artifacts are one of key obstacles in many image deblurring problems [22].
To reduce boundary artifacts in image deblurring, many methods have been proposed. One group of methods imposes certain conditions on pixels in Î©âˆ’Î›. Examples include periodic, reflective, and antireflective boundary conditions [23â€“27].
Other group of methods [17, 28, 29] does not impose any conditions on pixels in Î©âˆ’Î›, and let the iteration itself determine results in Î©âˆ’Î›. In [29], this approach is called the free boundary condition method.
Before we close this section, it is worth to mention some research works related to fast direct deblurring methods. It is well known that if the imposed boundary condition is one of periodic, reflective, or antireflective boundary conditions, then the image deblurring with symmetric PSFs (for a periodic boundary condition, the symmetry of PSF can be omitted) can be directly computable by using FFT for periodic boundary condition, discrete cosine transform (DCT) for reflective boundary condition, and discrete sine transform (DST) for antireflective boundary condition [1, 23, 25].
These fast transformbased direct deblurring methods, however, often present severe boundary artifacts. To reduce boundary artifacts, one can smooth the boundaries of the observed image to decay to 0 (to make the imposed boundary condition to be more feasible) before those direct deblurring methods are applied to. This approach can reduce boundary artifacts in some degree, but, at the same time, makes it more difficult to recover near boundary image pixels.
The performance of direct deblurring methods depends heavily on the feasibility of imposed boundary conditions. The difficulty of imposing correct boundary conditions is the main reason why iterative deblurring approaches with free boundary conditions have been considered, despite the fact that fast direct deblurring methods are available [17, 28, 29].
Considering these facts, we suggest the free boundary conditionbased RL approach for the image deblurring problem (1). To reduce computational burden and accelerate the slow convergence of RL, we will propose blockiterative methods in the next section.
3 Proposed method
3.1 Block partition
BIRL decomposes the original deblurring problem \(\mathbf {g}={\mathcal T}\mathbf {f}\) (here the noise term is ignored for the simplicity of the presentation) into several (say t) block deblurring problems
by partitioning Î› into Î› _{ i }, where \(\mathbf {g}^{[i]} = \mathbf {g}\mid _{\Lambda _{i}}\), the restriction of g on Î› _{ i }, and \({\mathcal T}_{i} = {\mathcal T} \mid _{\Omega _{i}}\), the restriction of \({\mathcal T}\) on some subset Î© _{ i }âŠ‚Î©. Since only pixels that contribute to the observation on Î› _{ i } can be recovered in (9), the subset Î© _{ i } can be defined by
for i=1,2,â€¦,t. Notice that pixels only in Î© _{ i } can contribute the observation g ^{[i]} (defined on Î› _{ i }) and Î› _{ i }âŠ‚Î© _{ i } for i=1,2,â€¦,t.
Throughout this paper, Î› _{ i } are assumed to be mutually disjoint, unless stated otherwise.
In any cases, Î› _{ i } are selected to satisfy
3.2 Separated BIRL
Separated BIRL: Given N,
In the step S7, weights r ^{[i]} are defined by
The weights r ^{[i]} in (11) are motivated by the following interpretation. Recall that \({\mathcal T}\) represents the truncated convolution by the PSF k that is nonnegative and has 1 as its sum of all components. Thus, with the assumption that mutually disjoint blocks Î› _{ i }, i=1,2,â€¦,t are selected to form \(\Lambda = \cup _{i=1}^{t}\Lambda _{i}\), \(({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}})_{j_{1},j_{2}}\) can be interpreted as the probability with which the pixel (j _{1},j _{2})âˆˆÎ© contributes the observation on Î› _{ i }. This argument shows that weights r ^{[i]} make the intensity at the pixel (j _{1},j _{2}) to depend on f ^{[i],N} (the deblurred image from g ^{[i]} defined on Î› _{ i }) proportionally to the probability of the contribution of the pixel (j _{1},j _{2}) to g ^{[i]} (the observation on Î› _{ i }).
For certain deblurring problems, it is desirable to use overlapped blocks Î› _{ i }. In such case, separated BIRL separately deblurs g ^{[i]} on overlapped blocks Î› _{ i } to produce a deblurred block image f ^{[i],N} for each i, and, after cropping out some of overlapped pixels of f ^{[i],N}, i=1,â€¦,t, combines them to the final output image.
3.3 Interlaced BIRL
Interlaced BIRL: Given N,
To explain a key point of interlaced BIRL, let us compare interlaced BIRL with RL. Notice that RL can be obtained from interlaced BIRL by replacing the inner for loop (steps I3, I4, and I5) with f ^{m+1}=RL(g,f ^{m},k,Î›). Suppose that selected blocks Î› _{ i }, i=1,2,â€¦,t, satisfy
where Î© _{ i } is computed by (10) from Î› _{ i }. In this case, one iteration of interlaced BIRL (from steps I3 to I5) updates most pixel values on Î©, including all pixel values on Î›, t times, while RL updates pixel values on Î© just once. Notice that, in the algorithmic point of view, the described benefit of interlaced BIRL is identical to that of OSEM [6] in emission tomography.
RL often uses the allone image I _{ Î© } as the initial guess f ^{0}. Interlaced BIRL, however, uses f ^{0}=RL(g,I _{ Î© },k,Î›) as the initial guess (see the step I1). This suggestion is made to update pixel values of f ^{0} on Î© _{ i }, without causing many discontinuities.
Figure 1 illustrates the procedure of interlaced BIRL. In Fig. 1, the observed image g is decomposed into four block images g ^{[i]}, i=1,2,3,4, defined on 2Ã—2 rectangularlydownsampled blocks Î› _{ i }, i=1,2,3,4. The initial guess f ^{0} is computed by RL(g,I _{ Î© },k,Î›). In m=0 and i=1, the step I4 updates pixel values of f ^{0} on Î© _{1} by \(\textbf {RL}(\mathbf {g}^{[1]}, \mathbf {f}^{0} \mid _{\Omega _{1}}, \mathbf {k}, \Lambda _{1})\) (I4). The resulting image is denoted by f ^{1/4}. This procedure makes pixel values of f ^{1/4} on Î© _{1} to be already quite close to the true image f at pixels on Î© _{1}, but it does not change pixel values on Î©âˆ’Î© _{1}, i.e., f ^{1/4}=f ^{0} on Î©âˆ’Î© _{1}. Thus, the step I4 would cause many discontinuities if the allone image I _{ Î© } were selected for the initial guess f ^{0}.
In the next subiteration (m=0 and i=2), pixel values on Î© _{2} are updated by \(\textbf {RL}\left (\mathbf {g}^{[2]}, \mathbf {f}^{1/4} \mid _{\Omega _{2}}, \mathbf {k}, \Lambda _{2}\right)\). Once pixel values are updated on remaining blocks, Î© _{3} and Î© _{4}, the resulting image is f ^{1} as shown in Fig. 1.
3.4 Algorithmic limitations of proposed methods
In the algorithmic point of view, separated BIRL can be used for any PSFs. Roughly speaking, the size of the PSF determines the minimum size of blocks that can be used for separated BIRL, and the number of blocks determines the maximum gain in separated BIRL by parallel computations.
The usefulness of interlaced BIRL, however, is limited to PSFs with small number of nonzero elements. To explain this, let us recall that interlaced BIRL uses pointwise computations for convolutions, (2) and (5). Thus, one iteration of interlaced BIRL with t downsampled blocks requires \(2{\mathcal S}_{\mathbf {k}} \cdot \Lambda  + 2t\Omega \) operation counts, where \({\mathcal S}_{\mathbf {k}}\) is the number of nonzero elements in the PSF k. On the other hand, one iteration of FFTbased RL requires 2CÎ© log2Î©+Î© operation counts, where C is a constant which depends on the way of implementing FFT algorithm. Therefore, in order for interlaced BIRL with t downsampled blocks to be useful as compared with FFTbased RL, the PSF k must satisfy
where the denominator t in the lefthand side is used by considering that interlaced BIRL with well chosen t downsampled blocks accelerates iterations t times. The condition (13) does not hold for PSFs with large numbers of nonzero elements.
In our simulation, we used PSFs k with \({\mathcal S}_{\mathbf {k}} \le 441\) and 480Ã—480 sized images. For such PSFs and images, the FFTbased RL implementation optimized for 512Ã—512 sized images was slower than the pixelwise computationbased RL implementation. The condition \({\mathcal S}_{\mathbf {k}} \le 441\) includes PSFs that are used in many important image deblurring applications. For instance, any PSFs that have not more than 441 nonzero elements (e.g., a PSF of the form of the 441Ã—441 diagonal matrix) satisfy this condition. Thus, interlaced BIRL is computationally more efficient than FFTbased RL in those image deblurring applications.
3.5 Free boundary condition
Proposed methods use the free boundary condition to suppress block artifacts. This suggestion is based on the observation that the free boundary condition successfully suppresses boundary artifacts for arbitrarily shaped images; note that periodic, reflective, and antireflective boundary conditions can be applied to rectangularshaped images only. For details, see [23â€“26].
The use of the free boundary condition appears in the step S4 and the step I4 by imposing no restriction on pixels on Î© _{ i }âˆ’Î› _{ i }. The step I1 of interlaced BIRL also uses the free boundary condition, again, by imposing no restriction on pixels on Î©âˆ’Î›.
3.6 Examples of blocks
Examples of blocks in this section are selected by using following two suggestions:

For separated BIRL, select blocks Î› _{ i }, i=1,2,â€¦,t, that make Î© _{ j }âˆ©Î© _{ n } as small as possible for all pairs (j,n), 1â‰¤j<nâ‰¤t.

For interlaced BIRL, select the block Î› _{ i } that makes Î© _{ i }â‰ˆÎ© and \({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}}\) to be uniform as much as possible for each i.
Simulation results in Section 4 will show why these two suggestions are important.
The work in this paper will be restricted to the test of following blocks (illustrated in Fig. 2) in deblurring problems modeled by the Gaussian PSF k _{ G } (Fig. 3a) and the diagonal PSF k _{ D } (Fig. 3b).
3.6.1 3.6.1 Rectangular blocks
Figure 2a shows 4Ã—4rectangular blocks of the observed image (Fig. 5a). Separated BIRL with 2Ã—2, 4Ã—4, 8Ã—8, and 16Ã—16 rectangular blocks will be tested in the Gaussian deblurring.
Overlapped rectangular blocks can be formed by adding several pixel rows and columns to boundaries of disjoint rectangular blocks. This type of overlapped blocks will be also tested in separated BIRL.
3.6.2 3.6.2 Diagonal blocks
Figure 2b shows 8 diagonal blocks of the observed image (Fig. 5b). Separated BIRL with 2, 4, 8, and 16 diagonal blocks will be tested in the diagonal deblurring.
3.6.3 3.6.3 Rectangularlydownsampled blocks
Figure 2(c) shows 4Ã—4rectangularlydownsampled blocks of the observed image (Fig. 5a). To be specific, if the pixel index at the left and upper corner of the observed image is (0,0), then 4Ã—4 downsampled blocks are defined by
for x,y=0,1,2,3, where \(4{\mathbb Z}\) is the integer subset formed by multiples of 4. With the same argument, other rectangularlydownsampled blocks can be defined. Interlaced BIRL with 2Ã—2, 4Ã—4, 6Ã—6, and 8Ã—8 rectangularlydownsampled blocks will be tested in the Gaussian deblurring.
3.6.4 3.6.4 Diagonallydownsampled blocks
Figure 2d shows 8 diagonallydownsampled blocks of the observed image (Fig. 5b).
To be specific,
for n=0,1,â€¦,7, form 8 diagonallydownsampled blocks.
Interlaced BIRL with 2, 4, 6, and 8 diagonallydownsampled blocks will be tested in the diagonal deblurring.
4 Simulation results and discussion
We conducted simulation studies to test the performance of proposed methods in Gaussian and diagonal deblurrings. In simulation studies, â€˜cameramanâ€™ (Fig. 4a) and â€˜girlâ€™ (Fig. 4b), of size 500Ã—500, were used as true images. PSFs k _{ G } and k _{ D } in Fig. 3 were applied to â€˜cameramanâ€™ and â€˜girlâ€™, respectively, to produce blurred images of size 480Ã—480 (both PSFs are of size 21Ã—21).
Blurred images were further corrupted by the Poissonian noise model in (1). Total sums of intensities of noisy blurred images (Fig. 5) were 2.6 and 3.1 billions for â€˜cameramanâ€™ and â€˜girlâ€™, respectively.
As mentioned in Section 3, interlaced BIRL is useful only when pixelwise computations (2) and (5), instead of FFTbased ones, are used for \({\mathcal T}\) and \({\mathcal T}^{*}\). In simulation studies, we used a personal computer equipped with 2.0 GHz Intel Core 2 Duo CPU and 8 GB RAM. In this computing environment, one RL iteration with FFTbased computations took 5.04 s both for Gaussian and diagonal deblurrings. On the other hand, one round of interlaced BIRL (the forloop in steps I3, I4, and I5) with pixelwise computations took 4.84 s for 4Ã—4 rectangularlydownsampled blocks for Gaussian deblurring (\({\mathcal S}_{\mathbf {k}_{G}} = 441\)) and 0.88 s for 8 diagonallydownsampled blocks for the diagonal deblurring (\({\mathcal S}_{\mathbf {k}_{D}} = 61\)). This result implies that interlaced BIRL is, at least computational point of view, useful for both deblurring problems in our simulation. Considering these facts, we used pixelwise computations (2) and (5) in our simulation.
Based on the experience that it is not easy to choose an unbiased stopping rule, we selected the image that had the smallest relative square error (RSE) within 1000 iterations as the deblurred image of the tested method. Here, the RSE is defined by
where \({\hat f}_{i_{1},i_{2}}\) and \(f_{i_{1},i_{2}}\) are intensities of the deblurred image and the true image, respectively.
4.1 Standard RL
Figure 6a, b shows deblurred images by RL from Figs. 5a and 5b, respectively. Figure 6a was obtained by 420 iterations with RSE = 0.52 % and Fig. 6b by 102 iterations with RSE = 0.65 %. All simulation results by BIRL will be compared with these images.
4.2 Separated BIRL
4.2.1 4.2.1 Rectangular blocks for Gaussian deblurring
Table 1 shows results of separated BIRL with rectangular blocks in the Gaussian deblurring. For instance, computation times used for the block partition (â€˜BPâ€™ column), the single block iteration (â€˜SBâ€™ column), and the combining of deblurred block images (â€˜CBâ€™ column) were listed in Table 1. The smallest RSEs and their corresponding iteration numbers (â€˜INâ€™ column) were also listed in Table 1.
As â€˜the number of blocksâ€™ (hereafter abbreviated by NB) increased, computation times for the block partition (BP) and the combining of block images (CB) increased. Increments in computation times were, however, very small. The time for the single block iteration, however, linearly decreased as NB increased. There were also some slight increments in RSE and IN as NB increased.
Figure 7 shows deblurred images by separated BIRL with 4Ã—4 (a) and 16Ã—16 (b) rectangular blocks. Figure 7a was obtained at the 432th iteration with RSE = 0.53 % and Fig. 7b at the 513th iteration with RSE = 0.55 %. No noticeable block artifacts are shown in Fig. 7a, b. As differences in RSE results in Table 1 might indicate, Fig. 7b looks slightly smoother than Fig. 7a, while Fig. 7a looks slightly smoother than Fig. 6a. As shown in Fig. 7a, however, the degradation in Fig. 7a in the comparison with Fig. 6a is not big enough to give up the efficiency of separated BIRL in parallel computations. See results in the â€˜PCâ€™ column in Table 1.
Results in Table 1 and Fig. 7 show that separated BIRL with, at least up to 4Ã—4, rectangular blocks achieves the desired goal (combining deblurred block images to final images without block artifacts, while maintaining deblurring quality and approximation rate) in the Gaussian deblurring.
4.2.2 4.2.2 Diagonal blocks for diagonal deblurring
Table 2 shows the same data as Table 1 for diagonal blocks for diagonal deblurring. As in Table 1, the computation time for the single block iteration linearly decreases as NB increases, while there are some negligible increments in computation times for the block partition and the combining of deblurred block images. Unlike in Table 1, however, RSE and IN results are virtually unchanged as NB increases.
Figure 8 shows deblurred images by separated BIRL with 8 (a) and 16 (b) diagonal blocks. Figure 8a was obtained at the 103th iteration with RSE = 0.65 % and Fig. 8b at the 104th iteration with RSE = 0.65 %. Again, no noticeable boundary artifacts are shown in Fig. 8. As RSE results in Table 2 might indicate, visual differences in Figs. 6b and 8a, b are hardly noticeable.
Results in Table 2 and Fig. 8 show that separated BIRL with, at least up to 16, diagonal blocks achieves the desired goal (combining deblurred block images to final images without block artifacts, while maintaining deblurring quality and approximation rate) in the diagonal deblurring.
4.3 Interlaced BIRL
4.3.1 4.3.1 Rectangularlydownsampled blocks for Gaussian deblurring
Table 3 shows results of interlaced BIRL with rectangularlydownsampled blocks in the Gaussian deblurring. The computation time in the â€˜BIâ€™ column slightly increases as NB increases; one RL iteration took 4.40 s, while the one round of interlaced BIRL iteration with 4Ã—4 rectangularlydownsampled blocks took 4.84 s (see Table 3). The increment from 4.40 to 4.80 s was caused by updating pixel values on Î© _{ i } for all i=1,2,â€¦,16 (see step I4) in interlaced BIRL.
RSE results in Table 3 show that as NB increases, interlaced BIRL reaches its smallest RSE at an earlier iteration in the Gaussian deblurring (the iteration is accelerated by about NB times).
Figure 9 shows deblurred images by interlaced BIRL with 4Ã—4 (a) and 8Ã—8 (b) rectangularlydownsampled blocks. Figure 9a was obtained at the 26th iteration with RSE = 0.52 % and Fig. 9b at the second iteration with RSE = 0.91 %. The comparison with Fig. 6a, obtained at the 420th iteration with RSE = 0.52 % by RL, shows that interlaced BIRL with 4Ã—4 rectangularlydownsampled blocks (Fig. 9a) maintains the deblurring quality comparable to RL, while interlaced BIRL with 8Ã—8 rectangularlydownsampled blocks (Fig. 9b) produces severe ringing artifacts.
Results in Table 3 and Fig. 9 show that interlaced BIRL with up to 4Ã—4 rectangularlydownsampled blocks achieves the desired goal (the accelerated iteration, with deblurring quality maintained) in the Gaussian deblurring.
Poor results by interlaced BIRL with 8Ã—8 rectangularlydownsampled blocks for the Gaussian deblurring can be explained by the following argument. The performance of the ith interlaced BIRL subiteration (the step I4),
highly depends on the denominator \({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}}\) (often called normalization term in emission tomography). As mentioned earlier in Section 3.2, \(({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}})_{j_{1},j_{2}}\) can be interpreted as the probability with which the pixel (j _{1},j _{2})âˆˆÎ© _{ i } contributes the observation on Î› _{ i }. It is true that the value at the pixel (j _{1},j _{2})âˆˆÎ© _{ i } with bigger \(({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}})_{j_{1},j_{2}}\) is often recovered faster or more accurately than the value at the pixel with smaller one. Thus nonuniform and small \({\mathcal T}^{\ast }\textbf {I}_{\Lambda _{i}}\) often leads to slow convergence and a nonuniform deblurring effect in the ith interlaced BIRL subiteration.
Figure 10 shows \({\mathcal T}_{i}^{\ast }\textbf {I}_{\Lambda _{i}}\), where Î› _{ i } are one of pixel subsets formed by 4Ã—4 (a) and 8Ã—8 (b) rectangularlydownsampled blocks and \({\mathcal T}_{i}\) is the blurring transform associated with the Gaussian PSF k _{ G } and Î› _{ i }. The argument in the preceding paragraph implies that more uniform and higher intensities in Fig. 10a than in 10b give the main reason why Fig. 9a, deblurred by interlaced BIRL with denominators whose intensities look like Fig. 10a, is better than Fig. 9b, deblurred by interlaced BIRL with denominators whose intensities look like Fig. 10b.
4.3.2 4.3.2 Diagonallydownsampled blocks for diagonal deblurring
Table 4 shows the same data as Table 3 for diagonallydownsampled blocks for the diagonal deblurring. Again, as in Table 3, the computation time for the one round of interlaced BIRL iterations slightly increases as NB increases and interlaced BIRL reaches its smallest RSE at an earlier iteration, with the acceleration rate of NB times.
Figure 11 shows deblurred images by interlaced BIRL with 4 (a) and 8 (b) diagonallydownsampled blocks, and their zoomed parts in (c) and (d). Figure 11a was obtained at the 26th iteration with RSE = 0.65 % and Fig. 11b at the 12th iteration with RSE = 0.67 %. The comparison with Fig. 6b, obtained at the 102th iteration with RSE = 0.65 % by RL, shows that interlaced BIRL with 4 diagonallydownsampled blocks (Fig. 11a) maintains the deblurring quality comparable to RL, while interlaced BIRL with 8 diagonallydownsampled blocks exhibits some ringing artifacts (see zoomed parts Fig. 11c, d) with a slightly larger RSE.
Results in Table 4 and Fig. 11 show that interlaced BIRL with up to 6 diagonallydownsampled blocks achieves its desired goal (the accelerated iteration, with deblurring quality maintained) in the diagonal deblurring.
Interlaced BIRL with diagonallydownsampled blocks for the diagonal deblurring did not accelerate the iteration as much as interlaced BIRL with rectangularlydownsampled blocks did for the Gaussian deblurring. This phenomenon can be explained, again, by the uniformity and the largeness on denominators. Figure 12 shows \({\mathcal T}_{i}^{\ast }\textbf {I}_{\Lambda _{i}}\), where Î› _{ i } are one of pixel subsets formed by 4 (a) and 8 (b) diagonallydownsampled blocks and \({\mathcal T}_{i}\) is the blurring transform associated with the diagonal PSF k _{ D } and Î› _{ i }. The uniformity comparison between Figs. 10a and 12a gives a partial reason why the diagonal deblurring is not easy to be accelerated as much as the Gaussian deblurring by interlaced BIRL. It is also true that more uniform and higher intensities in Fig. 12a than in 12b makes Fig. 11a to be better than 11b.
4.4 Miscellaneous results
Simulation studies described so far were repeated with Gaussian and diagonal PSFs of size 11Ã—11, 35Ã—35, and 51Ã—51, different test images, different image sizes, and Gaussian noise models. We briefly report results of those simulation studies as follows.

In Gaussian deblurrings, as the size of Gaussian PSF increased, separated BIRL maintained the convergence rate of RL to smaller ranges of rectangular blocks.

In diagonal deblurrings, separated BIRL maintained the convergence rate of RL at least up to 16 diagonal blocks for all diagonal PSFs.

In Gaussian deblurrings, as the size of Gaussian PSF increased, interlaced BIRL accelerated iterations to wider ranges of rectangularlydownsampled blocks.

In diagonal deblurrings, as the size of diagonal PSF increased, interlaced BIRL accelerated iterations to wider ranges of diagonallydownsampled blocks.

Smoother images had more chance of exhibiting block artifacts in separated BIRL with rectangular blocks for Gaussian deblurrings. In separated BIRL with diagonal blocks for diagonal deblurrings, the test image did not affect block artifacts.

The use of Gaussian noise models itself did not affect the performance of proposed methods.

Deblurring with noisier data had less chance of exhibiting block artifacts in separated BIRL with rectangular blocks for Gaussian deblurrings. In separated BIRL with diagonal blocks for diagonal deblurrings, the noise level did not affect block artifacts.

Separated BIRL worked better for larger sized images, while the performance of interlaced BIRL did not depend on the image size.
We tested proposed methods with an image taken from a real camera. Figure 13a shows a blurred and noisy image taken by the famous photographer Robert Capa [30]. Figure 13b, c, d shows deblurred images by RL, separated BIRL with 4 horizontal blocks, and interlaced BIRL with 4 horizontallydownsampled blocks, respectively. Here, horizontal blocks for separated BIRL and horizontallydownsampled blocks for separated BIRL were selected by following two suggestions explained in Section 3.6 (the estimated PSF for Fig. 13a is known to be spread horizontally [30]).
Figure 13b, c shows 100th iterates of RL and separated BIRL with 4 horizontal blocks, respectively. No noticeable difference between Fig. 13b, c indicates that separated BIRL achieved its desired goal (the block artifactfree block deblurring) in this simulation. Figure 13d shows the 25th iterates of interlaced BIRL with 4 horizontallydownsampled blocks. Again, no noticeable difference between Fig. 13b, d indicates that interlaced BIRL achieved its desired goal (the accelerated iteration) in this simulation.
We tested proposed methods with an image taken from a real camera. Figure 13a shows a blurred and noisy image (of size 316Ã—480) taken by the famous photographer Robert Capa [30]. Figure 13b, c, d shows deblurred images by RL, separated BIRL with 4 horizontal blocks, and interlaced BIRL with 4 horizontallydownsampled blocks, respectively. Here, horizontal blocks for separated BIRL and horizontallydownsampled blocks for separated BIRL were selected by following two suggestions explained in Section 3.6 (the estimated PSF for Fig. 13a is known to be spread horizontally [30]; see Fig. 15a).
Figure 13b, c shows 100th iterates of RL and separated BIRL with 4 horizontal blocks, respectively. No noticeable difference between Fig. 13b, c indicates that separated BIRL achieved its desired goal (the block artifactfree block deblurring) in this simulation. Figure 13d shows the 25th iterates of interlaced BIRL with 4 horizontallydownsampled blocks. Again, no noticeable difference between Fig. 13b and 13d indicates that interlaced BIRL achieved its desired goal (the accelerated iteration) in this simulation.
We also tested proposed methods with a color image taken from a real camera. Figure 14a shows a color â€™summer houseâ€™ image of size 946Ã—952 in [30]. Figure 14b, c shows 100th iterates of RL and separated BIRL with 2Ã—2 rectangular blocks, respectively. No noticeable difference between Fig. 14b, c indicates that separated BIRL achieved its desired goal (the block artifactfree block deblurring) in this simulation. Figure 14d shows the 25th iterates of interlaced BIRL with 2Ã—2 rectangularlydownsampled blocks. Again, no noticeable difference between Fig. 14b, d indicates that interlaced BIRL achieved its desired goal (the accelerated iteration) in this simulation.
Figure 15 shows PSF images that were estimated from observed images, Figs. 13a for 15a and Figs. 14a for 15b, in [30].
We also conducted simulation studies on separated BIRL with â€˜improperly chosen blocks.â€™ Figure 16 shows deblurred images by separated BIRL with diagonal blocks for the Gaussian deblurring (a) and rectangular blocks for the diagonal deblurring (b). In combining deblurred block images, weights r ^{[i]}, i=1,â€¦,t, in (11) were used. Figure 16a did not show block artifacts, while Fig. 16b suffered from block artifacts.
Block artifacts in Fig. 16b were caused by boundary artifacts generated in diagonal deblurrings of rectangular blocks. In fact, the Gaussian deblurring also generated boundary artifacts for both rectangular and diagonal blocks. Those boundary artifacts were, however, confined only in outermost part of Î© _{ i }âˆ’Î› _{ i } and did not appear in the final image (Fig. 16a), since r ^{[i]} in (11) were very small for image pixels (j _{1},j _{2})âˆˆÎ© _{ i }âˆ’Î› _{ i } where boundary artifacts were strong. In diagonal deblurrings of rectangular blocks, however, boundary artifacts appeared in the final image (Fig. 16b) because of the exact opposite reason. The asymmetrical and slow decay of the diagonal PSF k _{ D } makes such difference.
On the other hand, diagonal deblurrings of diagonal blocks do not produce block artifacts, as shown in Fig. 8; most of boundary artifacts from diagonal deblurrings of diagonal blocks appear only in Î©âˆ’Î› and hence can be easily removed by cutting out pixels in Î©âˆ’Î›. These results show that, in case when the formula (11) is used for the combining of deblurred block images, the shape of blocks is important for the diagonal deblurring but not for the Gaussian deblurring.
Selecting blocks depending on the PSF is not an easy task. To deal with this problem, we suggest to use overlapped rectangular blocks Î› _{ i } for separated BIRL, in the case when it is not certain which blocks should be chosen.
Figure 17 shows deblurred images by separated BIRL with overlapped rectangular blocks for Gaussian (a) and diagonal (b) deblurrings. To be specific, Fig. 17a shows the deblurred image by separated BIRL with 16Ã—16 overlapped rectangular blocks. These overlapped blocks were formed from 16Ã—16 disjoint rectangular blocks (the size of each block is 30Ã—30) by adding nine rows or nine columns to boundaries of blocks.
Separated BIRL with these overlapped rectangular blocks produces 256 deblurred block images and combines 256 deblurred block images of size 30Ã—30, after cutting out required amount of rows and columns from 256 deblurred block images, to the final image (Fig. 17a). Figure 17b can be obtained by a similar procedure.
Both images in Fig. 17 do not show block artifacts (see Fig. 18, which shows zoomed parts of Figs. 16b and 17b, respectively, for a better comparison). These results indicate that overlapped rectangular blocks can be used for separated BIRL for any kind of PSFs, at the cost of additional computations caused by overlapped pixels; for the Gaussian deblurring with 4Ã—4 rectangular blocks, disjoint blocks took 0.275 s for the iteration for the one block, while 4Ã—4 overlapped rectangular blocks formed by adding nine pixel rows and columns to boundaries took 0.337 s.
As mentioned in Section 2, interlaced BIRL was compared with the acceleration technique in [7]. Figure 19a shows the result of the acceleration technique in [7], which obtained the smallest RSE 0.54 % at the 41st iteration in 182.4 s (= 4.45 s Ã— 41). On the other hand, Fig. 9a, obtained at the 26th iteration with RSE 0.52 % by interlaced BIRL with 4Ã—4 rectangularlydownsampled blocks, took 136.6 s (10.81 s for the block partition + 4.84 s Ã— 26 for iterations). The comparison of Fig. 19a with Fig. 9a shows that the technique in [7] provided a slightly slower acceleration and larger RSE than interlaced BIRL with 4Ã—4 rectangularlydownsampled blocks. It also shows that the technique in [7] produced slightly more ringing artifacts and noise amplification than interlaced BIRL. For a better visual comparison, see Fig. 19b, c, which shows zoomed parts of Figs. 19a and 9a, respectively.
5 Conclusions
In this paper, we extended RL to blockiterative versions, separated BIRL, and interlaced BIRL, for image deblurring applications. We conducted simulation studies to test proposed methods in Gaussian and diagonal deblurrings. Simulation results showed that separated BIRL can have a benefit of parallel computations for Gaussian and diagonal deblurrings, with a wide range of rectangular and diagonal blocks, respectively. Simulation results also showed that interlaced BIRL can accelerate the iteration for Gaussian and diagonal deblurrings but only with a limited range of rectangularly and diagonallydownsampled blocks, respectively.
In this work, proposed methods were tested only for Gaussian and diagonal PSFs, k _{ G } and k _{ D } in Fig. 3. It is clear that proposed methods can be extended to other PSFs as long as admissible blocks can be selected. But, as mentioned earlier, it is not clear how blocks should be formed for a specific PSF. Simulation results in Section 4 indicates that overlapped rectangular blocks can be used as universal admissible blocks for separated BIRL, at the cost of additional computations caused by overlapped pixels. For interlaced BIRL, however, such universal admissible blocks are not known yet.
References
AK Jain, Fundamentals of Digital Image Processing (PrenticeHall, Englewood Cliffs, NJ, 1989).
W Richardson, Bayesianbased iterative method of image restoration. J. Opt. Soc. Am. 62(1), 55â€“59 (1972).
L Lucy, An iterative techniques for the rectification of observed distributions. Astronomical J. 79(6), 745â€“754 (1974).
M Bertero, P Boccacci, G DesiderÃ¡, G Vicidomini, Image deblurring with Poisson data: from cells to galaxies. Inverse Probl. 25, 123006 (2009).
YW Tai, P Tan, M Brown, RichardsonLucy deblurring for scenes under a projective motion path. IEEE Trans. PAMI. 33(8), 1603â€“1618 (2011).
H Hudson, R Larkin, Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans. Med. Imaging. 13(4), 601â€“609 (1994).
D Biggs, M Andrews, Acceleration of iterative image restoration algorithms. Appl. Optics. 36(8), 1766â€“1775 (1997).
L Yuan, J Sun, L Quan, H Shum, Progressive interscale and intrascale nonblind image deconvolution. ACM Trans. Graph. 27(3), 74â€“17410 (2008).
S Setzer, G Steidl, T Teuber, Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21(3), 193â€“199 (2010).
Y Wang, H Feng, Z Xu, Q Li, C Dai, An improved RichardsonLucy algorithm based on local prior. Opt. Laser Technol. 42(5), 845â€“849 (2010).
M TemerinacOtt, Tilebased LucyRichardson deconvolution modeling a spatiallyvarying PSF for fast multiview fusion of microscopical images (Technical report 260, University of Freiburg, 2010). http://lmb.informatik.unifreiburg.de//Publications/2010/Tem10a.
JL Wu, CF Chang, CS Chen, An adaptive RichardsonLucy algorithm for single image deblurring using local extrema filtering. J. Appl. Sci. Eng. 16(3), 269â€“276 (2013).
L Shepp, Y Vardi, Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging. 1(2), 113â€“122 (1982).
A Dempster, N Laird, D Rubin, Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Series B. 39(1), 1â€“38 (1977).
NY Lee, Y Choi, A modified OSEM algorithm for pet reconstruction using wavelet processing. Comput. Methods Prog. Biomed. 80(3), 236â€“245 (2005).
A Reilhaca, S TomeÃ¯a, I Buvatb, C Michel, F Keherenc, N Costesa, Simulationbased evaluation of OSEM iterative reconstruction methods in dynamic brain pet studies. NeuroImage. 39(1), 359â€“368 (2008).
M Bertero, P Boccacci, A simple method for the reduction of boundary effects in the Richardsonâ€“Lucy approach to image deconvolution. Astron. Astrophys. 437, 369â€“374 (2005).
B Anconelli, M Bertero, P Boccacci, G DesiderÃ¡, M Carbillet, H Lanteri, Deconvolution of multiple images with high dynamic range and an application to LBT LINCNIRVANA. Astron. Astrophys. 460, 349â€“355 (2006).
H Adorf, R Hook, L Lucy, F Murtagh, Accelerating the RichardsonLucy restoration algorithm. Proc. 4th ESO/STECF Data Analysis Workshop, European Southern Observatory, 99â€“103 (1992).
T Holmes, Y Liu, Acceleration of maximumlikelihood image restoration for fluorescence microscopy and other noncoherent imagery. J. Opt. Soc. Am. A. 8(6), 893â€“907 (1991).
L Kaufman, Implementing and accelerating the EM algorithm for positron emission tomography. IEEE Trans. Med. Imaging. 6(1), 37â€“51 (1997).
A Tekalp, M Sezan, Quantitative analysis of artifacts in linear spaceinvariant image restoration. Multidimensional Syst. Signal Process. 1, 143â€“177 (1990).
M Ng, R Chan, WC Tang, A fast algorithm for deblurring models with Neumann boundary conditions. SIAM J. Sci. Comput. 21(3), 851â€“866 (1990).
S SerraCapizzano, A note on antireflective boundary conditions and fast deblurring models. SIAM J. Sci. Comput. 25(4), 1307â€“1325 (2003).
M Donatelli, S SerraCapizzano, Antireflective boundary conditions and reblurring. Inverse Probl. 21(1), 169â€“182 (2005).
M Donatelli, C Estatico, A Martinelli, S SerraCapizzano, Improved image deblurring with antireflective boundary conditions and reblurring. Inverse Probl. 22(6), 2035â€“2053 (2006).
R Liu, J Jia, Reducing boundary artifacts in image deconvolution. IEEE Int. Conf. Image Process.  ICIP., 505â€“508 (2008).
D Calvetti, J Kaipio, E Somersalo, Aristotelian prior boundary conditions. Int. J. Math. Comput. Sci. 1, 63â€“81 (2006).
NY Lee, B Lucier, Preconditioned conjugate gradient method for boundary artifactfree image deblurring. Technical report, Carnegie Mellon University, Department of Mathematical Sciences (2013). http://www.math.cmu.edu/CNA/Publications/publications2013/011abs/13CNA011.pdf.
Deblur Famous/Interesting Photos. http://www.juew.org/deblurFamousPhoto.html.
Acknowledgements
This work was supported by the Inje Research and Scholarship Foundation in 2012 and conducted while the author was a visiting scholar at the Center for Nonlinear Analysis, Department of Mathematical Sciences, Carnegie Mellon University. The author would like to express his sincere appreciation to Prof. Lucier at Purdue University for his helpful suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The author declares that he has no competing interests.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Lee, NY. Blockiterative RichardsonLucy methods for image deblurring. J Image Video Proc. 2015, 14 (2015). https://doi.org/10.1186/s1364001500692
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s1364001500692
Keywords
 Parallel computation
 Block artifacts
 Ordered subsets
 Accelerated iteration