Variational segmentation model for images with intensity inhomogeneity and Poisson noise

Abstract

In this paper, we propose a variational segmentation model to deal with intensity inhomogeneity and Poisson noise. An energy functional is first proposed, which uses a data-fidelity term deduced from Poisson distribution instead of the usual L ₂ norm as a measure of fidelity. Due to the new data-fidelity measure, this energy functional can fit the image intensity more accurately while it can diminish the influence of Poisson noise on segmentation results. We then reformulate the energy function as globally convex formulation, which allows for more flexible initialization. The final convex energy functional is minimized via the dual formulation instead of the usually used gradient descent method. Experimental results show that the proposed model can efficiently segment images with intensity inhomogeneity and Poisson noise.

1 Introduction

Image segmentation is one of the fundamental and important tasks in image analysis and computer vision. The segmentation problem can be formulated as follows: given an image I ∈ L²(Ω) on a two-dimensional domain Ω (assumed to be bounded, smooth, and open), one seeks out a closed ‘edge set’ C and all the connected components Ω_1,…,Ω _k of Ω\C, so that by certain suitable visual measure, the image I is discontinuous along C while smooth or homogeneous on each segment Ω _i (i = 1, …, k). Until now, a wide variety of techniques including variational methods [1] has been proposed to solve the image segmentation problem.

The variational segmentation methods are characterized by deriving an energy functional from some a priori mathematical model and minimizing this energy functional over all possible partitions. The Mumford-Shah model [2] is a classical variational segmentation method, which contains a data-fidelity term, regularization on the model, and regularization on the partitioning. Based on this framework, Chan and Vese [3] developed the frequently used variational segmentation model, which simplifies the Mumford-Shah model to the case of piecewise constant approximations of the image intensity. However, the Chan-Vese model tends to rely on global information to guide contour evolution, and thus fails to segment the images with intensity inhomogeneity [4, 5]. Besides, each region (foreground or background) in Chan-Vese model is also considered as a Gaussian distribution with different mean and same deviation [6]; therefore, they are not suitable for images with Poisson noise.

Intensity inhomogeneity often occurs in real images, especially in medical images, such as X-ray radiography/tomography and magnetic resonance (MR) images [4, 5, 7]. The intensity inhomogeneity usually refers to the slow, non-anatomic intensity variations of the same tissue over the image domain. For example, in MR images, it often appears as an intensity variation across the image, which arises from radio frequency coils as well as variations in object susceptibility. Although the presence of intensity inhomogeneity is usually hardly noticeable to a human observer, variational segmentation models such as the Chan-Vese model [3] are highly sensitive to the spurious. Thus, segmentation of such medical images usually requires intensity inhomogeneity correction as a preprocessing step [8].

Poisson noise also appears in a wide class of real-world applications, e.g., positron emission tomography in medical imaging [9], fluorescence microscopy [10] and radiography images [11, 12]. It is signal-dependent and obeys a Poisson distribution, which describes a signal-dependent perturbation of an image. In particular, the radiograph images are typical images with Poisson noise, which are determined by photon counting statistics and are described as particle-limited, emphasizing the quantized; furthermore, due to technical limitations or artifacts introduced by the object being imaged, the radiograph images such as X-ray is often created with intensity inhomogeneity. In [13], Lee et al. studied the segmentation problem of images with Poisson noise. However, this model was based on intensity homogeneous (roughly constant) statistics and, thus, it was not able to deal with intensity inhomogeneity.

In order to handle intensity inhomogeneity, Li et al. [5] proposed the region-scalable fitting (RSF) model (originally termed as local binary fitting (LBF) model [4]) in a variational level set formulation. In the RSF model, the RSF energy is defined over the neighborhood of each image pixel, and the active contour is deformed to minimize the integration of the RSF energy over the whole image domain. With two extra regularization terms, this final energy minimization problem is converted to solve a level set evolution equation by using the gradient descent method. The RSF model can deal with intensity inhomogeneity accurately, but it is quite sensitive to contour initialization. Because the gradient descent is a very slow numerical method, the RSF model generally converges slowly when implemented numerically. Besides, the RSF model is derived from the Mumford-Shah model [2] that implicitly assumes the given image to be biased by additive Gaussian noise; thus it is not suitable for images with signal-dependent noise (e.g., Poisson noise).

Following the RSF variational model, various extensions and analysis have been studied [7, 14–16], among others. In [7], Wang et al. proposed a local and global intensity fitting model in a variational level set formulation. Zhang et al. [14] proposed a local image fitting (LIF) model by minimizing the difference between an original image and the fitted image. He et al. [15] presented a scheme of improvement on the RSF model in terms of robustness to initialization and noise. Wang et al. [16] proposed a novel algorithm by using a piecewise smooth approximation to image. However, these models have some common drawbacks; in particular, they are not suitable to segment images with Poisson noise because these models are also formulated in view of the Mumford-Shah energy functional [2].

In this study, we propose a variational model to segment images with intensity inhomogeneity and Poisson noise. Based on the RSF model [5] and inspired from variational Poisson denoising model [17], we first propose a new energy functional with the data-fidelity term deduced from Poisson distribution, which is more suitable for images corrupted by Poisson noise. We then reformulate this energy functional into a convex formulation to guarantee the global minima. We also use the weighted total variation (TV) norm as the regularization term to detect the boundaries more easily. Finally, in order to implement the proposed model, we adopt the dual formulation which is introduced by Chan et al. [18] and Chambolle [19] for denoising and is later adapted by Bresson et al. in [20] for segmentation.

The remainder of this paper is organized as follows. In Section 2, we briefly review the level set method, two denoising models [17, 21] and the RSF model [5]. The proposed model is introduced in Section 3. The numerical results are given in Section 4. This paper is concluded in Section 5.

2 Related works

2.1 Level set method

According to the level set method [22], a closed curve C(t) is represented implicitly by the zero level set of a Lipschitz function ϕ (x, t), called a level set function, with the following properties:

$$\left\{\begin{array}{l}\varphi \left(x,t\right)>0,x\in \mathrm{inside}\left(C\right)\\ \varphi \left(x,t\right)=0,x\in C\\ \varphi \left(x,t\right)<0,x\in \mathrm{outside}\left(C\right)\end{array}\right.$$

(1)

Evolving the curve C in normal direction with speed F amounts to solving the following level set evolution equation:

$$\frac{\partial \varphi }{\partial t}=F\left|\nabla \varphi \right|,$$

(2)

with the initial condition ϕ (x, 0) = ϕ ₀(x).

Alternatively, the evolution equation for the level-set function also can be directly derived from the minimization problem for the energy functional over the level set functions via the gradient descent [3–7]:

$$\frac{\partial \varphi }{\partial t}=F{\delta }_{ϵ}\left(\varphi \right),$$

(3)

where δ _ϵ (ϕ) is the smooth Dirac function, which is the derivative of the smooth Heaviside function. A segmentation of the image is given by the two regions {x|ϕ(x, t) > 0} and {x|ϕ(x, t) < 0}. The steady state solution of the Equation 3 hopefully gives a useful edge contour extraction or segmentation of the image.

2.2 Two related denoising models

A very successful image denoising model is that of Rudin, Osher, and Fatemi (ROF) model [21] which uses TV regularization with data-fidelity term. Let $I:\Omega \subset {\mathbb{R}}^2\to \mathbb{R}$ be a given gray image defined on image domain Ω, the ROF model minimizes the energy functional as

$$\underset{u\in \mathit{BV}\left(\Omega \right)}{\min }\left\{{\displaystyle {\int }_{\mathrm{\Omega }}\left|\nabla u\right|}\mathit{dx}+\mu {\displaystyle {\int }_{\mathrm{\Omega }}{\left|I-u\right|}^2\mathit{dx}}\right\},$$

(4)

where BV(Ω) is the space of functions of bounded variation defined on Ω, and μ > 0 is a parameter to be chosen.

In (4), ∫_Ω| ∇ u|dx is the TV regularization term to remove the noise, and ∫  _Ω |I − u|² dx is the L ₂ norm data-fidelity term that measures the dissimilarity between original image I and the reconstructed image u. It is shown in [23] using probability arguments that the L ₂ norm data-fidelity term is most appropriate for removing additive Gaussian noise. However, many important data contain noise that is signal dependent and obeys a Poisson distribution; thus removing this noise without losing image features requires a data-fidelity term reflecting the noise characteristics.

In [17], Le et al. proposed a variational model for denoising images with Poisson noise:

$$\underset{u\in \mathit{BV}\left(\Omega \right)}{\min }\left\{{\displaystyle {\int }_{\mathrm{\Omega }}\left|\nabla u\right|}\mathit{dx}+\mu {\displaystyle {\int }_{\mathrm{\Omega }}\left(u-Ilog\left(u\right)\right)\mathit{dx}}\right\}.$$

(5)

The energy in (5) differs from the energy functional of ROF model only in the data-fidelity term; the L ₂ norm data-fidelity of the original model is replaced by F(u) = ∫_Ω(u − I log(u))dx as a measure of fidelity. We refer the reader to [17] for the derivation of this data-fidelity term using Bayesian statistics. As pointed out in [17], F(u) is more suitable for Poisson noise. Recently, Chartrand and Staneva [24] have shown that the data-fidelity term reflecting the noise characteristics of the image can provide a better image reconstruction.

2.3 Region scalable fitting model

Let $I:\Omega \subset {\mathbb{R}}^2\to \mathbb{R}$ be a given gray image defined on image domain Ω. Let C be a closed contour in Ω, which separates Ω into two regions: Ω₁ = inside(C) and Ω₂ = outside(C). For a given point x ∈ Ω, the RSF energy is defined as [4, 5]

$${ϵ}_x^{\mathrm{RSF}}\left(C,f_1\left(x\right),f_2\left(x\right)\right)={\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in {\mathrm{\Omega }}_i}{K}_{\sigma }\left(x-y\right){\left|I\left(y\right)-f_i\left(x\right)\right|}^2}}\mathit{dy}$$

(6)

where K _σ (z) is a kernel function.

To obtain the entire object boundary, it is required to find a contour C that minimizes the energy ${ϵ}_x^{\mathrm{RSF}}$ for all x ∈ Ω. This can be achieved by minimizing the integral of ${ϵ}_x^{\mathrm{RSF}}$ over Ω:

$${ϵ}^{\mathrm{RSF}}\left(C,f_1,f_2\right)={\displaystyle {\int }_{\mathrm{\Omega }}{ϵ}_x^{\mathrm{RSF}}\left(C,f_1\left(x\right),f_2\left(x\right)\right)}\mathit{dx}$$

(7)

In addition, it is necessary to smooth the contour by penalizing its length. Therefore, the following energy functional is defined in [5]:

$$ϵ\left(C,f_1,f_2\right)={ϵ}^{\mathrm{RSF}}\left(C,f_1,f_2\right)+\nu \left|C\right|$$

(8)

To handle topological changes, the energy ϵ(C, f ₁, f ₂) is incorporated into a variational level set formulation with two extra internal energy functionals. They then use the gradient descent method to solve the variational level set formulation.

The RSF model can deal with intensity inhomogeneity accurately and efficiently; however, it easily gets stuck in local minima for most of the contour initializations. This makes the RSF model sensitive to the selection of initial contours. Besides, the RSF model uses L ₂ norm to measure the difference between the fitted image and the original image; thus, it is not suitable for images with Poisson noise.

3 The proposed model

3.1 Intensity fitting energy and its level set formulation

In this section, we first define an intensity fitting energy for a given contour C and then change it into the energy directly defined on level set functions.

Let $I:\Omega \subset {\mathbb{R}}^2\to \mathbb{R}$ be a gray image and C be a closed contour in Ω with inside(C) = Ω ₁ and outside(C) = Ω ₂. For any x ∈ Ω, the local fitting energy (LFE) of the contour C at x is defined as

$$E_x^{\mathrm{LFE}}\left(C,h_1\left(x\right),h_2\left(x\right)\right)={\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in {\mathrm{\Omega }}_i}{K}_{\sigma }\left(x-y\right)\left[h_i\left(x\right)-I\left(y\right)log\left(h_i\left(x\right)\right)\right]}}\mathit{dy},$$

(9)

where h _i (x)(i = 1, 2) are two values that approximate locally image intensities in Ω₁ and Ω₂, respectively.

In order to extract the entire boundary of object, we compute the integral of the energy $E_x^{\mathrm{LFE}}\left(C,h_1\left(x\right),h_2\left(x\right)\right)$ over Ω:

$$\begin{array}{l}E\left(C,h_1,h_2\right)\\ ={\displaystyle {\int }_{\Omega }{E}_x^{\mathrm{LFE}}\left(C,h_1\left(x\right),h_2\left(x\right)\right)}\mathit{dx},\\ ={\displaystyle {\int }_{x\in \Omega }\left[{\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in {\mathrm{\Omega }}_i}{K}_{\sigma }\left(x-y\right)\left[h_i\left(x\right)-I\left(y\right)log\left(h_i\left(x\right)\right)\right]}}\mathit{dy}\right]}\mathit{dx}\end{array}$$

(10)

which is called the intensity fitting energy (IFE) of the contour C in this paper.

To allow contour splitting and merging naturally (i.e., a change of topology), we use a level set function to represent a contour C. The IFE functional of the contour C is thus changed into the energy functional which is directly defined on the level set functions.

Let ϕ be a level set function, and then the IFE functional in (10) can be expressed as

$$\begin{array}{l}E\left(\varphi ,h_1,h_2\right)\\ ={\displaystyle {\int }_{x\in \Omega }\left[{\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in \Omega }{K}_{\sigma }\left(x-y\right)\left[h_i\left(x\right)-I\left(y\right)log\left(h_i\left(x\right)\right)\right]}}{M}_i\left(\varphi \left(y\right)\right)\mathit{dy}\right]}\mathit{dx}\end{array}$$

(11)

where H(z) is the Heaviside function, M ₁(ϕ) = H(ϕ), M ₂(ϕ) = 1 − H(ϕ).

In practice, the Heaviside function H(z) is approximated by a smooth function H _ϵ (z), which is typically defined by [3, 5]

$$H_{ϵ}\left(z\right)=\frac{1}{2}\left[1+\frac{2}{\pi }arc\tan \left(\frac{z}{ϵ}\right)\right],$$

(12)

where ϵ is a positive constant. Therefore, the IFE functional in (11) is rewritten as

$$\begin{array}{l}E\left(\varphi ,h_1,h_2\right)\\ ={\displaystyle {\int }_{x\in \Omega }\left[{\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in \Omega }{K}_{\sigma }\left(x-y\right)\left[h_i\left(x\right)-I\left(y\right)log\left(h_i\left(x\right)\right)\right]}}{M}_i^{ϵ}\left(\varphi \left(y\right)\right)\mathit{dy}\right]}\mathit{dx}\end{array}$$

(13)

where $M_1^{ϵ}\left(\varphi \right)=H_{ϵ}\left(\varphi \right)$, $M_2^{ϵ}\left(\varphi \right)=1-H_{ϵ}\left(\varphi \right)$.

3.2 Description of the model

The IFE functional in (13) is not convex with respect to ϕ ; hence, the corresponding optimization problem can sometimes get stuck in undesirable local minima. In this section, we propose to solve this problem by determining a globally convex formulation. Our idea comes from [25], which reformulated the piecewise constant Chan-Vese model [3] into a convex model.

In the following, we first derive the gradient descent flow equation of (13) and then find a simplified flow which has the coincident steady state solution with the original gradient descent flow equation and finally define a new and convex energy in view of the simplified flow.

Fixing ϕ , we minimize the functional E(ϕ, h ₁, h ₂) with respect to the functions h ₁(x) and h ₂(x). By the calculus of variations, it can be shown that the functions h ₁(x) and h ₂(x) that minimize E(ϕ, h ₁, h ₂) satisfy the following Euler-Lagrange equations:

$${\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)\left[1-\frac{I\left(y\right)}{h_i\left(x\right)}\right]M_i^{ϵ}\left(\varphi \left(y\right)\right)\mathit{dy}}=0,i=1,2$$

(14)

From (14), we obtain

$$h_i\left(x\right)=\frac{{\displaystyle \int K_{\sigma }\left(x-y\right)I\left(y\right)M_i^{ϵ}\left(\varphi \left(y\right)\right)\mathit{dy}}}{{\displaystyle \int K_{\sigma }\left(x-y\right)M_i^{ϵ}\left(\varphi \left(y\right)\right)\mathit{dy}}},i=1,2$$

(15)

We derive the gradient descent flow equation of (13) with respect to ϕ. Making the interchange of variables x and y (i.e., x = y, y = x) for (13), we have

$$\begin{array}{l}E\left(\varphi ,h_1,h_2\right)\\ ={\displaystyle {\int }_{y\in \Omega }\left[{\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{x\in \Omega }{K}_{\sigma }\left(y-x\right)\left[h_i\left(y\right)-I\left(x\right)log\left(h_i\left(y\right)\right)\right]}}{M}_i^{ϵ}\left(\varphi \left(x\right)\right)\mathit{dx}\right]}\mathit{dy}.\end{array}$$

(16)

Interchanging the order of integration and noting that K _σ (−z) = K _σ (z), we obtain

$$\begin{array}{l}E\left(\varphi ,h_1,h_2\right)\\ ={\displaystyle {\int }_{x\in \Omega }\left[{\displaystyle \sum _{i=1}^2{\displaystyle {\int }_{y\in \Omega }{K}_{\sigma }\left(x-y\right)\left[h_i\left(y\right)-I\left(x\right)log\left(h_i\left(y\right)\right)\right]}}{M}_i^{ϵ}\left(\varphi \left(x\right)\right)\mathit{dy}\right]}\mathit{dx}.\end{array}$$

(17)

Fixing h ₁(y) and h ₂(y) by minimizing (17) with respect to ϕ using the gradient descent method, we can obtain the gradient descent flow equation of (13) as

$$\frac{\partial \varphi }{\partial t}={\delta }_{ϵ}\left(\varphi \right)F\left(x\right)$$

(18)

with

$$\begin{array}{l}F\left(x\right)=-{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)\left[h_1\left(y\right)-I\left(x\right)log\left(h_1\left(y\right)\right)\right]\mathit{dy}}\\ +{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)\left[h_2\left(y\right)-I\left(x\right)log\left(h_2\left(y\right)\right)\right]}\mathit{dy},\end{array}$$

(19)

where δ _ϵ (z) = H′ _ϵ (z) = ϵ/π(ϵ² + z²) is a smooth Dirac function.

Since δ _ϵ (z) is a non-compactly supported function, the gradient flow Equation 18 and the following equation have the same stationary solutions:

$$\frac{\partial \varphi }{\partial t}=F\left(x\right)$$

(20)

Based on (20), we define a convex energy functional as

$$E\left(\varphi \right)={\displaystyle {\int }_{\Omega }\varphi \left(x\right)F(x})\mathit{dx},$$

(21)

which is clearly the energy functional associated with the gradient descent flow (20).

It is necessary to smooth the level set function ϕ to avoid the occurrence of small, isolated regions (e.g., noise points) in the final segmentation results. Most of models focus on the regularizations as length regularization [3], mean curvature regularization [22], and H¹ regularization [26]. In this study, we employ the weighted TV norm [20] as the regularization term:

$${\mathbf{TV}}_g\left(\varphi \right)={\displaystyle {\int }_{\mathrm{\Omega }}g\left(x\right)\left|\nabla \varphi \left(x\right)\right|}\mathit{dx}$$

(22)

where g(x) = 1/(1 + β|∇I|²) is the edge detector function, and β is a non-negative parameter. The weighted TV norm can better smooth the level set function and makes the model detect boundaries more easily. Thus, we obtain the following energy functional

$$E\left(\varphi \right)={\displaystyle {\int }_{\mathrm{\Omega }}g\left(x\right)\left|\nabla \varphi \left(x\right)\right|}\mathit{dx}+\lambda {\displaystyle {\int }_{\mathrm{\Omega }}\varphi \left(x\right)F(x})\mathit{dx}$$

(23)

where λ > 0 is a parameter. The energy E(ϕ) is a convex functional (not strictly), but it does not have a stationary solution because of homogeneous of degree one in ϕ; therefore, we need to restrict the solution to lie in a finite interval such as [0, 1].

With the constraints in ϕ, we arrive at the entire model as follows:

$$\underset{0\le \varphi \le 1}{\min }E\left(\varphi \right)=\underset{0\le \varphi \le 1}{\min }\left\{{\displaystyle {\int }_{\mathrm{\Omega }}g\left(x\right)\left|\nabla \varphi \left(x\right)\right|}\mathit{dx}+\lambda {\displaystyle {\int }_{\mathrm{\Omega }}\varphi \left(x\right)F(x})\mathit{dx}\right\}.$$

(24)

3.3 Minimizing the proposed model by dual formulation

We solve the constrained minimization problem (24) by the dual formulation presented in [18, 19] instead of the usually used gradient descent method. The dual formulation is proposed by Chan et al. [18] and Chambolle [19] to solve the ROF minimization problem (4) for image denoising and is later adopted in [20] for image segmentation.

By claim 1 in [25], we first change the constrained problem (24) into the following unconstrained problem:

$$\underset{\varphi }{\min }\left\{{\displaystyle {\int }_{\mathrm{\Omega }}g\left(x\right)\left|\nabla \varphi \left(x\right)\right|\mathit{dx}}+{\displaystyle {\int }_{\mathrm{\Omega }}\left[\mathit{\lambda \varphi }\left(x\right)F\left(x\right)+\mathit{\alpha \psi }\left(\varphi \left(x\right)\right)\right]\mathit{dx}}\right\}$$

(25)

with $\psi \left(z\right)=max\left\{0,2|z-\frac{1}{2}|-1\right\}$ .

Then we introduce one auxiliary variable φ and propose to minimize the following approximation to (25):

$$\begin{array}{l}\underset{\varphi ,\phi }{\min }\{{\displaystyle {\int }_{\mathrm{\Omega }}g\left(x\right)\left|\nabla \varphi \left(x\right)\right|\mathit{dx}}+\frac{1}{2\theta }{{\displaystyle {\int }_{\mathrm{\Omega }}\left[\varphi \left(x\right)-\phi \left(x\right)\right]}}^2\mathit{dx}\\ +{\displaystyle {\int }_{\mathrm{\Omega }}\left[\mathit{\lambda \phi }\left(x\right)F\left(x\right)+\mathit{\alpha \psi }\left(\phi \left(x\right)\right)\right]\mathit{dx}}\}\end{array}$$

(26)

with θ > 0. One can note that if θ → 0, the functional (26) is exactly the minimization problem (25). Moreover, the functional in (26) is still convex, which means that we can compute its global minimizer. However, (26) is a minimization problem in two variables, thus, we have to perform an alternating minimization respect to ϕ and φ. The solving process of minimization is described in the following two steps:

In the first step, with φ being fixed, the solution of (26) is

$$\varphi =\phi -\theta \mathrm{div}p,$$

(27)

in which the dual variable p = (p¹, p²) satisfies the following equation:

$$g\left(x\right)\nabla \left(\theta \mathrm{div}p-\phi \right)-\left|\nabla \left(\theta \mathrm{div}p-\phi \right)\right|p=0.$$

(28)

The above Equation 28 is solved by a fixed point method: setting the step τ ∈ [0,  1/8] and p⁰ = 0,

$$p^{n+1}=\left[p^n+\tau \nabla \left[\mathrm{div}\left(p^n\right)-\frac{\phi }{\theta }\right]\right]{\left[1+\frac{\tau }{g\left(x\right)}\left|\nabla \left(\mathrm{div}\left(p^n\right)-\frac{\phi }{\theta }\right)\right|\right]}^{-1},$$

(29)

where div and ∇ are the discrete divergence and gradient operators [19], respectively:

$${\left(divp\right)}_{i,j}=\left\{\begin{array}{l}{p}_{i,j}^1-p_{i-1,j}^1,\mathrm{if}1<i<M\\ p_{i,j}^1,\mathrm{if}i=1\\ -p_{i-1,j}^1,\mathrm{if}i=M\end{array}\right.+\left\{\begin{array}{l}{p}_{i,j}^2-p_{i,j-1}^2,\mathrm{if}1<j<N\\ p_{i,j}^2,\mathrm{if}j=1\\ -p_{i,j-1}^2,\mathrm{if}j=N\end{array}\right.$$

(30)

$${\left(\nabla u\right)}_{i,j}=\left({\delta }_x^{+}{u}_{i,j},{\delta }_y^{+}{u}_{i,j}\right)$$

(31)

with

$$\begin{array}{l}{\delta }_x^{+}{u}_{i,j}=\left\{\begin{array}{l}{u}_{i+1,j}-u_{i,j},\mathrm{if}1\le i<M\\ 0,\mathrm{if}i=M\end{array}\right.,\\ {\delta }_y^{+}{u}_{i,j}=\left\{\begin{array}{l}{u}_{i,j+1}-u_{i,j},\mathrm{if}1\le j<N\\ 0,\mathrm{if}j=N\end{array}\right..\end{array}$$

(32)

In the second step: with ϕ being fixed, the solution of (26) is

$$\phi =\min \left\{max\left\{\varphi \left(x\right)-\mathit{\theta \lambda F}\left(x\right),0\right\},1\right\}$$

(33)

4 Numerical results

The proposed model has been tested with synthetic and real images from different modalities. The level set function ϕ is simply initialized as a binary step function taking 1 inside a region and 0 outside. The set M = {x|ϕ(x) > 0.5} is used to extract the objects. Besides, unless otherwise specified, we use the following default parameter setting: the time step Δt = 1/8, the space step Δx = Δy = 1, σ = 5, β = 50/255², θ = 0.1, λ = 1. We record the iteration number and the CPU time from our experiments with Matlab codes run on an PC, with AMD Athlon (tm) 2.70GHz CPU, 2.00 GB memory, and Matlab 7.4 on Windows 7.

First, we show the segmentation process of our model for five typical images with intensity inhomogeneity. The five images, which are plotted in Figure 1a,f,k,p,u, are a synthetic image, a real image with T-shaped object, two blood vessel images, and a real potatoes’ image, respectively. The synthetic image is a well-known image with intensity inhomogeneity. The T-shaped image is a real image with intensity inhomogeneity due to non-uniform illumination. For the two blood vessel images, some vessel boundaries are also quite weak. The potatoes’ image is a multiple objects image corrupted by intensity inhomogeneity. The first three images (Figure 1a,f,k) have been used in the RSF model. Due to the intensity inhomogeneity, the piecewise constant models such as the modes [3, 13] cannot segment these images. As shown in Figure 1, our model successfully extracts the object boundaries for these challenging images. The results shown in Figure 1 using our model are very similar to those of the RSF model [5]; however, our model has less iteration numbers and computation (CPU) times than the RSF model for the five images (see Table 1). It is clear that our model is more efficient than the RSF model. It is guaranteed by the following reasons: first, our proposed energy functional is convex, which can result in the fast convergence; second, we apply the dual formulation to the optimization problem, which enables convergence faster.

Figure 1

Segmentation results of the proposed model for five synthetic and real images with intensity inhomogeneity. First column (a,f,k,p,u) are the original images with the initial contours. (a-y) The curve evolution process from the initial contours (in the first column) to the final contours (in the fifth column) is shown in every row for the corresponding image.

Table 1 Iterations and CPU time (in seconds) by proposed and RSF models for Figure 1

Second, we demonstrate the robustness of the proposed model to the Poisson noise in Figure 2. Figure 2a is the original image (115 × 115) that contains only two distinct gray levels, Figure 2e was generated by adding intensity inhomogeneity to the original image, and Figure 2i was created by adding Poisson noise to the second image (Figure 2e). From the second column, we can clearly see that our model can handle these three images very well even for the image with intensity inhomogeneity (Figure 2f) or intensity inhomogeneity and Poisson noise (Figure 2j). To clearly see the profile of the proposed model, we plot the 1-D cross sections at the middle column of h ₁(x), h ₂(x), $u\left(x\right)={\displaystyle {\sum }_{i=1}^2{M}_i}\left(\varphi \left(x\right)\right)h_i\left(x\right),$ and the original image I(x). Column 3 shows that the fitting functions h ₁(x) and h ₂(x) are smooth enough for the three images without or with Poisson noise. In addition, the final fitting image u(x) can better fit the original image I(x) (see Figure 2d,h); in particular, the fitting image u(x) shown in Figure 2l is similar to the one given in Figure 2h, which shows that our model can more fit the original image while it can diminish the influence of Poisson noise. The segmentation results, the fitting functions (h ₁(x), h ₂(x)), and the final fitting image u(x) demonstrate the robustness of the proposed model to the Poisson noise.

Figure 2

Segmentation results of the proposed model for three synthetic images. First column (a,e,i), the original images with the initial contours. Second column (b,f,j), the final segmentation results. Third column (c,g,k), 1-D cross sections of h ₁(x) (blue dashed line), h ₂(x)(green dashed line) associated with the original image (black solid line). Fourth column (d,h,l), 1-D cross sections of u(x) (red dashed line) associated with the original image (black solid line).

Third, we apply the proposed and RSF models to two synthetic images with Poisson noise (see Figures 3 and 4) and four real radiograph images (see Figure 5). It should be pointed out, the results of RSF model depends on the regularization parameter v; we choose the optimal values of the parameter v by using a ‘coarse-to-fine’ scheme on the test images.

Figure 3

Segmentation results of the proposed and RSF models to a synthetic image with Poisson noise. (a) The original image, (b) the initial contour, (c) the result of the RSF model with regularization parameter v = 0.004 × 255², (d) the result of the proposed model, (e) final level set function of the RSF model (270 iterations), and (f) final level set function of the proposed model (100 iterations).

Figure 4

Segmentation results of the proposed and RSF models to a synthetic image with intensity inhomogeneity and Poisson noise. (a) The original image, (b) the initial contour, (c) the result of the RSF model with regularization parameter v = 0.003 × 255², (d) the result of the proposed model, (e) final level set function of the RSF model (300 iterations), and (f) final level set function of the proposed model (200 iterations).

Figure 5

Segmentation results of the proposed and RSF models on four real radiograph images. First row (a-d), the original images with the initial contours. Second row (e-h), results of the RSF model (from left to right, the regularization parameters are v = 0.003 × 255², v = 0.0045 × 255², v = 0.004 × 255², and v = 0.004 × 255², respectively). Third row (i-l), results of the proposed model. Fourth row (m-p), final level set functions of the proposed model for the corresponding image.

Figure 3 shows the segmentation results using both models for a synthetic palm image (108 × 130) by adding Poisson noise. Figure 3a is the original image and Figure 3b shows the initial contour. The results of the RSF model and our model are shown in Figure 3c,d. Clearly, our model achieves better segmentation accuracy for this image with Poisson noise; it separates excellently the two middle fingers that stuck almost together. The final contour obtained by our model reflects accurately the true palm shape. It is shown from Figure 3 that our model can get accurate segmentation results for image with Poisson noise, while the RSF model is sensitive to Poisson noise.

Figure 4 shows the segmentation results for a synthetic image with intensity inhomogeneity which contains seven distinct gray levels and with added Poisson noise. Figure 4a is the original image and Figure 4b shows the initial contour. The results of RSF model and our model are shown in Figure 4c,d. Although the RSF model can extract part of the objects accurately, it fails to segment the white disc and is sensitive to noise (see Figure 4c). In contrast, the proposed model produces satisfactory segmentation result for all discs (see Figure 4d). The final level set functions corresponding to both models are shown in Figure 4e,f. From Figure 4f, we can see that the level set function of our model is almost close to a two-valued function (the object is 1 and the background is 0). Experiments in Figure 4 show that the proposed model can achieve better segmentation results for the images with intensity inhomogeneity and Poisson noise compared to the RSF model.

In Figure 5, we give four examples on real radiograph images. Four test images, which are shown in row 1, are a foot image (177 × 116), a cervical vertebra image (123 × 117), a hip joint image (124 × 118.), and a pastern joint image (140 × 125), respectively. In our model, we use θ = 0.03 for the third image and θ = 0.2 for the fourth image. The foot image in Figure 5a is corrupted by Poisson noise and has severe intensity inhomogeneity. The cervical vertebra image (Figure 5b) has complex object shapes and the object is also corrupted by Poisson noise. The last two images are inhomogeneous and the boundary is very weak. Figure 5a,b,c,d shows the original images along with initial contours. It can be seen from Figure 5e,f,g,h that the RSF model cannot segment correctly these images. Figure 5i,j,k,l shows the corresponding segmentation results of our model; as can be seen, our model accurately detects the objects in these images. These four examples illustrate the abilities of the proposed model to deal with intensity inhomogeneity, weak boundaries, complex object shapes, and immunity to Poisson noise.

Fourth, we elaborate the robustness of the proposed model to contour initialization. In Figure 6, the initial and final contours are green lines and red contours, respectively. Figure 6a,b,c,d shows the segmentation results of our model for the synthetic image (Poisson noise-free) with four different initial contour locations (squares with the same size but different positions). From Figure 6a,b,c,d, we observe that our model successfully extracts all objects of the complicated image for each of the four initial contours. Furthermore, with the same initialization shown in Figure 6a,b,c,d, we show the segmentation results for the synthetic image added Poisson noise (see Figure 6e,f,g,h). It can be seen from Figure 6e,f,g,h that our model segments correct the synthetic image with Poisson noise for these initial contours.

Figure 6

Segmentation results of the proposed model for a synthetic image and a real image. The initial and final contours are displayed by the green lines and red contours, respectively. The first and third rows (a-d, i-l), final results of the proposed model for the original images (Poisson noise-free). The second and fourth rows (e-h, m-p), final results of the proposed model for the images added Poisson noise.

Figure 6i,j,k,l displays the segmentation results of our model for the T-shaped image with different initial contour sizes. The initial contours are chosen as circles at the center of image but with different radius. In Figure 6m,n,o,p, we also show the segmentation results of the T-shaped image with added Poisson noise for the same initialization shown in Figure 6i,j,k,l. The segmentation results in the second and third rows of Figure 6 show that our model can obtain accurate segmentation for all of the four initial contour sizes (images without or with Poisson noise). Experiments in Figure 6 show that our model really allows for more flexible initialization even if the images were added Poisson noise.

5 Conclusion

Inspired from the RSF model [5] and the variational Poisson denoising model [17], we propose a new variational model to segment images with intensity inhomogeneity and Poisson noise. We first propose an energy functional based on the data-fidelity measure deduced from Poisson distribution. We then reformulate the energy functional as globally convex formulation to guarantee the global minima (not the local minima), which makes our model less sensitive to initialization. The dual method is employed to minimize the convex energy functional with an extra regularization term (weighted TV norm). The proposed model is tested on many synthetic and real images; the results demonstrate that it can efficiently cope with intensity inhomogeneity and Poisson noise.