Open Access

Active contours with neighborhood-extending and noise-smoothing gradient vector flow external force

EURASIP Journal on Image and Video Processing20122012:9

DOI: 10.1186/1687-5281-2012-9

Received: 15 January 2012

Accepted: 10 May 2012

Published: 10 May 2012

Abstract

We propose a novel external force for active contours, which we call neighborhood-extending and noise-smoothing gradient vector flow (NNGVF). The proposed NNGVF snake expresses the gradient vector flow (GVF) as a convolution with a neighborhood-extending Laplacian operator augmented by a noise-smoothing mask. We find that the NNGVF snake provides better segmentation than the GVF snake in terms of noise resistance, weak edge preservation, and an enlarged capture range. The NNGVF snake accomplishes this with a reduced computational cost while maintaining other desirable properties of the GVF snake, such as initialization insensitivity and good convergences at concavities. We demonstrate the advantages of NNGVF on synthetic and real images.

Keywords

image segmentation active contour gradient vector flow Laplacian operator neighborhood-extending and noise-smoothing gradient vector flow

1. Introduction

D uring the last two decades, variational and PDE-based methods for image segmentation and analysis have become standard tools [1]. Active contours or snakes which have deeply influenced variational approaches to image segmentation since their introduction [2] are curves that can conform to object boundaries or other image features under the influence of internal and external forces [2]. Generally, active contours can be categorized as parametric snakes [2] or as geometric snakes [35] according to their representation. Parametric snakes require an explicit representation while geometric snakes are defined implicitly. Here, we show how to construct an effective external force for parametric active contour models that can also be integrated into geometric active contours using a level set formulation [4].

Since the external force defines the evolution of an active contour, many external force models have been proposed [514]. Among these, the gradient vector flow (GVF) [10] has been most successful, as it provides a large capture range and the ability to capture concavities by diffusing the gradient vectors of an edge map generated from the image. Although the GVF model has proved effective and has widely been used in image segmentation, it has some disadvantages, such as a high computational cost, substantial noise sensitivity, and an inability to capture and preserve weak edges. Various improved models based on the GVF have been developed. For example, generalized gradient vector flow [11], harmonic gradient vector flow [12], motion gradient vector flow [13] and generalized dynamic directional gradient vector flow [14], but none of these are able to resolve all of the problems mentioned above.

We propose a novel external force for active models, called neighborhood-extending and noise-smoothing gradient vector flow (NNGVF), which incorporates a neighborhood-extended Laplacian operator mask and modifies the mask by adding a noise-smoothing mask. The proposed NNGVF snake outperforms the GVF snake in terms of computation, capture range, noise resistance, and weak edge preserving ability, while maintaining other desirable properties of the GVF snake such as initialization insensitivity and good convergence at concavities.

2. Background

2.1. Snakes: active contours

A snake is an elastic curve c(s) = [x(s),y(s)], s [0, 1] which deforms to minimize the energy functional [2]:
E snake = 1 2 α c s 2 + β c ss 2 + E ext c ( s ) d s ,
(1)
where cs(s) and css(s) are the first and second derivative of c(s) with respect to s weighed by positive α and β, respectively. A typical external force for a gray-scale image I is Eext = -|G σ *I|, where G σ is the Gaussian kernel with standard deviation σ and where * denotes convolution. Using standard variational methods, the Euler equation to minimize Esnake is expressed
α c ss ( s ) - β c ssss ( s ) - E ext = 0 .
(2)
This can be considered as a force balance equation:
F int + F ext = 0 ,
(3)

where Fint = αcss(s)-βcssss(s) and Fext = -Eext. The internal force Fint forces the snake contour to be smooth while the external force Fext attracts the snake to the desired image features.

2.2. GVF: gradient vector flow external force

Typical shortcomings of the external force using the gradient vector of the edge map of the image include a limited capture range and convergence to concavities [10]. In order to solve these problems, Xu and Prince [10] proposed the GVF model. The GVF is a vector field v(x, y) = (u(x, y), v(x, y)) obtained by minimizing the following energy functional [15]:
E GVF = μ ( u x 2 + u y 2 + v x 2 + v y 2 ) + f 2 v - f 2 d x d y ,
(4)
where f is the edge map of an image, and variable μ is a regularization parameter. The functions u(x, y) and v(x, y) are at least C2 when EGVF is minimized. The Euler equation to minimize EGVF is:
u t = μ 2 u - | f | 2 u - f = 0 ,
(5a)
v t = μ 2 v - | f | 2 v - f = 0 ,
(5b)

where 2 is the Laplacian operator.

3. NNGVF snakes

3.1. Extended neighborhood

A simple 4-neighborhood approximation to the Laplacian operator is:
2 f = 2 f x 2 + 2 f y 2 = f ( i , j + 1 ) - 2 f ( i , j ) + f ( i , j - 1 ) + f ( i + 1 , j ) - 2 f ( i , j ) + f ( i - 1 , j ) = f ( i , j + 1 ) + f ( i , j - 1 ) + f ( i + 1 , j ) + f ( i - 1 , j ) - 4 f ( i , j )
(6)
which can be achieved by convolution of the image with the mask:
G 4 = 0 1 0 1 - 4 1 0 1 0 .
(7)
There are a number of alternative Laplacian templates that may be used to extend the spatial extent of the Laplacian operator to a larger neighborhood, such as 8-neighborhood, 16-neighborhood, and 24-neighborhood approximations. By calculating the Laplacian operator over a larger neighborhood, more image information will be used. The 24-neighborhood mask that approximates the Laplacian operator is
G 2 4 = 1 1 1 1 1 1 1 1 1 1 1 1 - 24 1 1 1 1 1 1 1 1 1 1 1 1 .
(8)

3.2. Decomposition of the Laplacian operator

The particular Laplacian operator [16] can be expressed as the difference between an all-pass (AP) filter and a "round mean" (RM) filter, whose masks are shown in (10). The Laplacian operator can thus be defined as proportional to
G 24 = ( A P 24 - R M 24 ) ,
(9)
A P 2 4 = 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 , R M 2 4 = 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 0 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 1 / 24 .
(10)

In this model, the AP filter is the 2D linear identity (do-nothing) filter, while the RM filter is a 2D low-pass filter. The difference yields high-frequency components over a large area. Since the purpose of the AP filter is to estimate the image at the center pixel, but is highly sensitive to noise [17], it is advisable to replace the AP filter with a better designed filter that can augment edge-preservation and noise robustness.

3.3. The proposed NNGVF external force

Motivated by Wang [17], we utilize the 5 × 5 noise smoothing (NS) filter shown in (12) instead of the AP filter, yielding considerably enhanced NS. We dub this model the NNGVF model. The NNGVF model is defined as the equilibrium solution of the following vector partial differential equation:
v t = μ ( v * N S 2 4 - v * R M 2 4 ) - f 2 ( v - f ) ,
(11)
where μ is a regularization parameter and
N S 2 4 = 0 0 1 / 12 0 0 0 0 1 / 12 0 0 1 / 12 1 / 12 1 / 3 1 / 12 1 / 12 0 0 1 / 12 0 0 0 0 1 / 12 0 0 .
(12)

In (11), NS24 and RM24 are 5 × 5 masks, which make use of larger areas of image information. Since the convolution is used in (11), the computational cost of NNGVF is greatly reduced relative to GVF.

4. Experimental results

Next, we demonstrate some desirable properties of the NNGVF snake and compare the performances of the NNGVF and GVF snakes. Since NNGVF is an improvement over GVF, we focus primarily on some common concerns encountered in snake-based image segmentation, which include (1) capture range enlargement and U-shape convergence, (2) weak edge preservation, (3) noise robustness, and (4) real images. The parameters for all snakes in our experiments are α = 0.1, β = 0 and time step τ = 1. The weight μ for the GVF and NNGVF snakes is set to 0.15 in all experiments unless otherwise stated.

4.1. Capture range enlargement and U-shape convergence

One of the advantages of the GVF snake is that its capture range is large. However, by expanding the calculations of the Laplacian operator to a larger neighborhood, more image information is used, so the NNGVF snake capture a larger range. Since the NNGVF snake computes the Laplacian operator using only a simple convolution, the computation time for the NNGVF is less than for GVF. As examples, we used the U-shape and room images, which have also been employed in [10, 11] to verify some general properties of NNGVF snake. Figure 1 shows that the NNGVF snake also can locate objects correctly from a distant initialization, can adhere to boundaries with gaps, and converges to concavity. For the room image, the execution time of NNGVF is 0.14 s while that of GVF is 0.42 s. For the U-shape image, the execution time of NNGVF is 0.17 s while that of GVF is 0.54 s. We can see that compared to the GVF snake, the NNGVF snake can capture a larger range with less computation time.
Figure 1

Results of snakes: (a) the GVF snake at iterations 60, (b) NNGVF snake at iterations 40, (c) GVF snake at iterations 90, and (d) NNGVF snake at iterations 50. In each panel, the left is the evolution of snake and the right is the force field.

4.2. Weak edge preserving

As pointed out in [18], imposing the Laplacian operator as a smoothing constraint fields has strong isotropic smoothing and poor edge preservation. Because of this, the GVF does not effectively preserve weak edges. Figure 2 shows the performance of the GVF and NNGVF snakes on a synthetic image, where there is a gap neighbored by a line segment. In Figure 2a, the GVF snake leaks and converges incorrectly to the line segment; while in Figure 2b, the NNGVF snake succeeds in preserving the weak edge.
Figure 2

Convergences of snakes: (a) the GVF snake, (b) NNGVF snake. In each panel, the left is the evolution of snake and right is the force field.

4.3. Noise robustness

Since the Laplacian operator used in GVF is sensitive to noise, the NNGVF also provides an advantage. The U-shape image used in [10] is employed here to generate noisy images. The first noisy image in Figure 3a is created by adding "salt & pepper" noise of intensity 0.06; the second one in Figure 3a by "speckle" noise with mean 0 and variance 0.08; the third one in Figure 3a by adding "Gaussian" noise of mean 0 and variance 0.01 and the fourth one in Figure 3a is by adding "Rayleigh" noise of mean 0 and variance 0.02. Figure 3 shows the results. It can be seen that the GVF snake was noticeably attracted to noise, but the proposed NNGVF snake moved into the concavity successfully.
Figure 3

Results of snakes: (a) the noisy images with top to bottom salt and pepper noise, speckle noise, Gaussian noise and Rayleigh noise, respectively; (b) convergences of GVF snakes, (c) convergences of NNGVF snakes.

4.4. Real images

Figure 4 shows segmentation results of the GVF and NNGVF snakes on two real medical images: one cardiac CT image (the first row) and one lung CT image (the second row). In these two examples, the algorithm must cope with noise and weak edges. For the cardiac image, we aim to extract the endocardium of the left ventricle, which is a weak edge. One can see that the GVF snake cannot convergence onto the desirable region on the image, whereas the NNGVF snake performs well on the weak edges and supplies good noise resistance. It can be seen that the GVF force field leaks at weak edges and gets trapped by the noise, while the NNGVF force field converges to the endocardium correctly. For the lung image, we aim to extract the parenchyma in the left and the cancer in the right part. The GVF snake again leaks at weak edges and gets trapped by the noise, whereas the NNGVF snake performs well on weak edges and supplies good noise resistance. Furthermore, in Figure 4b, d, the execution times of NNGVF are 1.26 and 1.15 s, respectively; while in Figure 4a, c, the execution times of GVF are 3.92 and 3.78 s, respectively.
Figure 4

Convergences of snakes: (a) the GVF snake at iterations 250, (b) NNGVF snake at iterations 120, (c) GVF snake at iterations 200, and (d) the NNGVF snake at iterations 100. In each panel, the left is the evolution of snake and the right is the force field.

5. Conclusion

We proposed a novel external force called NNGVF for active contours. The NNGVF snake deploys the GVF as a convolution operation using a neighborhood-extending Laplacian mask, modifying the mask to improve noise-smoothing, yields a good performance in terms of capture range, weak edge preservation, and noise robustness while maintaining the other desirable properties of GVF, such as initialization insensitivity and good convergence at concavities. The experimental results showed that the NNGVF snake outperforms the GVF snake in terms of computation as well.

Declarations

Acknowledgements

This work is supported by the NSFC (60805004).

Authors’ Affiliations

(1)
Beijing Laboratory of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology
(2)
Laboratory of Image and Video Engineering, Department of Electrical and Computer Engineering, The University of Texas

References

  1. Aubert G, Kornprobst P: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. 2nd edition. Springer, New York; 2006.Google Scholar
  2. Kass M, Witkin A, Terzopoulos D: Snakes: active contour models. Int J Comput Vis 1988, 1(4):321-331.View ArticleGoogle Scholar
  3. Caselles V, Catte F, Coll T, DIBOS F: A geometric model for active contours in image processing. Numerische Mathematik 1993, 66(1):1-31.MATHMathSciNetView ArticleGoogle Scholar
  4. Paragios N, Mellina-Gottardo O, Ramesh V: Gradient vector flow fast geometric active contours. IEEE Trans Pattern Anal Mach Intell 2004, 26(3):402-407.View ArticleGoogle Scholar
  5. Lankton S, Tannenbaum A: localizing region based active contours. IEEE Trans Image Process 2008, 17(11):2029-2039.MathSciNetView ArticleGoogle Scholar
  6. Cohen LD: On active contour models and balloons. CVGIP: Image Understand 1991, 53(2):211-218.MATHView ArticleGoogle Scholar
  7. Cohen LD, Cohen I: Finite-element methods for active contour models and balloons for 2-D and 3-D images. IEEE Trans Pattern Anal Mach Intel 1993, 15(11):1131-1147.View ArticleGoogle Scholar
  8. Leroy B, Herlin I, Cohen LD: Multi-resolution algorithms for active contour models. 12th International Conference on Analysis and Optimization of Systems 1996, 58-65.Google Scholar
  9. Muralidhar GS, Bovik AC, Giese JD, Sampat MP, Whitman GJ, Haygood TM, Stephens TW, Markey MK: Snakules: an evidence-based active contour algorithm for the annotation of spicules on mammography. IEEE Trans Med Imag 2010, 29(10):1768-1780.View ArticleGoogle Scholar
  10. Xu C, Prince J: Snakes, shapes and gradient vector flow. IEEE Trans Image Process 1998, 7(3):359-369.MATHMathSciNetView ArticleGoogle Scholar
  11. Xu C, Prince J: Generalized gradient vector flow external forces for active contours. Signal Process 71: 131-139.
  12. Wang Y, Jia Y, Liu L: Harmonic gradient vector flow external force for snake model. Electron Lett 2008, 44(2):105-107.MathSciNetView ArticleGoogle Scholar
  13. Ray N, Acton ST: Motion gradient vector flow: an external force for tracking rolling leukocytes with shape and size constrained active contours. IEEE Trans Med Imag 2004, 23(12):1466-1478.View ArticleGoogle Scholar
  14. Liu L, Wu Y, Wang Y: A novel method for segmentation of the cardiac MR images using generalized DDGVF snake models with shape priors. Inf Technol J 2009, 8(4):486-494.View ArticleGoogle Scholar
  15. Xu C, Prince JL: Gradient vector flow: a new external force for snakes. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1997, 66-71.Google Scholar
  16. Jain R, Kasturi R, Schuck BG: Machine Vision. McGraw Hill; 1995.Google Scholar
  17. Wang X: Laplacian operator-based edge detectors. IEEE Trans Pattern Anal Mach Intell 2007, 29(5):886-890.View ArticleGoogle Scholar
  18. Ning J, Wu C, Liu S, Yang S: NGVF: an improved external force field for active contour model. Pattern Recognit Lett 2007, 28(1):58-63.View ArticleGoogle Scholar

Copyright

© Liu and Bovik; licensee Springer. 2012

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.