Image segmentation algorithm by piecewise smooth approximation
© Wang and He; licensee Springer. 2012
Received: 24 October 2011
Accepted: 3 September 2012
Published: 24 September 2012
We propose a novel image segmentation algorithm using piecewise smooth (PS) approximation to image. The proposed algorithm is inspired by four well-known active contour models, i.e., Chan and Vese’ piecewise constant (PC)/smooth models, the region-scalable fitting model, and the local image fitting model. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. In this article, pursuing the same idea we introduce different energy functional and PS function to search for the optimal PS approximation of the original image. The initial function with our model can be chosen as a constant function, which implies that the proposed algorithm is robust to initialization or even free of manual initialization. Experiments show that the proposed algorithm is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared ship images with low contrast and complex background.
Image segmentation is a popular problem in image processing and computer vision, which has been studied extensively in past decades. The segmentation goal is to separate image domain into a collection of distinct regions, upon which other high-level tasks such as objects recognition and tracking can be further performed. Due to the presence of noise, complex background, low intensity contrast with weak edges, and intensity inhomogeneity , image segmentation is still a difficult problem in practical applications, especially for traditional segmentation methods [2–8]. Traditional methods, like Canny edge detection , are simple and fast, but they always need further edge linking operation to produce continuous object boundaries . To address these issues, more recent methods including implicit active contours [9–17] have been developed for image segmentation.
Implicit active contours are active contour models implemented via the level set method . One of the remarkable advantages of active contour models is that it can provide smooth and closed contours, which is generally impossible in traditional segmentation methods. Existing implicit models can roughly be categorized into two classes: edge-based models [10, 11, 13] and region-based models [12, 14–16]. Edge-based models use local edge information (image gradient) to perform contour extraction, which are usually to noise and weak edges. Region-based models utilize the global and/or local image statistics inside and outside the active contour (evolving curve) to find a partition of image domain. They generally have better performance in the presence of weak or discontinuous boundaries and less sensitive to initialization.
A major category of the region-based level set methods is proposed to minimize the well-known Mumford and Shah (MS) functional . Due to the difficulty of directly minimizing the MS functional, different approximation methods have been proposed to allow more efficient energy minimization. For example, the piecewise constant (PC) models [12, 20] approximate image domain by a set of homogenous regions, but which is not true for images with intensity inhomogeneity. In [21, 22], more advanced piecewise smooth (PS) models have been proposed to improve the PC model performance in terms of intensity inhomogeneity. However, due to rather complicated algorithms, these models are usually computationally expensive. As indicated in , the models using global image statistics usually have difficulty to segment many real-world images with intensity inhomogeneity.
To address this issue, recently localized region-based models [14, 16, 23–25] have been proposed. For example, Li et al.  proposed a region-scalable fitting (RSF) model (originally termed as local binary fitting (LBF) model ), which draws upon spatially varying local region information and thus is able to deal with intensity inhomogeneity. The RSF model has better performance than PC and PS models in segmentation of accuracy and computational efficiency. However, since there are totally four convolutions to be computed at each iteration in the implementation, the RSF model is still computationally expensive. For more efficient segmentation, Zhang et al.  proposed a novel active contour model driven by local image fitting (LIF) energy. The LIF energy is constructed by the local image information, which can be viewed as a constraint of the differences between the fitting image and the original image. The complexity analysis and experimental results show that this model is more efficient than the LBF model, while yielding similar results.
These localized methods [14, 16, 25, 26] are, however, to some extent sensitive to contour initialization. Since the segmentation results typically depend on the selection of initial contours, these methods need user intervention to define the initial contours professionally. This means that they may be fraught with the problems of how and where to define the initial contours. The multiple-seed initialization used by Vese and Chan  also cannot always produce better results than the single-seed initialization . Therefore, it is still a great challenge to find an efficient way to tackle the initialization problem.
In this article, inspired by the above-mentioned four models [Chan–Vese (CV), PS, RSF, and LIF models], we propose a novel image segmentation algorithm without initial contours using PS approximation. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. Pursuing the same idea, in this study, we introduce different energy functional and PS functions to search for the optimal PS approximation of the original image. In the proposed algorithm, the initial function can be chosen as a constant function, which implies that the proposed algorithm allows for robustness to initialization or even free of manual initialization. Experiments show that the proposed algorithm works well for images with intensity inhomogeneity. Moreover, it is very appropriate for infrared images with low contrast and complex background.
The remainder of this article is organized as follows. In “Related studies” section, we briefly review several classical region-based models and indicate their limitations. The proposed algorithm is introduced in “The LIF model results in a local fitted image” section. The proposed model presents experimental results using a set of synthetic and real images. This article is summarized in “Experimental results” section.
Chan–Vese (CV) PC models
Chan and Vese  restricted the MS minimal partition problem  to PC functions and proposed a technique that implements efficiently the PC MS model via level set methods . In level set methods, a contour C is represented implicitly by the zero-level set of a Lipschitz function , which is called a level set function. In what follows, we let the level set function take positive and negative values inside and outside the contour C, respectively.
The CV model is implemented by an alternative procedure: for each iteration and the corresponding level set function , we first compute the optimal constants and , then obtain by minimizing with respect to . This process is repeated until the zero-level set of is exactly on the object boundary.
where c1 and c2 are the optimal constants c1 and c2 that minimize the global fitting energy (1) are the averages of the image intensities in the region and , respectively. Such optimal constants can be far away from the original image data, if the intensities outside or inside the contour are not homogeneous. As a result, the CV model generally fails to segment images with intensity inhomogeneity.
The PS model is also implemented by an alternative procedure: for each iteration and the corresponding level set function , we first obtain and by solving Equations (5) and (6), then obtain by minimizing the functional with respect to . Repeat the process until the zero-level set of is exactly on the object boundary.
where u+ and u- are obtained by solving Equations (5) and (6). In the PS model, two coupled equations must be solved to obtain u+ and u- before each iteration, and the computational cost is very expensive. Moreover, in the implementation of PS model, u+ and u- must be extended to the whole image domain, which is difficult to implement and also increases the computational cost. In summary, the high complexity limits the application of PS model in practice.
The RSF model is implemented via an alternative procedure: for each iteration and the corresponding level set function , we first compute the fitting values and , then obtain by minimizing with respect to . This process is repeated until the zero-level set of is exactly on the object boundary.
However, the smooth functions f1 and f2 are computed directly from (9) but no longer obtained by solving equations.
The RSF model improves the global PC and PS models on intensity inhomogeneity and is more computationally efficient than the PS models. Nevertheless, four convolutions to be computed at each iteration still demand a high computational cost. Besides, the RSF model is to some extent sensitive to contour initialization (initial locations, sizes and shapes); it obtained different segmentation with different initial contours, even with the same parameter settings.
Here, W k (x) is a rectangular window, which is generally chosen as a truncated Gaussian window K σ (x) with standard deviation σ and of size 4 k × 4 k (k is the greatest integer smaller than σ).
Similar as the above three models, the LIF is also implemented by an alternative procedure: for each iteration and the corresponding level set function , we first compute the fitting values and , then obtain by minimizing with respect to . This process is repeated until the zero-level set of is exactly on the object boundary.
The LIF model results in a local fitted image
It is also a PS approximation of the original image but with different fitting values m1 and m2.
By using local region information, the LIF model is able to provide desirable segmentation results in the presence of intensity inhomogeneity. Although it is more efficient than the RSF model, it is still sensitive to contour initialization, like most existing localized active contours [14, 16, 25, 26]. In practice, the LIF model initial contour has to carefully be selected for a satisfactory segmentation.
This can be seen from a similar experiment as the RSF model in Figures 1 and 2. Figure 1(k)-(o) shows the segmentation results of the LIF model for the vessel image with five distinct initial locations, which reveal that the LIF model extracts accurately the object only for the second and fourth initial contours (Figure 1 (l, n)). In Figure 2(k)-(o), we demonstrate the segmentation results of the LIF model for the vessel image with five initial contours of different sizes and shapes. From the results, we observe that the LIF model fails to segment the vessel image for the third and fifth initial contours (Figure 2 (m, o)), although it better captures the object for the other three initial contours (Figure 2 (k, l, n)).
The proposed model
The above-mentioned four models share the similar algorithm structure to find a PS (constant) approximation of the original image. The main difference is how to define both the energy functional to be minimized and the PS function approximating the original image. In this section, pursuing the same idea we introduce different energy functional and PS function to search for the optimal PS approximation of the original image.
with a scale parameter σ > 0. The functions and are really the weighted averages of the image intensities in the regions and respectively, with as the weight assigned to the intensity I(y). The smoothness of and can be confirmed by the Gaussian convolutions in (14).
The energy functional in (19) is in fact a square error of the approximation of the image I (x) by the PS function ; therefore, we call the piecewise smooth fitting (PSF) energy.
The regularizing term intrinsically maintains the regularity of the level set function without the need for extra re-initialization procedures.
where v, μ > 0 are constants to balance the terms.
Initialize the level set function, and set n = 0.
Compute the smooth functions and.
Obtain by minimizing the energy functional.
If the zero-level set of is exactly on the object boundary, then stop; otherwise, let n = n + 1 and , then return to Step 2.
The CV PC model can be considered as an extreme case of the proposed model for. This can be seen from the fact that .
where ρ is a constant. Such constant initialization completely eliminates the need of initial contours. We do not need to consider the problems such as the determination of where and how to define the initial contours.
where is the approximation of the right-hand side of Equation (24) by the above spatial difference scheme.
We next present the experiments on both synthetic and real images with intensity inhomogeneity, noise and complex background. The function is simply initialized to a constant function: . Unless otherwise specified, we use the following default setting of the parameters which are determined by experiments: , , , , space step (implied pixel spacing) and time step . All experiments were run under Matlab R2007a on a PC with Dual 2.7 GHz processor.
Figure 4 shows the segmentation results of four methods for four real images: a leaf image (122×120) with texture background, a chip image (182 × 180), a breast cyst image (91 × 92), and a CT medical image (117 × 123). Because of the influence of the texture, material properties, breast tissue, and the technical limitations, the backgrounds in these images are non-uniform, noise or even complex. Figure 4(e)-(h) give the contours detected by Canny algorithm; we see that some contours detected by this algorithm are not continuous (Figure 4(e)). From the results of the CV model shown in Figure 4(i)-(l), we observe that, the CV model works well for images with less non-uniform (Figure 4(i) and (j)), however, if the level of non-uniform is higher, it fails to obtain exact segmentation results and identifies incorrectly some part of the background/foreground as the background/foreground (Figure 4(k) and (l)). Figure 4(m)-(p) shows the results of the RSF model; with fine-tuning parameters (given in the Figure Caption), the RSF model obtain satisfactory segmentation results expect for a little deficit in the lower left corner of the object in Figure 4(p). The results of our model and the corresponding 3D plots of final level set functions are shown in the last two rows of Figure 4. As can be seen from Figure 4(q)-(t), our model has effectively extracted object boundaries from the non-uniform and noisy background. For these four images, we chose large values for v to enhance the model’s robustness to noise: σ = 5 for Figure 4(a), for Figure 4(b), for Figure 4(c) and for Figure 4(d).
Figure 5 show experiments on multiple objects and inside holes segmentation. Figure 5(a)-(d) is: a potato image (157 × 157) with inhomogeneous intensities, a cell image (195 × 183) with one cell crossing the image boundary, a wrench image (200 × 200) with interior holes in each object, and a synthetic image (118 × 134) with seven different intensity values, respectively. Figure 5(e)-(h) give the contours detected by Canny algorithm; we see that it extracts the satisfactory object contours for images with obvious intensity gradient (Figure 5(f)-(h)), but the contour detected for another image is disturbed by non-uniform intensity (Figure 5(e)). From the results of the CV model shown in Figure 5(i)-(l), we observe that, the CV model can deal with homogeneous images with multiple objects or inside holes (Figure 5(f) and (g)), but if the intensity in such images are inhomogeneous, it fails to obtain the whole boundaries for all of the objects (Figure 5(i) and (l)). Figure 5(m)-(p) shows the best segmentation results of the RSF model we obtained by tuning parameters; it obtain the exact result only for the last image (Figure 5(p)). The results of our model and the corresponding 3D plots of final level set functions are shown in the last two rows of Figure 5. As can be seen from Figure 5(q)-(t), our method detects all the objects, even though the object intensity is inhomogeneous and some edges are blurry.
The third example presented in Figure 6 demonstrates that our model is especially suitable for segmentation of infrared ship images with low contrast, blurry edges and complex (noisy) background. Because of the limitation of thermal imaging and the actual water-mountain-sky conflicts, infrared images always suffers from low contrast and complex background; thus, it is not a trivial task for most existing models to extract the objects (ships) in such images. As can be seen from Figure 6(e)-(h), Canny algorithm has difficulty dealing with such infrared ship images, due to the presence of noise, low contrast, complex background and so on. To suppress noise effectively, a larger σ or a higher threshold T is needed, but a larger σ can result in edge shifting or inaccurate edge location (Figure 6(s)) and a higher T always causes edge leaking (Figure 6(r)). From the results of the CV model shown in Figure 6(i)-(l), we observe that, the CV model obtain acceptable results in Figure 6(j) and (k), but due to the intensity inhomogeneity and noise, it can’t obtain the exact object boundaries in Figure 6(i) and (l). Figure 6(m)-(p) shows the segmentation results of the RSF model; it obtain the accurate results only for the first and the last image (Figure 6(m) and (p)). In contrast, our model, starting with a nonzero constant function, has successfully extracted the ships from the complex backgrounds, as shown in Figure 6(q)-(t). For noisy images, we increase the value of v to improve the model’s performance: for Figure 6(c) and for Figure 6(d).
In this article, we propose a novel image segmentation algorithm by PS approximation. The proposed model, which pursues the basic ideas behind four well-known region-based models, introduces different energy functional and PS functions to find the optimal PS approximation of the original image. The initial function can be chosen as a constant function, which completely eliminates the need of initial contours. The proposed model is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared images with low contrast and complex background.
This work is supported by the National Natural Science Foundation of China under Grant No. 61202349 and No. 11201506.
- He L, Peng Z, Everding B, Wang X, Han CY, Weiss KL, Wee WG: A comparative study of deformable contour methods on medical image segmentation. Image Vision Comput. 2008, 26(2):141-163. 10.1016/j.imavis.2007.07.010View ArticleGoogle Scholar
- Papari G, Petkov N: Edge and line oriented contour detection: state of the art. Image Vision Comput. 2011, 29: 79-103. 10.1016/j.imavis.2010.08.009View ArticleGoogle Scholar
- Pal NR, Pal SK: A review on image segmentation techniques. Pattern Recogn. 1993, 26(9):1277-1294. 10.1016/0031-3203(93)90135-JView ArticleGoogle Scholar
- Wang S, Siskind JM: Image segmentation with ratio cut. IEEE Trans. Pattern Anal. Mach. Intell. 2003, 25(6):675-690. 10.1109/TPAMI.2003.1201819View ArticleGoogle Scholar
- Nath SK, Palaniappan K: Fast graph partitioning active contours for image segmentation using histograms. EURASIP J. Image Vide. 2009, 2009: 9.Google Scholar
- Chabrier S, Rosenberger C, Emile B, Laurent H: Optimization-based image segmentation by genetic algorithms. EURASIP J. Image Vide. 2008, 2008: 1-10.Google Scholar
- Canny J: A computational approach to edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence 1986, 8: 679-714.View ArticleGoogle Scholar
- Kimmel R, Bruckstein AM: On regularized Laplacian zero crossings and other optimal edge integrators. Int. J. Comput. Vis. 2003, 53(3):225-243. 10.1023/A:1023030907417View ArticleGoogle Scholar
- He L, Zheng S, Wang L: Integrating local distribution information with level set for boundary extraction. J. Vis. Commun. Image R. 2010, 21: 343-354. 10.1016/j.jvcir.2010.02.009View ArticleGoogle Scholar
- Caselles V, Catte F, Coll T, Dibos F: A geometric model for active contours in image processing. Numer. Math. 1993, 66: 1-31. 10.1007/BF01385685MATHMathSciNetView ArticleGoogle Scholar
- Caselles V, Kimmel R, Sapiro G: Geodesic active contours. Int. J. Comput. Vis. 1997, 22(1):61-79. 10.1023/A:1007979827043MATHView ArticleGoogle Scholar
- Chan TF, Vese LA: Active contours without edges. IEEE Trans. Image Process. 2001, 10(2):266-277. 10.1109/83.902291MATHView ArticleGoogle Scholar
- Li C, Xu C, Gui C, Fox MD: Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR). 1st edition. San Diego, CA, USA; 2005:430-436.Google Scholar
- Li C, Kao C, Gore JC, Ding Z: Minimization of region-scalable fitting energy for image segmentation. IEEE Trans. Image Process. 2008, 17(10):1940-1949.MathSciNetView ArticleGoogle Scholar
- Wang L, Li C, Sun Q, Xia D, Kao C: Active contours driven by local and global intensity fitting energy with application to brain MR image segmentation. Comput. Med. Imaging Graph. 2009, 33(7):520-531. 10.1016/j.compmedimag.2009.04.010View ArticleGoogle Scholar
- Zhang K, Song H, Zhang L: Active contours driven by local image fitting energy. Pattern Recognit. 2010, 43: 1199-1206. 10.1016/j.patcog.2009.10.010MATHView ArticleGoogle Scholar
- Wang Y, He C: Adaptive level set evolution starting with a constant function. Appl. Math. Model. 2012, 36: 3217-3228. 10.1016/j.apm.2011.10.023MATHMathSciNetView ArticleGoogle Scholar
- Osher S, Sethian JA: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 1988, 79: 12-49. 10.1016/0021-9991(88)90002-2MATHMathSciNetView ArticleGoogle Scholar
- Mumford D, Shah J: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 1989, 42(5):577-685. 10.1002/cpa.3160420503MATHMathSciNetView ArticleGoogle Scholar
- Wang X, He L, Wee WG: Deformable contour method: a constrained optimization approach. Int. J. Comput. Vis. 2004, 59(1):87-108.View ArticleGoogle Scholar
- Vese LA, Chan TF: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 2002, 50(3):271-293. 10.1023/A:1020874308076MATHView ArticleGoogle Scholar
- Tsai A, Yezzi A, Willsky AS: Curve evolution implementation of the Mumford–Shah functional for image segmentation, denoising, interpolation, and magnification. IEEE Trans. Image Process. 2001, 10(8):1169-1186. 10.1109/83.935033MATHView ArticleGoogle Scholar
- Lankton S, Tannenbaum A: Localizing region-based active contours. IEEE Trans. Image Process. 2008, 17(11):2029-2039.MathSciNetView ArticleGoogle Scholar
- Brox T, Cremers D: On the statistical interpretation of the piecewise smooth Mumford–Shah functional. In Proceedings of the Scale Space Methods and Variational Methods (SSVM ’07), vol. 4485 of Lecture Notes in Computer Science. Ischia, Italy; 2007:203-213.Google Scholar
- Piovano J, Rousson M, Papadopoulo T: Efficient segmentation of piecewise smooth images. In Proceedings of the Scale Space Methods and Variational Methods (SSVM ’07). Lecture Notes in Computer Science, Ischia, Italy; 2007:709-720.View ArticleGoogle Scholar
- Li C, Kao C, Gore J, Ding Z: Implicit active contours driven by local binary fitting energy. In Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR). Washington, USA; 2007:1-7.Google Scholar
- Gao S, Bui TD: Image segmentation and selective smoothing by using Mumford–Shah model. IEEE Trans. Image Process. 2005, 14(10):1537-1549.View ArticleGoogle Scholar
- Li C, Xu C, Gui C, Fox MD: Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 2010, 19(12):3243-3254.MathSciNetView ArticleGoogle Scholar
- Aubert G, Kornprobst P: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. 2nd edition. Springer, New York; 2002.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.