 Research
 Open Access
 Published:
Research of segmentation method on color image of Lingwu long jujubes based on the maximum entropy
EURASIP Journal on Image and Video Processing volume 2017, Article number: 34 (2017)
Abstract
This paper researches on methods of the color image segmentation method of Lingwu long jujubes based on the maximum entropy to achieve the accuracy of image segmentation and improve accuracy of machine recognition. According to law between the color of Lingwu long jujubes and characteristic of environment, starting from the hue information, this paper is first to explore the difference between the hue of Lingwu long jujubes and the environment which it lives and then use maximum entropy to segment image. It finds optimal threshold by mathematical criterion judging the accuracy of image segmentation. The method of preprocessing of image is mean filter firstly. Then, it extracts hue information of true color image and uses maximum entropy for image segmentation, judging accuracy of image segmentation by segmentation area whether it is in accordance with the 3σ principle. Mathematical morphology is used for smoothing image and eliminating small holes. Finally, segmented image will be obtained through labeling the image by using methods of labeled image and using characteristic parameters for extracting feature. By comparing the segmentation effect with artificial method of the 30 Lingwu long jujubes images, it proves that the color image segmentation method of Lingwu long jujubes based on the maximum entropy has good effect to extract the object region. The accuracy of segmentation rate is up to 89.60%. The time that the algorithm run is 1.3132 s.
Introduction
Lingwu long jujubes are important economical fruits in Ningxia. To improve packing efficiency and reduce labor intensity, there is an increasing demand for automatic picking techniques. The identification of fruits plays a key role in automatic packing. Not only is image segmentation basic of image processing which effects directly the performance of image recognition but also it is a difficulty for digital image processing. Sweden, Canada, and Finland research image segmentation algorithms based on machine vision [1,2,3,4,5,6,7]. No algorithm, however, is aimed at Lingwu long jujubes. Existing approaches on image segmentation and feature extraction for Lingwu long jujubes have many disadvantages. Although image recognition of fruits based on machine vision starts relatively late, it develops rapidly. Professor Jiangming Kan from Beijing forestry university researches computer visionbased method of automatic measurement of trunk and branch diameters of standing trees and threedimensional reconstruction [8, 9]. Doctor Yutan Wang studies on Lingwu long jujubes’ localization and maturity recognition [10]. Algorithm should be effectiveness or high efficiency [11, 12]. This paper attempts to use improved image segmentation algorithm based on maximum entropy method in image recognition of Lingwu long jujubes, which provides more selection and comparison for image segmentation algorithm of Lingwu long jujubes in the process of automatic picking.
According to actual picking environment of Lingwu long jujubes, this paper finds the rules between hue of Lingwu long jujubes and the surrounding environment and extracts hue information of true color image. Then, it puts forward that the way of combining maximum entropy method and mathematical criterion selects adaptively optimal threshold for image segmentation by analyzing hue histogram of a set of images. Finally, aiming at disadvantages of the image after segmentation, mathematical morphology, labeling, and future selection are used for post processing to extract accurate target feature.
Materials and methods
Image acquisition of Lingwu long jujubes
Hardware equipment of image acquisition includes Levono mobile phone and HP PC computer. For the convenience of experiment, this paper sets the revolution of images to 340 * 300. The type of mobile phone is Lenovo P700 and eight million pixels. The parameters of PC are as follows: HP Pavilion g series, Inter(R) Core(TM) i32310M CPU @ 2.10 GHz 2.10 GHz, RAM 2.00, and Windows 7 ultimate ×32. The image is collected under environment of sunny and stored in JPG.
Preprocessing of image
The image will be polluted by various noises in the process of acquisition and transmission, which affects seriously the effect of image segmentation in post processing because noises are amplified probably. This paper uses 3 × 3 mean filtering [13] for image smoothing to eliminate some noises (Fig. 1).
The distribution of Z neighborhood space in the position of (i, j) is following Table 1.
The pixel value in the position of (i, j) is replaced with g(i, j).
As shown in Fig. 2, two images containing noises are filtered by 3 × 3 mean filtering.
The method, processing channel R, G and B by the same template, has a little effect on each channel, avoiding effectively the image distortion caused by neglecting the link among channels.
Extraction of hue
It can be learned from the image of mature Lingwu long jujubes that there is the difference between Lingwu long jujubes and its surrounding environment in hue. Figure 3 shows the pixelvalue cross sections along line segments which run through part of foreground image and background image. It shows that the value of R component is higher than G component and B component about 20 dB in the position of foreground image and part of background image.
As shown in Fig. 4, the results of image segmentation are based on the above statistical law.
It can be learned that the segmented images based on statistical law are a set of defects. For example, parts of Lingwu long jujubes are neglect. Target image contains a lot of lying in background image, with serous adhesion between target image and background image. However, based on the statistical facts, the R component of Lingwu long jujubes has the dominant position. That is to say that the hue of the Lingwu long jujubes is different from others with tendency of red. Therefore, the hue information extracted that can be obtained by converting the color space is used for post processing. As shown in the formula [14], this paper extracts hue by transforming RGB color space into HSV color space.
As shown in Fig. 5, hue component is extracted by library function, MATLAB’s own function.
Deflection of hue
The value of hue is described in terms of angle in HSV color space, and it ranges from 0° to 360°, which is characterized by the end to end. That results in the values in the area of 0°+ and 0°− have difference although the color is similar near the area of 0°. That is the reason why images of hue component appear the phenomenon which the part of 0°+ is partial black and 0°− is partial white. This paper will make the hue rotate certain angle solves the problem. As shown in Fig. 6, it is the result of hue deflection.
Maximum entropy method and mathematical criterion
Maximum entropy method [15]
Assuming that the number of gray levels of an image is L, the gray scale of the image is {0, 1, 2, ⋅ ⋅ ⋅ ⋅ ⋅⋅, L − 1}. The number of pixels with gray scale k is n _{ k }. The total number of pixels of the image is
The probability of the occurrence of gray is
In the above formula, p _{ k } ≥ 0 and \( {\displaystyle \sum_{k=0}^{L1}{p}_k}=1 \).
Threshold T is chosen to classify images into two categories: D _{1} = {0, 1, ⋅ ⋅ ⋅ ⋅ ⋅⋅, T} and D _{2} = {T + 1, ⋅ ⋅ ⋅ ⋅ ⋅⋅, L − 1}, where T ∈ {0, 1, ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ L − 1}. The entropy of class D _{ 1 } and class D _{ 2 } are
where \( P{\mathrm{th}}_1={\displaystyle \sum_{k=0}^T{p}_k} \), Pth_{2} = 1 − Pth_{1}.
The entropy of the image is
The entropy is used as the criterion to judge the effect of segmentation images in different threshold (Fig. 7). What the entropy is larger means the more information is contained in the segmented image. In other words, it has the better segmentation result. Therefore, the optimal threshold
Mathematical criterion
3σ criterion is also called the Lagrandian criterion. When the experimental data are normal or nearly normal distribution, it can be determined by a certain probability interval that is desired data.
In the normal distribution, σ represents standard deviation and μ represents the value of mean. x = μ is the axis of symmetry.
The mathematical (3σ) criterion [16]:

The probability of the numerical distribution is 0.6826 in the area (μ − σ, μ + σ).

The probability of the numerical distribution is 0.9544 in the area (μ − 2σ, μ + 2σ).

The probability of the numerical distribution is 0.9974 in the area (μ − 3σ, μ + 3σ).
It can be considered that the probability is almost entirely concentrated in the interval (μ − 3σ, μ + 3σ) beyond the possibility of this range accounted for less than 0.3%.
For the images of Lingwu long jujubes, the histogram of its hue is obtained, as shown in Fig. 8.
As can be seen from Fig. 8, data distribution of each peak approximate normal distribution in the histogram. The mean and standard deviation of class D _{ 1 } respectively is
The mean and standard deviation of class D _{ 2 } respectively is
Distinguishing the value function of different objects
It can be considered that two objects can be distinguished when gray distribution meets formula (13). The value of α can be adjusted properly according to different condition. The value of α is less than 3.
Adaptive adjustment of threshold
It can be seen from the adjusted histogram that the gray value of the hue component is in the smallest part compared with other parts, which is convenient for the threshold adaptive adjustment to segment the target correctly.
Image segmentation is the process of distinguishing target objects from background objects. It is possible to use multiple maximum entropy method to segment the target objects correctly when more than two different regions need to be segmented in the image. Each segmentation needs to judge the position of target regions relative to the threshold. It may be found how to select the threshold that the hue of Lingwu long jujubes is located in the smallest part in hue histogram.
According to mathematical (3σ) criterion, it is considered that the two targets can be distinguished when its gray distribution satisfies formula (13). There are multiple targets in the image in the process of actual segmentation. Therefore, the image often needs to be segmented several times for segmenting the target objects. But there is a case where the target objects are not divided and the formula (13) is satisfied. In other word, whether the regions segmented are target objects or not, the formula (13) is satisfied when two regions can be distinguished.
This paper transforms the value function from formula (13) to formula (14) according to the characteristics of the hue of Lingwu long jujubes.
Two objects can be distinguished in the condition J ≥ 0.
Assuming J(q) is the value of value function when image is segmented q times, when it meets the relation,
J(q) > 0 and J(q + 1) < 0 or
J(q − 1) < 0 and J(q) > 0
T corresponding to the J(q) is taken as the division threshold.
The results of the segmented image are shown in Fig. 9.
Generally, number 1 represents target objects and number 0 represents background objects. So there need be transformation of image complement, as shown in Fig. 10.
Post processing of image segmented
For post processing of image, there are several methods including mathematical morphology, labeling, and feature selection.
Mathematical morphology
Mathematical morphology contains four basic operations: dilation, erosion, open, and close.
The dilation is used to smooth the boundary and fill the small void, the formula
Erosion is used to eliminate small and nonsensical points, the formula
Open is used to eliminate small and meaningless points and separate the target object of adhesion, the formula
Close is used to fill small holes, connecting adjacent objects, the formula
As a result of the segmented image, a part of the background objects is also divided into target objects. At the same time, there are bad phenomena including small holes in the image and adhesion between background objects and target objects in some location. Based on the above situation, this paper uses open algorithm to eliminate the small and meaningless points, with the separation of the target object of adhesion; the effect is shown in Fig. 11.
Labeling and feature selection
Labeling is a method of processing an area in which the same areas are marked with the same marks and the different areas are marked with different marks.
The feature parameters include the area, perimeter, compactness, moment, and eccentricity. Different objects can be differentiated and extracted by using those feature parameters.
In this paper, the area is selected as the characteristic parameter extracting target. Firstly, the different object in the image is marked by the method of labeling, and then, the area of each object is calculated to extract the target objects through threshold set.
Algorithm steps
In order to improve the accurate segmentation rate of Lingwu long jujubes, this paper comprehensively uses mathematical criterion, mathematical statistical analysis, maximum entropy method, and feature selection to segment the final target objects successfully (Fig. 12). The algorithm flow chart is shown in Fig. 13.
Key steps:

Step 1: input original image.

Step 2: preprocessing of original image.

Step 3: extract hue information.

Step 4: maximum entropy method is used to obtain threshold T in the range from 0 to G, then calculate the mean M1,M2 and standard deviation Sd1, Sd2.

Step 5: calculate J(p) by J(p) = M _{1} − M _{2} − α(Sd _{1} + Sd _{2}).

Step 6: judging whether (J(p) > 0 and J(p + 1) < 0) or (J(p1) < 0 and J(p) > 0) when cycle two or more time. If J(p) meets the criteria, go to step 7. If not, let G = T and go to step 4.

Step 7: image is segmented though the threshold which J(p) obtained is corresponding to T.

Step 8: post processing by using mathematical morphology.

Step 9: output image segmented.
The result of experiment and analysis
In this paper, 30 images of Lingwu long jujubes are segmented. The reliability of the algorithm is tested by the wrong segmentation rate. The statistical results are shown in Table 2.
The number of pixels in the artificial segmentation is calculated by Photoshop which extracts the target objects of Lingwu long jujubes. The number of pixels extracted wrongly is the difference between region in the artificial segmentation and region extracted correctly by the algorithm (Fig. 14). Wrong rate is a ratio that takes into account the numbers of pixels segmented wrongly and the numbers of pixels in the artificial segmentation. It can be learned from the table that the wrong rate is 10.40% and the correct rate is 89.60%. For different images that have similar background, the results are not same. The highest error rate can reach 28.19%, while the smallest error rate is only 3.06%. On the experiment of 8th, 13th, 15th, and 21st, the accuracy of the segmentation is different although those images have similar ratio between the area of Lingwu long jujubes and the area of whole image. The rate of wrong segmentation is 28.19% on the experiment of 8th, while the others are less than 10%. For no. 8, no. 17, and no. 27, the wrong rate is more than 20%.
The possible reasons that appear in the above phenomenon are as follows:

1.
The image segmentation is performed by the hue information, and the more cyan region of Lingwu long jujubes is not identified.

2.
There is the same trend of hue information among adjacent regions in target objects and background objects.
The average running time of 30 similar images is 1.3132 s. Compared to image in the artificial segmentation, the effect of segmentation by algorithm of this paper is following Fig. 15c, d.
It can be learned from 14 (a), 14 (b), 14 (c) and 14 (d), both images segmented of the new algorithm and images of the artificial segmentation have the same effect of segmentation, with accurate regions of extraction except the small error in the local area.
Discussion and conclusions
In this paper, the feature of Lingwu long jujubes can be extracted well, but the dependence of feature extraction is large:

1.
The feature extraction depends on the red range of the image and not consider the blue part of the date.

2.
The angle of rotation is uncertainly. If hue angle of rotation is too large, the local hue value of the date changes greatly, which is not conducive to the segmentation.

3.
Uncertainly of parameters in mathematical criterion. It is possible that the same parameter may have a large difference in the two types of divisions with respect to the slight difference in hue.
Based on the characteristics of the growth environment of Lingwu long jujubes, this paper is first to make statistical analysis of information to find out the difference between Lingwu long jujubes and other parts in the image and then extract the color information by transforming color space from RGB and HSV. Combining the distribution characteristics of red, the hue is rotated at certain angle. By analyzing the histogram of hue information, there are many valleys and peaks and each mountain peak is approximately normal distribution. There are many valleys and peaks, and each mountain peak is approximately normal distribution by analyzing the histogram of hue information. Therefore, the maximum entropy method and mathematical criterion are adopted to adaptively select the appropriate threshold. Finally, mathematical morphology, labeling, and feature selection are used to post processing to extract accurate target objects. A set of mathematical methods such as statistical analysis, maximum entropy, mathematical criteria, and mathematical morphology are used in this paper, which can well control the adaptive threshold selection and find the optimal threshold. The correct rate of segmentation is 89.60% through the test of 30 similar images of Lingwu long jujubes. In the literature [17, 18], the segmentation of the Lingwu long jujube image is based on the L * a * b color space and the color difference fusion. The correct segmentation rate of Lingwu long jujube is over 92.6% and the running time is 2.1889s. In this paper, the average running time, which is higher than 0.8757 s, is 1.3132 s. Although it is a little lacking in the correctness of segmentation, it greatly improves the operation speed. This method can meet the requirement of image realtime processing and can provide the theoretical basis for intelligent harvesting of Lingwu jong jujubes based on the requirement of segmentation accuracy.
References
K Vestlund, Aspects of automation of selective cleaning. Doctoral thesis, Swedish University of Agricultural Sciences, 2005
K Vestlund, T Hellström, Requirements and system design for a robot performing selective cleaning in young forest stands. J. Terramech. 43(2006), 505–525 (2006)
T Hellström, Autonomous navigation for forest machines. A Project Pre. 48(2), 115–130 (2002)
T Østensvik, KB Veiersted, E Cuchetc, P Nilsen, JJ Hanse, C Carlzon, J Winkel, A search for risk factors of upper extremity disorders among forest machine operators: a comparison between France and Norway. Int. J. Ind. Ergon. 2008(38), 1017–1027 (2008)
O Ringdahl, Path tracking and obstacle avoidance for forest machines. Master’s thesis, University of Umea, 2003:31–48.
NA Clark, RH Wynne, DL Schmoldt, M Winn, An assessment of the utility of a nonmetric digital camera for measuring standing trees. Master’s thesis, Virginia Polytechnic Institute and State University, 1998
AD Legues, JA Ferland, CC Ribeiro, JR Vera, A Weintraub, Atabu search approach for solving a difficult forest harvesting machine location problem. Eur. J. Oper. Res. 179(2007), 788–805 (2007)
JM Kan, Computer vision based method for 3D reconstruction of the standing trees. Doctor's thesis, Bejing Forestry University, 2008.
JM Kan, WB Li, RS Sun, Computer vision based method of automatic measurement of trunk and branch diameters of standing trees. J. Bejing For. Univ. 29(4), 5–9 (2007)
YT Wang, Research on methods of Lingwu long jujubes’ localization and maturity recognition based on machine vision. Doctor's thesis, Bejing Forestry University, 2014.
C Yan, Y Zhang, J Xu, F Dai, L Liang, QH Dai, F Wu, A highly parallel framework for HEVC coding unit partitioning tree decision on manycore processors. IEEE Signal Process. Lett. 21(5), 573–576 (2014)
C Yan, Y Zhang, J Xu, F Dai, J Zhang, QH Dai, F Wu, Efficient parallel framework for HEVC motion estimation on manycore processors. IEEE Trans. Circuits Syst. Video Technol. 24(12), 2077–2089 (2014)
J Yang, Shuzi Tuxiang Chuli Ji MATLAB Shixian (Publishing House of Electronics Industry, Beijing, 2014)
M Sun, Shuzi Tuxiang Chuli Yu Fenxi Jichu—MABLAB He VC++ Shixian (Publishing House of Electronics Industry, Beijing, 2013)
T Cove, Elements of information theory (China Machine Press, Beijing, 2008)
GH Wu, Gailv Lun Yu Shuli Tongji (China Renmin University Process, Beijing, 2011)
YT Wang, WB Li, S Pang, JM Kan, Segmentation method of Lingwu long jujubes based on L*a*b* color space. TelkomnikaIndonesian J. Electr. Eng. 11(9), 5344–5351 (2013)
YT Wang, JM Kan, WB Li, CD Zhan, Image segmentation and maturity recognition algorithm based on color features of Lingwu long jujube. Adv. J. Food Sci. Technol. 5(12), 1625–1631 (2013)
Funding
This work is supported by the National Natural Science Foundation of China (31660239) and the Ningxia Natural Science Foundation of China (NZ15007).
Authors’ contributions
YTW and YPD drafted the main part of the manuscript. YPD carried out the experiments. JRX organized the experimental data. JRX, BHL, CHM, and YYG conceived of the study, participated in the design of the experiments, and helped modify the manuscript. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wang, Y., Dai, Y., Xue, J. et al. Research of segmentation method on color image of Lingwu long jujubes based on the maximum entropy. J Image Video Proc. 2017, 34 (2017). https://doi.org/10.1186/s1364001701825
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s1364001701825
Keywords
 Maximum entropy
 Image processing
 Image segmentation
 Adaptive threshold
 Lingwu long jujubes