Preservation of image edge feature based on snowfall model smoothing filter

Abstract

This paper proposed a snowfall model as a novel smoothing filter. The pixel composition of the image was similar to the geographic features, so it could be smooth because of snow accumulation. In the snowfall processing, luminance changes are linked to terrain and snowfall amount. Curvature and luminance gradient decided the amount of snowfall; the amount of snowfall became large on the parts where the curvature was large, and it became little on the parts where the gradient was steep. Snowfall algorithm simulates the natural snowfall process, which nonlinear diffusion and the target feature could be preserved well. Snowfall model has the same function as the Gaussian filter. The number of regions was reduced after Gaussian filter and snowfall model smoothing, respectively. The contrast experiment was carried out based on Watershed algorithm. The image area segmentation that pretreated through snowfall model was compared with Gaussian filter smoothing. The experimental result showed that the proposed snowfall model was a smoothing filter. It was able to realize edge preservation, which was the original purpose, and it was also possible to apply to region segmentation.

1 Introduction

Image smoothing makes images easier for feature extracting and recognizing, and it could eliminate image space noise [1, 2]. Usually, image noise is mainly composed of high-frequency component; smoothing filter can enhance image low-frequency component to remove the high-frequency noise [3,4,5]. The typical smoothing process includes mean filter, median filter, and Gaussian low-pass filter. Mean filter is used to reduce noise in the image process, and it is an intuitive and simple method to make the image smooth [6]. Its principle is reducing the amount of intensity variation on pixel and the next [7]. Mean filter is based on the template for image convolution operation to achieve smooth processing [8]. It uses the average grayscale value of template pixels instead of the object pixels [9]. Mean filter output is an average value of the pixels within the filter template territory. Processing result reduces the image grayscale sharp changes and reduces the noise, but also has the negative effect of the fuzzy edges [10]. Median filter is a nonlinear digital filtering technique, and it is used to perform noise reduction in an image [11]. Pixels on the local area are sorted according to grayscale and according to the intermediate value of statistical sorting to replace the object pixel [12]. If the nature of the noise is not relevant to a random noise with the contents of the image, the median filter is effective and the effectiveness is better than the mean filter [13, 14]. Based on the median filter method, random spike noise signals are removed and edge blur degradation is reduced [15, 16]. Gaussian low-pass filter has the properties of having no overshoot to a step function input while minimizing the rise and fall time [17]. The pixels of space distribution are used as a reference value in an image denoising process; the weight distribution is determined based on distance to the target pixel, according to the calculate principle, the Gaussian low-pass filter blur effect close to human natural vision [18, 19]. Using a weight template the standard deviation for the convolution operation can realize the image smoothing process, with a standard deviation value becoming larger and the image smoothing effect becoming stronger. The above-described smoothing process methods do not retain the image edge effect, so the image processing results lose the edge characteristics of the original image [20, 21].

In the process of image generation and transmission, noise may be physically included due to various factors. Since noise is an obstacle to image processing, so it needs to be removed [22]. When the mechanism for generating such noise can be mathematically modeled [23], a noise cancelation method corresponding to the model could be used, but if it is not, the noise is reduced by the smoothing method [24, 25]. Image noise contains many high-frequency components; smoothing is a method of using a low-frequency emphasis filter to blur images to reduce noise [26].

In our research, we proposed a new smoothing method that reduces the noise through pseudo-snow in the image, making the snowfall in the natural world and the terrain becoming flat. Therefore, this research proposed the snowfall model as a smoothing technique for an image.

2 Snowfall model algorithm

In general, when it was snowing, even if the amount of snow was small, a smooth convex or concave flat surface would be smoothed. As the amount increases, the snow covered sharp edges and concave parts, resulting in flat grounds. Figure 1 shows an example of the smoothing effect when the snow falls twice. The distribution of image grayscale was deemed topographic map in image process; natural snow covered terrain was simulated based on mathematical modeling of the snow course. The image was compared to geographical features, and it was smoothed so that the snow may pile. The steps of features of the snowfall model algorithm are shown as follows.

• (1) All of image pixels additive operation execution;

• (2) Concave waveform smoothing process as a priority target;

• (3) Convexity larger waveform as a reservation target.

Fig. 1

Snowfall processing illustration. a Edge grayscale gradient. b Stage one snowfall process. c Stage two snowfall process

Based on the above image processing, desired result of the smoothing process is shown in Fig. 1.

In order to achieve the smoothing effect of the snowfall model, the luminance slope and curvature parameters need to be defined. The luminance gradient value and curvature of convex waveform were required as snowfall calculation parameters to achieve self-adaptive smoothing process. Curvature and luminance gradient decided the amount of snowfall. The upward convex waveform curvature was given a negative value and recorded as negative curvature, and downward convex waveform curvature was given a positive value and recorded as positive curvature. When curvature value and gradient value were relatively large, we need to give less snowfall in the image process. Similarly, if positive curvature value and gradient value were relatively large, we need more snowfall to achieve a blur effect. Snowfall calculation parameters were divided into horizontal and vertical directions. One-dimensional differential filter and two-dimensional differential filter were used to calculate the gradient value.

To prevent an unnatural snowfall effect, planarization process of waveform was needed for getting a natural snowfall effect. In a snowfall process as shown in Fig. 2a, curvature values of all grayscale were calculated, then the value of the curvature has changed waveform section as a processing target. Curvature value was from positive to negative corresponding to the waveform changes from concave to convex; thus, the curvature value became 0 from positive. The average of the adjacent grayscale values was set as the target grayscale value after snowfall process, the first planarization process effect as shown in Fig. 2b and waveform processing by convex to concave change as shown in Fig. 2c. Similarly, the average value of adjacent grayscale was set as the concave grayscale value, and snowfall effect after planarization process effect as shown in Fig. 2d.

Fig. 2

Snowfall correction process. a Luminance value. b Waveform is concave to convex. c Waveform is convex to concave. d Processing completed

Gradient value was an important parameter in image grayscale level changing, and it was a snowfall gradient calculation parameter in the snowfall model. Gradient values G_ra were calculated by the one-dimensional differential filter; because the positive and negative gradient values were not considered in the calculation, so the absolute value of the gradient values was calculated as shown in Eq. (1), in order to satisfy the calculation of snowfall, which was normalization between 0 and π/2. Figure 3 shows the waveform of the luminance and absolute luminance gradients.

$$ Gra=\left|\frac{g-0}{255-0}\times \frac{\pi }{2}\right| $$

(1)

Fig. 3

Luminance and luminance gradients

3 Method—smoothing based on the snow fall model

Curvature was used to indicate the curve or surface degree of bend. It had important applications in image processing, curvature K was calculated as Eq. (2).

$$ K=\frac{x^{\hbox{'}}\cdot {y}^{"}-{x}^{"}\cdot {y}^{\hbox{'}}}{{\left({\left({x}^{\hbox{'}}\right)}^2+{\left({y}^{\hbox{'}}\right)}^2\right)}^{3/2}} $$

(2)

First-order derivative and second-order derivative in the x direction of image function f(x, y) values were calculated by Eq. (3) and Eq. (4).

$$ \frac{\partial f}{\partial x}=f\left(x+1\right)-f(x) $$

(3)

$$ \frac{\partial^2f}{\partial {x}^2}=f\left(x+1\right)+f\left(x-1\right)-2f(x) $$

(4)

Similarly, first-order derivative and second-order derivative in the y direction were calculated by Eq. (5) and Eq. (6).

$$ \frac{\partial f}{\partial y}=f\left(y+1\right)-f(y) $$

(5)

$$ \frac{\partial^2f}{\partial {y}^2}=f\left(y+1\right)+f\left(y-1\right)-2f(y) $$

(6)

When the luminance values draw a concave waveform, the curvature value K was positive; conversely, curvature value k was a negative value. If the waveform of the grayscale value was linear, K was 0. Figure 4 shows the luminance value and curvature waveform corresponding relation.

Fig. 4

Luminance value and curvature

A pair of luminance gradient g(x) and curvature k(x) was calculated to produce S(x), i.e., a measure of snowfall amount at a pixel x. More specifically, a weight G(x) was defined so that cosG(x) was maximum with gradient g(x) = 0 and smaller with larger g(x) in magnitude.

In addition, another weight based on the curvature was used, where it became larger with negative values (concave) and smaller with positive ones (convex). This second weight was defined by exp(k(x)/10) as an example. A combination of the two weights was used to calculate S(x). Integrating the measure S(x) over an entire image, S(total) was to get a measure of snow amount for the entire image. S(total) was not depth of snow. Therefore, a scaling factor was needed and it was collated with real depth of snowfall. However, the process above corresponded to one-time snowfall. It was repeated, if further smoothing was necessary. If repeated once, it corresponded to depth. It should be noted that gradient and curvature should be calculated again prior to the next snowfall.

Snowfall amount S was calculated using gradient value and curvature value. Horizontal snowfall amount S_H was calculated by horizontal direction gradient absolute value G_H and curvature value K_H. Vertical direction S_V could be calculated as the same. The total amount of snowfall S was calculated by Eq. (7).

$$ {\displaystyle \begin{array}{l}S={S}_H+{S}_V\\ {}\kern0.75em =\exp \left(\frac{K_H}{10}\right)\cos {G}_H+\exp \left(\frac{K_V}{10}\right)\cos {G}_V\end{array}} $$

(7)

The curve of the relationship between the curvature K and snowfall indicated that the larger K means a greater amount of snowfall S. In addition, the relationship between the gradient absolute value G_ra and the snowfall amount S indicated that the greater the gradient absolute value G_ra, the smaller the amount of snowfall. For basic snowfall controlled parameters in the horizontal axis, the gradient of horizontal was G_H(x_i, y_j) and the curvature of horizontal was K_H(x_i, y_j). For basic snowfall controlled parameters in the vertical axis, the gradient of vertical was G_V(x_i, y_j) and the curvature of vertical was K_V(x_i, y_j). The detailed calculation in image processing of snowfall is shown in Fig. 5. In order to simulate the process and amount of natural snowfall as much as possible, the amount of snowfall in the horizontal direction and the amount of snowfall in the vertical direction are considered comprehensively; because the specific application needs are not considered at present, total snowfall is finally designed as the average of horizontal direction and vertical direction. In the subsequent research, the ratio of the amount of snowfall in the horizontal direction and the amount of snow in the vertical direction could be adjusted according to the specific application requirements.

Fig. 5

Calculation in image processing of snowfall

In the process of snowfall, the whole image is taken as the processing object. In the specific calculation process, in each local snowfall amount calculation, the adjacent five pixels of the target pixel is treated as the local snowfall range. If the x direction is calculated, the left and right adjacent five pixels are used as the local snowfall range. If the y direction is calculated, the top and bottom five pixels adjacent are used as the local snowfall range.

Based on the above calculation step, we calculate the snowfall amount to achieve image smooth processing, and additional operation of grayscale value might be causing an unnatural snowfall effect. Grayscale value waveform might be becoming convex waveform, although the original waveform was concave, or the original grayscale value waveform for convex became a concave waveform after the snowfall process; these phenomena are not natural snowfall effect in image smoothing.

4 Results and discussion

In this experiment, the snowfall model is used as the preprocessing of the region segmentation based on the watershed algorithm to verify the smoothing effect of the snowfall model. In order to verify the smoothing characteristics of the snowfall model, the image with complex background was processed with the snowfall model. With the increase of snowfall times, the smoothing effect is enhanced. The step length of snowfall chooses 30 after analyzing the effect of different snowfall times, which can clearly distinguish the difference of smoothing effect. In the comparison experiment, 30, 60, and 90 times the snowfall processing was carried out on the original image. Figure 6a is an original image, 30, 60, and 90 times was snowfall processed in the original image, with smoothing results as shown in Fig. 6b–d. As the number of snowfall increases, the smoothness is more obvious, so the smooth filtering effect of Fig. 6d is the most significant.

Fig. 6

Snowfall results of an image. a Original image. b 30 times snowfall. c 60 times snowfall. d 90 times snowfall

In order to compare and analyze the local effects of snowfall processing, a row of pixels at y = 150 is taken as the analysis object. The changes in the pixel values of the original image and the 30, 60, and 90 times snowfall processing are shown in Fig. 8. The x-axis of Fig. 8 represents positional information in the x-direction of the pixel, and the y-axis represents the luminance value. The red line represents the luminance distribution of the original image. The three lines from the bottom to top represent the change of luminance after the 30, 60, and 90 times of snowfall processing. By observing the waveform changes in Fig. 8, we can see that the small concave in the original image waveform becomes more and more smooth with the increase of snowfall times. From an analysis and comparison of three snowfall processes, waveform became smooth gradually. In Fig. 7, a handful of waveform values were less than the original image, the reason was the normalization in snowfall process.

Fig. 7

Snowfall and smoothing at y = 150 pixel

As expected, moderate luminance variation and concavity were smoothed while large luminance variation and steep edges were preserved. At the same time, this technique increased the luminance value with each snowfall in the same way as snow falls on the ground, resulting in increase in the average luminance value. However, it did not matter since this technique was expected for use in conjunction with differential filters such as the watershed segmentation.

Gaussian low-pass filter has the property of having no overshoot to a step function input while minimizing the rise and fall times. The pixels of space distribution were used as a reference value in the denoising process. The weight distribution was determined based on distance to the target pixel, so the Gaussian low-pass filter blur effect was close to human natural vision. Equation (8) is a two-dimensional Gaussian function of the standard deviation σ.

$$ G\left(x,y\right)=\frac{1}{2{\pi \sigma}^2}\exp \left(-\frac{x^2+{y}^2}{2{\sigma}^2}\right) $$

(8)

Using a weight template, which is from Eq. (8), the convolution operation could realize image-smoothing processing, with σ value becoming larger and the image smoothing effect becoming stronger.

The above-described smoothing process methods do not retain image edge effect, so the image processing results lose edge characteristics of the original image. To retain the original edge characteristics of the smoothing process was necessary for image preprocessing method; Fig. 8 is a grayscale level distribution of nine local neighborhoods, and average grayscale value was calculated on the minimum variance field as the output grayscale value of the center. This process could keep the original image edge features and remove noise and retain the smoothing process effect.

Fig. 8

Nine local area luminance change calculation

Based on the watershed algorithm for image segmentation, first we find the minimum (minima) of grayscale difference and carried on the label, the label number was the serial number of each region. Grayscale value was based on the baseline moving upward starting from 0 to 255; mobile search for the maximum value was the watershed point, so as to complete the image region segmentation based on watershed algorithm.

The watershed algorithm for image segmentation could be induced in regional excessive segmentation, so image-smoothing pretreatment was needed before the segmentation. To verify the validity of the snowfall model, Gaussian low-pass filter was as the pretreatment of the regional segmentation based on the watershed algorithm. Comparative analysis experiments of edge-preserving smoothing processing result showed that good blur effect was achieved based on optimal value parameters. The pretreatment processing result was segmented with regional segmentation based on the watershed algorithm. To ensure the effectiveness and rigor of the proposed algorithm, the original image and three kinds of snowfall image region segmentation experiments were implemented. Regional segmentation result is shown in Fig. 9. The result showed snowfall pretreatment processing had a different edge information. Compared with the conventional smoothing algorithm, the snowfall model could integrate the regions while maintaining the edges.

Fig. 9

Region segmentation results based on the snowfall model. a 30 times snowfall. b 60 times snowfall. c 90 times snowfall

The Caucasian low-pass filter and snowfall models with different parameters were compared in detail. With the increase of parameters σ, the smoothing effect of Gaussian filter is enhanced; with the increase of snowfall times, the smoothing effect of snowfall model has the same characteristic. In order to compare with the Gaussian filter, the segmentation results corresponding to different snowfall times and the segmentation results corresponding to the different Gaussian filter parameters σ are investigated and analyzed. Finally, the number of regions and the number of edges are both not much different as a group to make comparative analysis. Table 1 summarizes the changes in the number of boundary lines that overlap the number of regions and the boundary in each smoothing method with different parameters.

Table 1 The number of regions and edges in smoothing method

Figure 10 shows a comparison by drawing each of the smoothing methods of Table 1. The outline of the image edge and region segmentation image had carried on the contrast. The snowfall image edge information and Caucasian low-pass filter processing results are showed in Fig. 10; the vertical axis was the number of area information, and the horizontal axis was the number of overlap edge. The blue line above represents the snowfall model, and the red line below represents the Caucasian low-pass filter. An upward line proved the algorithm had better edge retention characteristics. The upper left point was the original data, with the smoothing process increasing the number of regions was slowly decreased; at the same time, dislocation phenomenon of image edge features was also enhanced. The adaptive snowfall model retained the image edge features while achieving a good regional segmentation. Based on the analysis of experimental data where we might draw a conclusion, in five kinds of preprocessing algorithm, the snowfall model was an optimal smoothing process algorithm.

Fig. 10

Quantitative analysis and comparison of information

Segmentation results based on the watershed algorithm with snowfall (seven times) and Gaussian filter (four times) are shown in Fig. 11; the difference of segmentation results was more obvious in the background region, and more regions were merged by using snowfall. Increasing the process number of Gaussian-filter and snowfall, the segmentation results were compared and analyzed. Less area was merged in larger grayscale value part (higher luminance part), and more area was merged in lower grayscale value part (lower luminance part) by using snowfall. Because region was not merged in the tiny part by snowfall, result in the overall number differences of region was not big, but snowfall had better effect than Gaussian-filter, such as background and character area.

Fig. 11

Results of image edge information comparison

5 Conclusions

Snowfall model as a novel edge preserving smoothing filter based on nature snowfall was presented in this paper. The moderate luminance variation and concavity are smoothed while large luminance variation and steep edges are preserved. At the same time, this technique increases luminance value with each snow fall in the same way as snow falls to the ground, resulting in an increase in average luminance value. In order to verify the smoothing effect of the snowfall model, it was applied to the pretreatment of region segmentation based on the watershed algorithm. In addition, compared with Gaussian-filter, experimental results showed that the snowfall model had the same function as the Gaussian-filter. Although region was not merged in the tiny part, smoothing effect of snowfall was better than Gaussian-filter in the lower luminance part. However, it does not matter since this technique is expected for use in conjunction with differential filters such as watershed segmentation. In the future, we would further study the parameters of snowfall model on smoothing effect and application, especially the matching degree with the watershed algorithm, and suppressing over-segmentation.