 Research
 Open
 Published:
A denoising method based on L0 gradient minimization and guided filter for ancient Chinese calligraphy works on steles
EURASIP Journal on Image and Video Processingvolume 2019, Article number: 32 (2019)
Abstract
A clear stele image of ancient Chinese calligraphy pieces is very useful for studying ancient Chinese calligraphy. However, due to hundreds of or even thousands of years of natural or artificial damage on stele, images of ancient Chinese stele calligraphy works usually suffer from a large amount of image noise, and which usually leads to a poor visibility. To address this problem, in this paper, we propose a denoising method based on L0 gradient minimization and guided filter. It consists of two main operations in sequence: First, L0 gradient minimization is utilized to obtain a randomnoise free map, and then the randomnoise free map is used as a guided image, and convoluted with its corresponding original noised stele image by a guided filter to obtain an edge preserved randomnoise free image. Finally, the eightconnection regionbased denoising technique is followed to remove antlike isolated blocks. Experiments demonstrate that the proposed method is superior to several recent published stele image denoising techniques in terms of preserving the character structures.
Introduction
Ancient Chinese calligraphy works are very useful for people to learn and study Chinese history and culture [1]. From these works, we can enjoy the beauty of ancient Chinese culture and learn about how the ancient Chinese lived; furthermore, we can learn about where Chinese history and culture come from and where it will go. Of these calligraphy works, most of them used steles as their carriers. However, because of hundreds of or even thousands of years of natural or artificial damage on steles, images of ancient Chinese stele calligraphy works usually suffer from poor visibility with a large amount of noise, and the structure of characters in regions of the stele images is too spoiled by noise to see, as shown in Fig. 1. Hence, it is highly desirable to remove noise from ancient Chinese stele calligraphy images.
However, to the best of our knowledge, though there are many works on image denoising [2,3,4], few works have focused on ancient Chinese calligraphy images. For this reason, a denoising method for ancient Chinese calligraphy works on steles based on L0 gradient minimization and guided filter is proposed in this paper. We first utilize L0 gradient minimization to smooth the noised stele image. The goal of this step is to obtain randomnoise free maps. Then, we take the randomnoise free map as a guided image and convolute it with its corresponding original noised stele image via a guided filter. The goal of this step is to obtain an edge preserved randomnoise free image. Finally, we used the eightconnection regionbased denoising technique as a tool to remove isolated blocks. Experiments show that our method is superior to several recent published stele image denoising techniques in preserving characters’ structures.
Our contributions in this work are summarized as follows:

1)
We put forward a L0 gradient minimization guided image filter for randomnoise removal. It consists of two operation steps: First, to smooth a noised stele image by using L0 gradient minimization technique. Then, to compute an edgepreserved noise free image by convoluting the smoothed image with the input noised stele image.

2)
We employ the eightconnected domain technique for antlike noise removal. The effectiveness of the proposed method is demonstrated by experimental results.
The remainder of this paper is structured as follows: Section 2 simply introduces the related works. Section 3 describes the proposed method in detail. Section 4 presents all the experimental results. Section 5 concludes the paper.
Related work
Interests regarding image preprocessing, including image denoising [5] and image enhancement [6], especially on ancient Chinese calligraphy image enhancement [1,2,3,4, 7,8,9,10] have seen increasing in recent years; for instance, Zheng et.al. [1] presented a denoising method for stele images using guided filter on the L channel. Wang et al. [7] used a threshold method to remove noise from a Chinese calligraphy image. In [3], an adaptive shock filter based on anisotropic diffusion is proposed. In [8], the Laplacian pyramid is combined with the fractional Fourier domain to extract the structures of characters with rotation angles at different scales. In [9], an anisotropic diffusion filter with the width information of different strokes was proposed for noise removal. In [10], structure characteristics of Chinese character strokes and runlength statistics were used for stele denoising. In [11], an integration of multiple denoising filters was proposed for removing block and line noise from stele images. In [12], KSVD dictionary learning was used for denoising as well as for preserving the character structures.
Although there are many works on stele image processing, research on stele image denosing are still very limited. Solutions for ancient Chinese calligraphy image denoising are still opened.
Recently, a piece wise filtering model termed L0 gradient minimization has been employed for image denoising. By optimizing an energy function based on prior gradient sparsity, the L0 gradient minimization model shows a decent ability to perform image denoising and a strong ability to preserve sharp features [13] in respect to traditional image denoising method, such as Gaussian filter and shock filter. Similar works are also found in [14, 15].
Motivated by the aforementioned research, in this paper, we employ L0 gradient minimization as well as a guided filter for ancient stele calligraphy image denoising.
Proposed method
Overview
Figure 2 shows the framework of our presented method. Firstly, a stele image is smoothed using L0 gradient minimization. The goal of this step is to obtain a randomnoise free map. This randomnoise free map is then convoluted with its corresponding original noised stele image to obtain an edge preserved randomnoise free image. Next, a connection region based denoising technique is used to remove isolated blocks. Details of each step are described as follows.
Randomnoise free map computation with L0 gradient minimization
The goal of the computation in this section is to obtain a randomnoise free map. For this purpose, there are several filters that can be chosen, such as the Gaussian filter, shock filter, and L0 gradient minimization [13]. Results with these filters were nearly identical. However, with respect to Gaussian filter and shock filter, the L0 gradient minimization model shows a decent ability to perform image denoising and a strong ability to preserve sharp features by optimizing an energy function based on prior gradient sparsity [13]. For what was mentioned above, in this section, we select the L0 gradient minimization as the tools for randomnoise free map computation. Firstly, we decompose the input image into two parts using L0 gradient minimization: a base image and detailed image. The L0 norm of the gradient specifies the number of nonzero gradients [13] and drives the minimization to produce an output that is comparable to input around strong edges (as shown in Fig. 2 by the color blocks), which is based on the previous gradient.
Let I(x,y) be an input image and B(x,y) be its corresponding base image, and D(x,y) be its corresponding detail image. The gradients of I(x,y) are denoted as ∇I(x, y). Formally, we define the energy function of L0 gradient minimization as:
where γ controls the level of detail in the final image and ∇I(x, y) represents the number of nonzero gradient and is described as:
where # denotes the pixel number and ∇ indexes the operator difference.
The gradient L0 norm is usually computed at positions containing sharp edges because it denotes the numbers of nonzero gradients. In term of this view, we multiply a binary map Edge(x,y) on the original regularization term ∇I(x, y) to speed up the process.
where Edge(x,y) denotes a binary map. Any technique that can obtain sharp edges can be used to get the binary map, such as applying the differential of Gaussian (DOG) filter on I(x,y) with a threshold, or applying the Laplacian of Gaussian filter on I(x,y) with a threshold. In this paper, we apply the differential of Gaussian (DOG) filter on I(x,y) with a threshold to get the binary map. The threshold value is determined based on experiments.
To easily calculate the L0 norm of gradient via Eq. (1), two dependent variables α_{x}(x, y) and α_{y}(x, y) are introduced, corresponding to\( \frac{\partial B\left(x,y\right)}{\partial x} \) and\( \frac{\partial B\left(x,y\right)}{\partial y} \), respectively. Equation (1) is reconstructed as:
where β is used to regulate the difference between the dependent variables and its corresponding gradients. To address this minimization, an alternating minimization strategy is used, viz. We keep one of the variable sets unchanged to obtain another variable set. In our work, we keep α_{x}(x, y) and α_{y}(x, y) unchanged and use the below cost function to first obtain B(x,y):
Then, we keep the evaluated B(x,y) unchanged and through the following cost function obtain α_{x}(x, y) and α_{y}(x, y)
where we model the L0 norm of gradient as:
where H( α_{x}(x, y) +  α_{y}(x, y) ) is a binary function returning 1 when ∣α_{x}(x, y) ∣ + ∣ α_{y}(x, y) ∣ ≠ 0; otherwise, it returns 0. By alternatively computing (5, 6), we obtain the final base image B(x,y) and its corresponding detail image D(x,y). Because all of the randomnoise remains in the detail image D(x,y), we take the final base image B(x,y) as the randomnoise free map. Figure 3 shows an example of randomnoise removal results with the L0 gradient minimization operation. It is observed that the L0 gradient minimization smoothed image (Fig. 3b) contains little noise compared with the original image (Fig. 3a).
Recover of the Chinese character structure using a guided filter
After the L0 gradient minimization operation, we obtain a randomnoisefree image, as shown in Fig. 3b. However, we find that some stroke edge details are removed as random noise by the L0 minimization operation, as indicated by the yellow box in Fig. 3. The guided filter is an edgepreserving filter [16]. Its goal is to smooth input images by calculating the content of the guidance image. Many studies shows that edges in an image after using guided filter will change differently. For step edges, it is still step edges after using guided filter, but their ranges become smaller, which means that the step edges become smoother after guided filter; For ridge edges, if the ridge edges with small size are unaffected by the other edges, their variances are close to 0, then the ridge edges will disappear and tend to the background; Valley edges will become larger than the input. From what was mentioned above, we can see that the guided filter has well preserving ability on image edges. Therefore, it can be used for image texture recover.
For what was mentioned above, to recover the oversmoothed stroke edge details, guided filter is employed in this section.
Formally, given a guidance image I and an input image I_{in}, the corresponding guided filtering output image\( {I}_{\mathrm{out}}^{\mathrm{guided}} \) is defined as:
where ω_{k} is a window centered at the pixel x, α_{k} and b_{k} are constants in ω_{k}, respectively. These two parameters are determined as:
where μ_{k} indexes the average value of input image I_{in} in window ω_{k} , and \( {\upsigma}_k^2 \) is the variance of input image I_{in} in window ω_{k} .n_{ω} is the number of pixels in window ω_{k}. \( {\overline{p}}_k \) represents the mean of the guided image in window ω_{k}. ε represents the regularization parameter which is used to determine the changing of pixels’ intensity. The initial value of parameter ε in Eq. (9) is determined by referring [16], and its finial optimized value is determined by experiment.
In this work, for the use of guided filter, we employ the randomnoisefree map obtained in above section as a guided image and convolute it with the input noised stele image to recover the Chinese character structure in a stele image. Figure 4 shows some of the results of recovering the Chinese character structure using the guided filter. As observed from the figure, the incomplete character strokes (as show in Fig. 4b) caused by the oversmoothing of the L0 gradient minimization operation are recovered well; these strokes are indicated by the yellow boxes in Fig. 4c.
Antlike blocks removal using the eightconnected region technique
From Fig. 4, we also find that although most of the noise in the input image is removed after L0 gradient minimization filtering, there remains a few isolated antlike blocks in the smoothed images, as denoted by the red circle in Fig. 4c.
One simple relationship between pixels is connectivity. Based on the connectivity of pixels, we can easily determine a pixel is an image component or image noise. To do this, there are two connected region techniques can be used: the fourconnected region technique and the eightconnected region technique [10]. Due to the fourconnected region technique considered as neighbors, only the four pixels that share an edge (not a corner) with the pixel in question: (x + 1,y), (x − 1,y), (x,y + 1), and (x,y − 1); pixels on the diagonal could not be taken as an edge though they are serve as an effective insulator between two pixel set, and as a result, this creates undesirable topological anomalies, whereas the eightconnected region technique consider a pixel as connected not only pixels on the same row or column, but also the diagonal pixels. For this reason, also inspired by the work described [10], we utilize the eightconnected region technique [10] in this section to address the antlike blocks problem. The processing to remove antlike blocks using eightconnected region technique consists of the following three basic steps:

1)
Compute the connected components using an eightconnected domainbased search technique;

2)
Calculate each component’s area first. Then, sort all of connected components in terms of their area from large to small. Many studies index that if the area value of a block is smaller than the area value of 2/3 of the array of permutations, then the block could be taken as an isolated noise component and could be removed from an image. For this reason, we take the area value of 2/3 of the array of permutations as the value of threshold T;

3)
Remove all of isolated components when the area is smaller than the value of threshold T.
Figure 5 shows partial results of the antlike block removal via the eightconnected region technique. It is observed that almost all of the antlike blocks are removed from the input stele image, and the characters in the image are highlighted and the structures of the strokes have been preserved perfectly (as shown in Fig. 5d) compared with that of the input stele image.
Experimental results and discussions
Experimental setting
To the best of our knowledge, there is no standard ancient Chinese stele calligraphy image set available for benchmarking currently. Therefore, we collected 100 stele calligraphy images from the released image set by [1]. We also generated 50 synthetically noised stele images using Adobe Photoshop CS6 to quantitatively evaluate the proposed method.
For comparison, several state of the art image denoising methods, including Zheng’s method [1], Zhang’s method [10], L0 gradient minimization smoothing [13], and blockmatching 3D filtering (BM3D) [17], as well as methods in our two prior works [11, 12] are employed in our experiments. The results of these methods are produced in MatLab with authors suggested parameter settings. All experiments are performed on a PC with 4G RAM and a 2.60 GHz Intel Dual Core processor using MatLab R2015.
Qualitative results
Figure 6 shows partial processing results with our method; the top row shows the original images, and the bottom row shows our processing results. From Fig. 6, it is observed that our method can remove most of the randomnoise and antlike noise without spoiling the characters.
In Fig. 7, we compared our proposed method with Zheng’s method [1]. It can be observed that Zheng’s method failed to remove larger block noise from the images (Fig. 7b). However, the proposed method worked well for removing all types of noise (as shown in Fig. 7c).
In Fig. 8, we compared our method with Zhang’s method [10]. It can be seen that both the two methods worked well in removing image noise, whereas our method obtained more smooth edges (as shown in Fig. 8c), compared with the results processed by Zhang’s method [10] where a small amount of “flash” remained in the edges (as shown in Fig. 8b).
Figure 9 compares the proposed method with the L0 gradient minimization algorithm [13] and the blockmatching 3D filtering (BM3D) algorithm [17]. It is observed that our proposed method retains more details than the other two methods and does not spoil the character structures.
In Fig. 10, we compare the proposed method in this paper with our two prior works, including an integrated denoising method [11] and the KSVD denoising method [12]. It is observed that the three methods were able to take out of most image noise from the input noisy images, whereas it should be noticed that the proposed method in this paper obtains the best results with clearer images.
Quantitative evaluation
In this section, we quantitative evaluate the performance of our method on synthetic and real noised stele images, respectively.
For evaluation on the synthetic noised stele calligraphy images, peak signaltonoise ratio (PSNR) and Structure Similarity Index (SSIM) [18] are employed as evaluation measures because of available ground truth, where the larger the values are, the better the results are. Referred to [18], the definition formulation of PSNR and SSIM are described as Eq. (11, 12), respectively.
where I is a noisefree image, K is its noisy approximation, and MSE is the mean squared error (MSE) of I and K.
where μ_{I} is the average of I, μ_{K} is the average of K, \( {\sigma}_I^2 \) is the variance of I, \( {\sigma}_K^2 \)is the variance of K, σ_{I, K} is the covariance of I and K, c_{1} and c_{2} are two variables to stabilize the division with weak denominator.
Table 1 shows the evaluation result. From Table 1, as we can observe that the values of SNR and SSIM for our method in this work are 32.834 and 0.9952, that for Zheng’s method [1] are 26.801 and 0.9841, that for Zhang’s method [10] are 29.010 and 0.9858, that for L0 [13] are 27.062 and 0.9742, that for BM3D [17] are 27.960 and 0.9801, that for our prior work [11] are 32.690 and 0.9882, and that for our prior work [12] are 32.783 and 0.9916. These results clearly indexes that our proposed method in this work achieves the best denoising performance of all evaluated method in this experiment in terms of both of PSNR and SSIM.
For evaluation on real noisy stele images, we employ the overall quality index [19] as evaluation measures because of unavailable ground truth. A larger overall quality index value indexes a better enhanced visibility. Table 2 shows the evaluation results in this experiment. From Table 2, it is observed that the proposed method obtained an average overall quality index value of 0.7056, which is much higher than that of other methods, including 0.6795 for Zheng’s method [1], 0.6865 for Zhang’s method [10], 0.6538 for L0 gradient minimization [13], 0.6895 for our prior work [11], and 0.6905 for our prior work [12]. This result indicates that our proposed method in this work remarkably improves the visibility of images in respect to other methods.
Conclusions
We put forward a denoising method based on L0 gradient minimization and guided filtering method for ancient Chinese stele calligraphy image denoising in this paper. Experiments by comparing our presented method with several state of the art denoising methods show the superior of our method for stele image denoising.
In our next work, we will focus on how to restore spoiled character structures for the reason of preserving the beauty of art work.
References
 1.
X. Zheng, Q. Miao, Z. Shi, Y. Fan, W. Shui, A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimedia Tools & Applications 75(14), 8719–8744 (2015). https://doi.org/10.1007/s110420152788x
 2.
J. Chen, X. Kang, Y. Liu, Z. Wang, Median filtering forensics based on convolutional neural networks. IEEE Signal Processing Letters 22(11), 1849–1853 (2015). https://doi.org/10.1109/LSP.2015.2438008
 3.
K. Guemri, F. Drira, in Proc. of 6th International Conference of Soft Computing and Pattern Recognition, Tunis, Tunisia. Adaptative shock filter for image characters enhancement and denoising (2015), pp. 279–283
 4.
H. Li, C.Y. Suen, A novel nonlocal means image denoising method based on grey theory. Pattern Recogn. 49(1), 237–248 (2016). https://doi.org/10.1016/j.patcog.2015.05.028
 5.
Y. B, An improved infrared image processing method based on adaptive threshold denoising. EURASIP Journal on Image and Video Processing 2019, 5 (2019) https://doi.org/10.1186/s1364001804018
 6.
Z. Shi, M. Zhu, B. Guo, M. Zhao, Nighttime low illumination image enhancement with single image using bright/dark channel prior. EURASIP Journal on Image and Video Processing 2018, 13 (2018) https://doi.org/10.1186/s1364001802514
 7.
S.Z. Wang, H.J. Lee, in Proc. of 6th International Conference on Document Analysis and Recognition, Seattle, WA, USA. Dualbinarization and anisotropic diffusion of Chinese characters in calligraphy documents (2001), pp. 271–275
 8.
H.T. Nguyen, N. LinhTrung, in Proc. of 10th International Conference on Information Sciences Signal Processing and their Applications, Kuala Lumpur, Malaysia. The Laplacian pyramid with rational scaling factors and application on image denoising (2010), pp. 468–471
 9.
T. Koga, E. Uchino, N. Suetake, G. Hashimoto, T. Hiro, M. Matsuzaki, in Proc. of the 7th Conference on Circuits, Systems, Electronics, Control and Signal Processing, Tenerife, Canary Islands, Spain. Edgepreserved smoothing method with special reference to intravascular ultrasound image using anisotropic diffusion filter controlled by weighted separability measure (2008), pp. 87–92
 10.
J. Zhang, J. Yu, G. Mao, and X. Ye, Denoising of Chinese calligraphy tablet images based on runlength statistics and structure characteristic of character strokes, Journal of Zhejiang University SCIENCE A, vol.7, no.7, pp.1178–1186, July. 2006,https://doi.org/10.1631/jzus.2006.A
 11.
Z. Shi, B. Xu, X. Zheng, M. Zhao, An integrated method for ancient Chinese tablet images denoising based on the assemble of image smooth and noise reduction. Multimedia Tools & Applications 75(19), 12245–12261 (2016). https://doi.org/10.1007/s1104201634213
 12.
Z. Shi, B. Xu, X. Zheng, and M. Zhao, A KSVD based Chinese character structure preserved denoising method” Multimedia Tools & Applications, vol.76, no.13, pp.14921U˝ 14936, 2017,https://doi.org/10.1007/s1104201642843
 13.
L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 1U˝ 12 (2011). https://doi.org/10.1145/2024156.2024208
 14.
Z. Jin, G. Zhou, D. Gao, Y. Zhang, EEG classification using sparse Bayesian extreme learning machine for brain–computer interface. Neural Comput. & Applic. (2018). https://doi.org/10.1007/s0052101837353
 15.
Y. Zhang, C. Nam, G. Zhou, J. Jin, X. Wang, A. Cichocki, in IEEE Transactions on Cybernetics. Temporally constrained sparse group spatial patterns for motor imagery BCI (2018), pp. 1–11. https://doi.org/10.1109/TCYB.2018.2841847
 16.
K. He, J. Sun, X. Tang, in Computer Vision ECCV 2010. ECCV 2010, ed. by K. Daniilidis, P. Maragos, N. Paragios. Guided image filtering (Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2010), pp. 1–14
 17.
C.A. Metzler, A. Maleki, R.G. Baraniuk, in Proc. of 2015 IEEE International Conference on Image Processing, Quebec City, QC, Canada. BM3DAMP: a new image recovery algorithm based on BM3D denoising (2015), pp. 3116–3120
 18.
A. Hore, D. Ziou, in Proc.of 20th International Conference on Pattern Recognition, Istanbul, Turkey. Image quality metrics: PSNR vs. SSIM (2010), pp. 2366–2369
 19.
Z. Wang, A.C. Bovik, Universal image quality index. IEEE Signal Processing Letters 9(3), 81–84 (2002)
Acknowledgements
The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.
Funding
Not applicable.
Availability of data and materials
Please contact author for data requests.
Author information
Affiliations
Contributions
All authors take part in the discussion of the work described in this paper. All authors read and approved the final manuscript.
Corresponding authors
Correspondence to Feihang Ge or Lifeng He.
Ethics declarations
Authors’ information
Feihang Ge received the B.E. degree and the M.S. degree in computer science and technology from Xi’an University of Technology in 2002 and 2005, respectively. He is currently studying for his Ph. D Degree at Aichi Prefectural University, Aichi, Japan. His research interests include intelligent image processing and computer vision. Email: fhge2018@163.com
Lifeng He received the B.E. degree from Northwest Institute of Light Industry, Shaanxi, China, in 1982, the second B.E. degree from Xian Jiaotong University, Shaanxi, in 1986, and the M.S. and the Ph.D. degrees in artificial intelligence and computer science from Nagoya Institute of Technology, Aichi, Japan, in 1994 and 1997, respectively. He is currently a Professor at Aichi Prefectural University, Aichi, and a Guest Professor at Shaanxi University of Science and Technology, Shaanxi. From September 2006 to May 2007, he was a Research Associate at The University of Chicago. His research interests include intelligent image processing, computer vision, medical image processing. Email: helifeng@ist.aichipu.ac.jp
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 L0 gradient minimization
 Guided filter
 Connection region
 Denoising
 Stele image enhancement