Open Access

Using Adaptive Tone Mapping to Enhance Edge-Preserving Color Image Automatically

  • Kuo-Jui Hu1,
  • Min-Yao Lu2,
  • Je-Chuang Wang1,
  • Tao-I Hsu3Email author and
  • Ting-Ting Chang2
EURASIP Journal on Image and Video Processing20102010:137134

DOI: 10.1155/2010/137134

Received: 14 April 2010

Accepted: 6 December 2010

Published: 14 December 2010

Abstract

One common characteristic of most high-contrast images is the coexistence of dark shadows and bright light source in one scene. It is very difficult to present details in both dark and bright areas simultaneously on most display devices. In order to resolve this problem, a new method utilizing bilateral filter combined with adaptive tone-mapping method is proposed to improve image quality. First of all, bilateral filter is used to decompose image into two layers: large-scale layer and detail layer. Then, the large-scale layer image is divided into three regions: bright, mid-tone, and dark region. Finally, an appropriate tone-mapping method is chosen to process each region according to its individual property. Only large-scale layer image is enhanced by using adaptive tone mapping; therefore, the details of the original image can be preserved. The experiment results demonstrate the success of proposed method. Furthermore, the proposed method can also avoid posterization produced by methods using histogram equalization.

1. Introduction

Human eyes can capture a very wide dynamic range of 10-order of magnitude through brightness adaptation. In other words, due to the existence of rods and cones in our retina, human visual system can accommodate the large variation by changing between scotopic vision and photopic vision. However, typical CMOS or CCD sensors can only capture a range of thousands in magnitude; typical displays in normal viewing conditions and printing media can only reproduce information over a range of just a few hundreds or less. As a result, even though humans can recognize the details clearly in both dark and bright regions in a scene, the image captured by digital camera may be either too dark or too bright to present details. This is due to the limited dynamic range of the digital camera. Hence, some image-processing techniques must be utilized to enhance these images.

At present, most application software provides straightforward tools for user to enhance images, like gamma correction or histogram equalization. Histogram equalization is an automatic technique without user setting any parameters. Using of gamma correction to enhance images, dark regions in the images can become clearer, but the bright regions will become overexposed. Applying of histogram equalization to enhance the image, the image is likely to be posterized. These results are shown as Figures 1(a)1(c).
Figure 1

Comparison of different image enhancement methods. (a) Original image; (b) Result of gamma correction (γ = 0.45); (c) Result of histogram equalization; (d) Result of algorithm provided by Wu [1].

Recently, high dynamic range (HDR) image processing is gaining its popularity. HDR tone-mapping techniques have been developed to compress the dynamic range of the image for displaying on prevalent monitors, which the dynamic range is often smaller. The tone-mapping operators for HDR image can be roughly classified into four categories: global operators [2, 3], local operators [4], frequency domain operators [5], and gradient domain operators [6]. These methods all require at least two images with different exposure levels to produce one HDR image. In some cases, it is not possible to obtain two different exposed images.

Lately, an automatic image enhancement algorithm for low dynamic range images is introduced by Wu [1]. The algorithm is able to lighten dark regions while retaining the details of bright regions. The principle behind the algorithm is that an image can be decomposed into a large-scale layer and a detail layer by using bilateral filtering. By applying histogram equalization only to the large-scale layer, details of the images can be preserved. However, due to the nature of histogram equalization, the algorithm still suffers from the phenomena of posterization. The effect is shown in Figure 1(d).

Cope with both work in [7] and method introduced by Wu [1] inspire the proposed one. The same bilateral filtering method is used in that work to preserve image details. The improvement in the proposed work is that the large-scale layer image is first divided into three regions according to its histogram, and then according to the individual visual property of each region, appropriate tone-mapping method is chosen to enhance individual region. The experiment results show the proposed method is effective and can avoid posterization.

2. Proposed Method

The flow chart of our algorithm is shown in Figure 2. Each stage in the flow is explained in the following sections.
Figure 2

Flow chart of the proposed algorithm.

2.1. Bilateral Filter

Bilateral filtering is a nonlinear filtering technique introduced by Tomasi and Manduchi [8]. In essence, bilateral filter is a Gaussian low-pass filter combined with an influence weight, which is the reciprocal of the intensity difference between the processed pixel and the neighborhood pixel. The weight decreases while the difference in intensity increases, so it can smooth image while preserving edges. The result after applying bilateral filter is shown in Figure 3.
(1)
In fact, only large-scale layer image is enhanced, and the detail layer image is kept unchanged. Adaptive tone mapping is then applied to enhance large-scale layer image. Thus, an image with clear edges and textures can be obtained. The image enhancement technique, adaptive tone mapping, is introduced to improve the perception quality in the next section.
Figure 3

The effect of bilateral filter (σd = 10, σr = 50).). (a) Original image; (b) Large-scale layer; (c) Detail layer. In order to avoid damaging color information of the image, only intensity image is processed by bilateral filter. Therefore, the color information must be separated from the original image. In this work, RGB color space is translated to color space by using (1), and the color-opposite data and is kept unchanged.

2.2. Adaptive Tone Mapping

As mentioned before, the histogram of large-scale layer image must be divided into three regions in advanced, the dark, the mid-tone, and the bright region, respectively. A fast, simple, and global thresholding technique, Otsu's method [9] is chosen to serve the purpose. Otsu's thresholding method is based on a very simple idea that a threshold is decided when the weighted within-class variance is minimized. This turns out to be the same as maximizing the between-class variance. Since it operates directly on the gray level histogram, so it is a fast and nonparametric method. The concept of Otsu's thresholding method is represented in Figure 4.
Figure 4

The diagram of Otsu's thresholding technique.

The large-scale layer can be divided into two classes, the class 1 and class 2, by applying Otsu's thresholding method.

The variance between class1 and class 2 from gray level 0 to 255 is computed recursively, so a gray level value, , which maximized the between-class variance can be obtained. The obtained regions class1 and class 2 can be divided further by applying Otsu's thresholding method again, respectively, resulting two threshold and can be obtained. In class 1, is found by computing the maximum between-class variance from gray level 0– , and is found by the same method but the range of gray level is from to 255. Hence, three segments, dark, mid-tone, and bright region, can be obtained. The result is shown in Figure 5.
Figure 5

The result by using Otsu's thresholding method to divide histogram. (a) Original image; (b) Large-scale layer; (c) Histogram of (b) and two thresholds, , ; (d) The result of tri-leveling image.

The gray levels from 0 to are defined as dark region, levels from to are defined as mid-tone region, and levels from to 255 are defined as bright region. Furthermore, , , and are defined as sets of pixel values in dark, mid-tone, and bright regions, respectively. Then, each region is processed by different tone-mapping methods according its individual property. In general, details existing in the dark shadow are hard to detect, so that dark regions should be lighten to reveal the details. On the contrary, bright regions do not need to be brighten because the details in the regions are clear enough. If the bright regions are continuously to be brighten, those areas may be overexposed. But, it is desired to increase contrast in the bright regions in order to stretch out dynamic range. An adaptive quartile sigmoid curve algorithm introduced by Chao-hua Wen et al. [10] is employed to achieve the goal. Therefore, (2) is applied to enhance dark and bright regions
(2)
where is a mapped region with gamma correction in , and is a mapped region from implemented with an adaptive quartile sigmoid curve . The in (2) is the gamma value and can be expressed as (3)
(3)

The mapping result of both and carry out the level increasing of the dark region near mid-tone region.

After mapping sets and to new sets and , two new bounds and will be obtained. The result is shown in Figure 6.
Figure 6

The diagram illustrates that I 1 and I 3 are mapped to new sets and .

In Figure 6, the solid curve is the mid-tone level distribution generated from Otsu's method recursively, while dotted curve is the mapped regions of both dark region and bright region using (2).

Although it is expected to map into region from and , when the region is too small, the rendering image of mid-tone region transacts sharply; hence, a discontinued tone will appear in the image, which should be avoided. In order to overcome the sharp transaction within the mid-tone region, a set of sophisticated thresholding values are derived based on the , define as . So, (4) is used to map a set to a new set while enhancing mid-tone to smooth out the discontinued tone from dark to bright region
(4)
where , , and , but then and must be defined. In fact, and are now the new bounds for mapping to . When , two situations are considered: (1) if then and , (2) if , then both and are defined in (5). The threshold value of is derived from the experiment result; if , then the above-mentioned discontinued tone will become significant. Hence, the value of and should be adjusted so that the condition will be satisfied
(5)
In the case of , the mapped dark region and bright region are overlapped, and two additional situations must be also considered: ((1) if , and , ((2) if , and , but and are still needed to be adjusted by.
(6)

Note that when the two new bounds and are defined, must be shifted from to , and must be shifted from to . This shifting is for gradual change in gray level.

3. Results and Discussions

The original high-contrast images shown in Figures 7(a), 7(b), and 7(c) are images processed separately by histogram equalization and algorithm used in [1]. Although details of dark region are revealed, it is obvious to observe overexposure and posterization in the bright region where mostly belong to the sky. In Figure 7(d), the dark region is effectively being enhanced while still retaining the details of bright region and posterization is avoided.
Figure 7

Comparison of our algorithm with other methods. (a) Original image; (b) Result of histogram equalization; (c) Result of the algorithm used in [1]; (d) Result of proposed algorithm.

Figure 8(a) is a high-contrast original image, and Figures 8(b) and 8(c) are results of histogram equalization and algorithm used in [1]. In Figure 8(b), most of the image is overexposed, and in Figure 8(c), the clouds are also overexposed so a few details disappeared. Posterization also occurs within the image like the clothes on the front people. Shown in Figure 8(d), the proposed method can not only brighten dark region, but also keep the details of the clouds.
Figure 8

Comparison of our algorithm with other methods. (a) Original image; (b) Result of histogram equalization; (c) Result of the algorithm used in [1]; (d) Result of proposed algorithm.

Figure 9(a) is an original underexposed image, Figures 9(b), 9(c), and 9(d) are the same results as mentioned above. Although most parts of Figure 9(b) are brighter than Figure 9(d), Figure 9(d) is closer to human visual system. In fact, only few parts in Figure 9(b) still overexposed. And in Figure 9(c), it is obviously to detect posterization and overexposure at the same time.
Figure 9

Comparison of the proposed algorithm with other methods. (a) Original image; (b) Result of histogram equalization; (c) Result of the algorithm used in [1]; (d) Result of proposed algorithm.

More results are shown in Figures 10, 11, and 12. From these comparisons of the results with different image enhancement methods, the proposed algorithm provides an effective improvement for underexposed images or high-contrast images. In addition, the proposed algorithm can enhance images automatically without setting complicated parameters; in other words, the proposed method is friendly to use for all user.
Figure 10

Comparison of our algorithm with other methods, and original image is shown as Figure 5(a). (a) Result of histogram equalization; (b) Result of the algorithm used in [1]; (c) Result of proposed algorithm.

Figure 11

Comparison of our algorithm with other methods. (a) Original image; (b) Result of histogram equalization; (c) Result of the algorithm used in [1]; (d) Result of proposed algorithm.

Figure 12

Comparison of our algorithm with other methods. (a) Original image; (b) Result of histogram equalization; (c) Result of the algorithm used in [1]; (d) Result of proposed algorithm.

In order to prove our tone-mapping method can effectively enlarge the gamut volume, the gamut volumes of Figures 7, 8, 9, 10, 11, and 12 are calculated and shown in Figures 13(a), 13(b), 13(c), 13(d), 13(e), and 13(f). In Figure 13, the white frame represents the gamut of the original image, the blue frame represents the gamut of the processed image by the method used in [1], and the red frame represents the gamut of the processed image by the proposed method. In short, white, blue, and red frames are separately represented as (a), (c), and (d) of Figures 712. It is noticeable that all red frames are bigger than white frames. In fact, even if the blue frames are bigger than the red frames, some drawbacks, such as posterization, unnatural color, and overexposure often lead to an unacceptable result. The values of the gamut volumes are listed in Table 1.
Table 1

The gamut volumes in (a), (c), and (d) of Figures 712.

Figure

(a)

(c)

(d)

7

85452

61373

117205

8

177707

562574

457091

9

500697

717327

598404

10

452703

565082

688151

11

738114

899999

870750

12

811021

716025

772591

Figure 13

Comparisons of the gamut volume in (a), (c), and (d) of Figures 7–12.

In order to illustrate the less color shifting in the proposed method, CIE94 color difference formula is used to evaluate the color difference between the original image and the mapped image. Both the color difference are calculated between colors in blue frame versus that in white frame and color in red frame versus that in white frame. In some cases that the gamut volumes in blue frame is larger than that in red frame, the rendered color difference of between (blue frame versus white frame) is worse than that in the (red frame versus white frame). In Table 2 the top value in the color bar, the red, represents as the biggest color difference, while the smallest color difference is represented as blue in the bottom of color bar. From the color difference in Table 2, the proposed method yields less color shift than the method used in [1]. This result demonstrates efficacy of our proposed method performance.
Table 2

Comparisons of the color differences in CIE94.

Figure

(a) versus (c) in CIE94

(a) versus (d) in CIE94

7

8

9

10

11

12

4. Conclusion and Future Work

An algorithm based on bilateral filter associated with adaptive tone mapping to effectively enhance images is proposed. From the experimental results demonstrates that the proposed algorithm outperforms and resolves posterization drawbacks as occurred in [1].

By easing the bilateral filter, the image is decomposed into two layers. The detail layer remains unprocessed so that the details can be retained within the adaptive tone mapped image. The color information is kept unchanged before the large-scale layer is obtained from bilateral filtering allowing the color appearance of input image can be remained. The adaptive tone mapping manipulates different regions according to properties of the corresponding regions, so that global smoothness can be accomplished without losing local details, and global dynamic range can be stretched with achieving promising contrast enhancement. Finally, the gamut volume and color difference using CIE94 color difference equation are calculated to demonstrate the proposed method is capable of producing less color shift while obtaining larger gamut volume than original image. Additionally, the enhancement of color image is effectively performed without requirement of manually parameters setting during the whole process.

Currently, the proposed algorithm is able to cope well with underexposed image or high-contrast image. Some problems still occurred when processing overexposed image, and these issues are under investigating.

Declarations

Acknowledgment

The work is supported by MOEA Project no. 8351AA1100 implemented by DTC/ITRI.

Authors’ Affiliations

(1)
Chung-Cheng Institute of Technology, National Defense University
(2)
Display Technology Center, Industrial Technology Research Institute
(3)
Department of Digital Multimedia Arts, Shih Hsin University

References

  1. Wu J-L: An automatic enhancement method for high contrast images via bilateral filtering. Journal of Science and Engineering Technology 2007,3(3):81-87.Google Scholar
  2. Reinhard E, Stark M, Shirley P, Ferwerda J: Photographic tone reproduction for digital images. ACM Transactions on Graphics 2002,21(3):267-276.View ArticleGoogle Scholar
  3. Duan J, Qiu G: Fast tone mapping for high dynamic range images. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004 2: 847-850.View ArticleGoogle Scholar
  4. Pizer SM, Amburn EP, Austin JD, Cromartie R, Geselowitz A, Greer T, ter Haar Romeny B, Zimmerman JB, Zuiderveld K: Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing 1987,39(3):355-368. 10.1016/S0734-189X(87)80186-XView ArticleGoogle Scholar
  5. Durand F, Dorsey J: Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics 2002,21(3):257-266.View ArticleGoogle Scholar
  6. Fattal R, Lischinski D, Werman M: Gradient domain high dynamic range compression. ACM Transactions on Graphics 2002,21(3):249-256.View ArticleGoogle Scholar
  7. Hu KJ, Chang TT, Lu MY, Li WUJ, Huang JF: Bilateral filtering and adaptive tone-mapping for qualified edge and image enhancement. Color Imaging XIV: Displaying, Processing, Hardcopy, and Applications, January 2009, Proceedings of SPIEGoogle Scholar
  8. Tomasi C, Manduchi R: Bilateral filtering for gray and color images. Proceedings of the IEEE 6th International Conference on Computer Vision, January 1998 839-846.Google Scholar
  9. Otsu N, Rosenfeld A: A threshold selection method from gray-level histogram. IEEE Transactions on Systems, Man and Cybernetics 1979,SMC-9(1):62-66.Google Scholar
  10. Wen CH, Lee JJ, Liao YIC: Adaptive quartile sigmoid function operator for color image contrast enhancement. Proceedings of the IS&T/SID 9th Color Imaging Conference, November 2001 280-285.Google Scholar

Copyright

© Kuo-Jui Hu et al. 2010

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.