Natural Enhancement of Color Image
© S. Chen and A. Beghdadi. 2010
Received: 1 April 2010
Accepted: 6 July 2010
Published: 22 August 2010
A new algorithm of Natural Enhancement of Color Image (NECI) is proposed. It is inspired by multiscale Retinex model. There are four steps to realize this enhancement. At first, the image appearance is rendered by content-dependent global mapping for light cast correction, and then a modified Retinex filter is applied to enhance the local contrast. Histogram rescaling is used afterwards for normalization purpose. At last, the texture details of image are enhanced by emphasizing the high-frequency components of image using multichannel decomposition of Cortex Transform. In the contrast enhancement step, luminance channel is firstly enhanced, and then a weighing map is calculated by collecting luminance enhancement information and applied to chrominance channel in color space CIELCh which enables a proportional enhancement of chrominance. It avoids the problem of unbalanced enhancement in classical RGB independent channel operation. In this work, it is believed that image enhancement should avoid dramatic modifications to image such as light condition changes, color temperature alteration, or additional artifacts introduced or amplified. Disregarding light conditions of the scene usually leads to unnaturally sharpened images or dramatic white balance changes. In the proposed method, the ambience of image (warm or cold color impression) is maintained after enhancement, and no additional light sources are added to the scene, and no halo effect and blocking effect are amplified due to overenhancement. It realizes a Natural Enhancement of Color Image. Different types of natural scene images have been tested and an encouraging performance is obtained for the proposed method.
Image enhancement is one of the most important issues in image processing which can produce more suitable results than its original version for further image analysis and understanding. Many different approaches have been proposed in literature which can be roughly grouped into two categories: spatial domain methods and frequency domain methods. A thorough and comprehensive tutorial can be found in publications . This paper, however, is mainly focusing on some recent methods based on Retinex theory  and its diversities that are applied to color image enhancement issue. Since the pioneering work of Land , wide applications of Retinex have been found in industrial and medical scenario and aerospace photography [4, 5]. Many algorithms have been proposed such as path version , iterative version [7, 8], and center/surround version .
However, image enhancement is not another application of detail retrieval or color constancy. These algorithms cannot be directly applied to this domain because most of them lead to several dramatic modifications such as light condition changes, color temperature alterations, and additional artifacts introduced or amplified. These techniques work efficiently indeed when serving as tools of extracting image details. However, disregarding light conditions of scene may result in unnaturally sharpened image appearance or dramatic white balance changes which are usually unwanted. Based on our previous work , we propose in this paper an algorithm of automatic Natural Enhancement of Color Image (NECI) method. There are four steps to realize this enhancement. At first, the image appearance is rendered by content-dependent global mapping for light cast correction, and then a modified Retinex filter is applied to enhance the local contrast. Histogram rescaling is used afterwards for normalization purpose. At last, the texture details of image are enhanced by emphasizing the high-frequency components of image using multi-channel decomposition of Cortex Transform . In the contrast enhancement step, luminance channel is firstly enhancement, and then a weighing map is calculated by collecting luminance enhancement information and applied to chrominance channel in color space CIELCh which enables a proportional enhancement of chrominance compared with the luminance. It avoids the problem of unbalanced enhancement in classical RGB independent channel operation.
In this work, it is believed that image enhancement should be different from detail retrieval or color constancy which often leads to several dramatic modifications to image such as light condition changes, color temperature alteration, or additional artifacts introduced or amplified. Disregarding light conditions of scene usually leads to unnaturally sharpened images or dramatic white balance changes which are usually unwanted. In the proposed method, the ambience of image (warm or cold color impression) is maintained after enhancement, and no additional light sources are added to the scene, and no halo effect and blocking effect are amplified due to over-enhancement which leads to a Natural Enhancement of Color Image.
The structure of this paper is organized into 6 sections including the current Section 1 of introduction. A problem statement will be given at first to show the insufficiency of natural enhancement of color image by some state-of-the-art methods. Section 3 describes the flowchart of the proposed NECI method with brief explanations and some typical results. Details of theoretical analysis and practical implementations will be presented in Section 4. Different types of natural scene images are shown in Section 5 followed by conclusion and perspective works in Section 6.
2. Problem Statement
3. Flowchart of NECI
In the first step of global mapping, gamma correction is often applied using logarithm curve. However, for many compression algorithms, the dark-zones of image are often heavily compressed by coding system and therefore more sensible to over-enhancement. Logarithm curve amplifies the small intensities of dark zone pixels which makes blocking effect or ringing effect more visible after enhancement. For low-intensity pixels, we designed a circular curve to replace gamma correction tone mapping in the proposed work which gives moderate gain in dark zone so that the hidden artifacts remain tolerable after enhancement. This step will be discussed in detail in Section 4.1.
Step two includes a luminance enhancement using modified Retinex and chrominance enhancement using enhancement map and histogram rescaling in color space CIELCh. Detailed discussion can be found in Section 4.2. The proposed NECI method is inspired by a computational model of multi-scale Retinex [12, 13], while an additional logarithm function is applied to mask image (estimation of background using Retinex filter) for local contrast calculation to avoid introducing halo effect or amplifying blocking or ringing effect of the compressed image. For chrominance enhancement, applying Retinex independently to three color channels (RGB) usually results in false colors and hue-shift. In the proposed work, only luminance channel is used for a local contrast calculation, and the enhancement information is used as reference map applying to chrominance channel in color space CIELCh so that a balanced enhancement of chromatic components can be achieved. A histogram rescaling operation is followed for black and white point correction at the end of contrast enhancement, and only 99% of histogram is used to remove the influence of a few pixels with extreme intensities. The chrominance enhancement and histogram rescaling will be also discussed in Section 4.2.
The enhanced image after histogram rescaling will be reconstructed from CIELCh to RGB space. The resulted image, although with contrast enhanced, may still need global mapping for a normal tone appearance. Therefore, another global mapping step using the same principle as step one is applied in step 3 as posttreatment to ensure the global appearance of the enhanced image.
Finally, the texture information will be enhanced using multi-channel decomposition of Cortex Transform. The advantage of Cortex Transform over other approaches (such as sharpening the contour with Laplacian filter) is that the multi-channel decomposition can better capture the texture information in several different spatial frequency bands whereas the Laplacian-like approach usually captures one subband frequency which usually cannot achieve gradually sharpened contour. The texture enhancement using Cortex Transform will be discussed in Section 4.3.
4. Analysis and Implementation Details
In this section, all of the four steps of NECI will be discussed in details with intermediate results to show the improvement of image contrast by each step.
4.1. Global Tone Mapping Using Modified Gamma Correction
4.1.1. Image Key Value and Adaptive Global Mapping
4.1.2. Adaptive Global Mapping for Low Intensities
where is original image, is the globally mapped image, and ( ) is the coordinate of the mapping circle center. It can be seen from (3) and Figure 6 that if image key value is smaller than 50, the arc of circle will cave in upwards compared to straight line (which is the no-mapping case) to amplify tone values of dark image. However, if the key value is larger than 60, the arc will cave in downwards compared to straight line to compress the brightness of image. For image with key value between 50 and 60, no global mapping is needed since the original picture has a normal dominant tone.
4.1.3. Adaptive Global Mapping for High Intensities of Underexposed Scene
For overexposed image, traditional luminance component of color space usually loses some information due the almost saturated brightness for this type of images. However, if the Principle Component Analysis (PCA) is applied to color image, then its primary vector contains usually more information than that of classical luminance channel since this PCA vector concentrates most of image information from all of three channels. By this means, the details in bright zone of overexposed image can be better extracted with PCA first vector than traditional luminance representation for further contrast enhancement procedures.
4.2. Luminance and Chrominance Enhancement
This step performs luminance and chrominance enhancement in two stages. First of all, the original RGB image is transferred to color space CIELCh to separate luminance, chrominance and hue components. The contrast information in luminance channel is enhanced using a modified multi-scale Retinex. The enhancement information is used in an enhancement map to weight the chromatic component. By this means, the luminance and chrominance will be enhanced proportionally to avoid unbalanced enhancement such as RGB independent channel operation.
Secondly, histogram rescaling is applied to both enhanced luminance and chrominance channel to realize normalization of the results. This statistical rescaling for black and white point correction removes the influences of some extreme intensity of pixels. The hue component in CIELCh remains unchanged to maintain the hue constancy property, and the image is reconstructed with enhanced luminance and chrominance together with the unchanged hue component from CIELCh space to RGB space.
4.2.1. Luminance Enhancement Using Modified One-Filter Retinex
This step is inspired by a computational model of multi-scale Retinex . An additional logarithm function is applied to image mask before taking ratio between the mapped luminance and the mask image to avoid halo effect. The reason for the application of logarithm will be demonstrated in a later part of this section. The principle of one-filter Retinex is at first recalled below for a self-complete introduction.
From (7) and (8), we can see that the number of Gaussian and its variance is proportional to image size, and the resulted filter has a pointed shape and a large base. If we make convolution using this filter with the original image as referred in (6), pixels not only from immediate neighbors but also from distance will make contribution to the background calculation, and their contributions are weighted by their distances from the center using Gaussian curves. This background compared with local background such as a block corresponds better to our perception of contrast.
where is the image luminance mask (estimation of local background), refers to then enhanced luminance (Retinex output), and refers to globally mapped luminance of step 1. The small positive constant ε is added to avoid problem of logarithm of zeros and division of zero.
4.2.2. Chrominance Enhancement Using Reference Map
4.2.3. Histogram Rescaling for White Point Correction
Concerning the hue component, since generally a hue-constant enhancement is preferred for coherent rendering of original image, the hue component after global mapping remains unchanged during enhancement. After reconstruction of the enhanced image from CIELCh to RGB space, posttreatment using similar global mapping as step 1 is applied to ensure the global appearance of the enhanced image.
4.3. Texture Enhancement Using Multi-Channel Decomposition of Cortex Transform
4.4. Implementation Details of NECI
Pretreatment using modified gamut mapping which is adaptive to image dominating tone;
- (ii)Image enhancement including.
luminance enhancement using modified one-filter Retinex,
chrominance enhancement using reference map,
histogram rescaling for enhanced luminance and chrominance,
hue component remains unchanged,
posttreatment using the same principle of global mapping.
Texture information enhancement using multi-scale decomposition of Cortex Transform.
5. Test Results
Different types of natural images have been tested, and the results obtained confirm an encouraging performance of the proposed method. In this section, some test results are shown below. They are grouped into four categories: low-key images, normal-key images, high-key images, and HDR images.
6. Conclusions and Perspective Works
An automatic method for Natural Enhancement of Color Image (NECI) is proposed in this paper in order to improve the luminance and chrominance contrast of image while avoiding dramatic white balance changes and artifacts. The proposed method applies four steps of image processing including pretreatment by adaptive global tone mapping using circular curve combined with gamma correction, luminance and chrominance contrast enhancement using modified multi-scale Retinex and histogram rescaling in color space CIELCH, another global mapping step as posttreatment, and finally texture information enhancement using multi-channel decomposition of Cortex Transform.
The global mapping algorithm adapts to different image dominant tone, and it is capable of enlightening dark scene or dimming bright scene back to normal-key appearance of images. The modified computational model of multi-scale Retinex increases local luminance contrast with artifact under control. Enhancing chrominance using a reference map calculated from luminance enhancement information generates more balanced enhancement compared to independent channel operation in RGB or LAB color spaces, and histogram rescaling renders image with more natural appearance by eliminating extremely dark or bright points caused by over-enhancement. Last but not least, the details of images which correspond to high-frequency components of images are enhanced by multi-channel decomposition of Cortex Transform.
Albeit with some empirical parameters, the proposed NECI method needs no parameter modification in practice, and its adaptability and robustness are proved by extensive tests and some of them are shown in a previous section.
The time of calculate is not sufficient to be a real-time application due to the complexity of Cortex Transform and contrast calculation. For some unnatural scene images such as medical images, parameters still need to be retrained for a better performance. As far as the HDRI is concerned, the Gaussian Markov segmentation will be integrated to NECI in future works to make different enhancements according to different regions. However, fusion of independently enhanced results will be another open issue.
- Gonzalez RC, Woods RE, Eddins SL: Digital Image Processing. Prentice Hall, Upper Saddle River, NJ, USA; 2004.Google Scholar
- McCann JJ: Capturing a black cat in shade: past and present of Retinex color appearance models. Journal of Electronic Imaging 2004,13(1):36-47. 10.1117/1.1635831View ArticleGoogle Scholar
- Land EH: Recent advances in retinex theory. Vision Research 1986,26(1):7-21. 10.1016/0042-6989(86)90067-2View ArticleGoogle Scholar
- Sobol R: Improving the retinex algorithm for rendering wide dynamic range photographs. Proceedings of the IS&T/SPIE Electronic Imaging Conference on Human Vision and Electronic Imaging (VII '02), 2002, San Jose, Calif, USA 4662: 341-348.Google Scholar
- Rizzi A, Marini D, Rovati L, Docchio F: Unsupervised corrections of unknown chromatic dominants using a brownian-path-based retinex algorithm. Journal of Electronic Imaging 2003,12(3):431-440. 10.1117/1.1584051View ArticleGoogle Scholar
- Funt B, Ciurea F, McCann J: Retinex in matlab. Proceedings of the 18th IS&T/SID Color Imaging Conference on Color Science, Systems and Applications, 2000 112-121.Google Scholar
- McCann J: Lessons learned from mondrians applied to real images and color gamuts. Proceedings of the 7th IS&T/SID Color Imaging Conference on Color Science, Systems, and Applications, 1999, Scottsdale, Ariz, USA 1-8.Google Scholar
- Rahman Z-U, Jobson DJ, Woodell GA: Retinex processing for automatic image enhancement. Journal of Electronic Imaging 2004,13(1):100-110. 10.1117/1.1636183View ArticleGoogle Scholar
- Chen S, Beghdadi A: Natural rendering of color image based on retinex. Proceedings of the IEEE International Conference on Image Processing (ICIP '09), November 2009, Cairo EgyptGoogle Scholar
- Daly S: The visible differences predictor: an algorithm for the assessment of image fidelity. In Digital Images and Human Vision. Edited by: Watson AB. MIT Press, Cambridge, Mass, USA; 1993:179-206.Google Scholar
- Meylan L: Tone mapping for high dynamic range images, Ph.D. thesis. EPFL, Lausanne, Switzerland; July 2006.Google Scholar
- Meylan l, Süsstrunki S: Bio-inspired color image enhancement. Human Vision and Electronic Imaging Conference, 2004, San Jose, Calif, USA, Proceedings of SPIE 5292: 46-56.Google Scholar
- Ciurea F, Funt B: Tuning Retinex parameters. Journal of Electronic Imaging 2004,13(1):58-64. 10.1117/1.1635365View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.