Prediction error preprocessing for perceptual color image compression
© Liu; licensee Springer. 2012
Received: 13 July 2011
Accepted: 16 March 2012
Published: 16 March 2012
In this article, a prediction error preprocessor based on the just noticeable distortion (JND) for the color image compression scheme is presented. The dynamic range of prediction error signals we can reduce, the lower bit rate of the reconstructed image we can obtain at high visual quality. We propose a color JND estimator that is incorporated into the design of the preprocessor in the compression scheme. The color JND estimator is carried out in the wavelet domain to present good estimates to the available amount masking. The estimated JND is used to preprocess the signal and is also used to incorporate into the design of the quantization stage in the compression scheme for higher performance. Simulation results show that the bit rate required by the compression scheme with the preprocessor is lower at high visual quality of the reconstructed color image. The preprocessor is further applied to the input color image of the JPEG and JPEG2000 coders for better performance.
Keywordsprediction error preprocessor just noticeable distortion visual masking
In the Internet, where the transmission bandwidth is limited, the growing demand for representing high-quality color images is expected. Since human eyes are ultimate receivers of visual in-formation, color image compression that is perceptually lossless to human visual perception is required. The color image compression scheme should take into account the properties of the human visual system (HVS) when considering the image quality as the critical performance to be achieved. The goal of the perceptual image compression is to represent a digital image at the lowest possible bit rate without intro-ducing perceivable distortion. To reach this goal, the perceptual coder has to remove not only statistical redundancy, but also perceptual redundancy of images. Accurately measuring the perceptual redundancy is important to the success of perceptual coding. Perceptual redundancy can quantitatively be measured as error detection thresholds or noise amplitudes of just noticeable distortion (JND) , by which signals can be neither undercoded nor overcoded.
In the research efforts of perceptual image coding, the determination of proper quan-tization steps with JND has been so far focused. Once the JND profile of the image is accurately measured, the quantization step size can appropriately be determined such that the coding distortion can properly be distributed and shaped with less objective distortion. By combining band sensitivities, background luminance, and texture masking, Safranek and Johnston  measured the JND threshold for each coefficient in a given subband to set the quantization level in a differential pulse code modulation (DPCM) quantizer. In , quantization matrices for the use in DCT-based compression were designed by exploiting visibility thresholds that are experimentally measured for quantization errors of the DCT coefficients. In , a JPEG compliant encoder utilizing perceptually based quantization was proposed to produce a perceptually equivalent image that has a high compression ratio. Chou and Li  estimated the JND threshold by the dominant between the luminance masking and the texture masking. The estimated JND profile is incorporated to tune the step size of a uniform quantizer in the proposed subband image coder. In , a model of the HVS based on the wavelet transform was proposed. The model has a number of modifications that make it more amenable to potential integration into a wavelet-based image compression scheme. The author concludes with suggestions on how the model can be used to determine a visually optimal quantization strategy for wavelet coefficients and produce a quantitative measure of image quality. In , the masking thresholds derived in a locally adaptive fashion based on subband decomposition are applied to the design of a locally adaptive perceptual quantization scheme for achieving high performance in terms of quality and bit rate. Tang  further investigated perceptual video coding by incorporating the motion attention model, visual sensitivity model, and visual masking model for the purpose of adaptive quantization. In , the sensitivity of the HVS to edges is considered to construct a classified vector quantization method for image compression. Nevertheless, these research efforts focused on developing the coding schemes for grayscale images. The perceptual compression schemes that are designed for color images can be found in [10–13]. Yang et al.  proposed a nonlinear additive model to estimate the spatial JND profiles for color image processing. In , the wavelet-based color image compression by exploiting the contrast sensitivity function (CSF) was presented. The method implements the CSF measure over spatial frequency of luminance and chrominance components into the task of noise spectrum shaping and achieves a visually optimal compression quality. Based on the uniformity of the uniform color space, Liu and Chou  built a color visual model that can estimate the perceptual redundancy for each color pixel as a visibility threshold of color difference to design the quantization strategy of the locally adaptive perceptual compression scheme for color images. In , the same visual model proposed in  is modified and incorporated into the JPEG-LS and JPEG2000 coder to improve the performance in both cases.
Based on the JNDs of images, most research efforts of perceptual coding have been concentrated on the design of proper quantizers. They attempt to discriminate between signal components which are and are not detected by human eyes . The main idea in perceptual coding is to hide the quantization error below the detection threshold. Meanwhile, perceptually irrelevant signal information is also removed to improve the standard coding paradigm of redundancy removal. Besides using JND thresholds to adapt quantization step sizes for image coding, the JND thresholds can also effectively be applied to certain stages in the image coding. In this article, a prediction error preprocessor based on the JND is investigated for higher performance in the design of the color image compression scheme. The proposed method is investigated under the guidance of visual tolerance such that the dynamic range of the prediction error signals is reduced to obtain lower coding bit rates without decreasing the visual quality of the reconstructed color image. That is, the prediction error preprocessor will be adapted by the JNDs of the color image to achieve this. Since the measure of JND profiles of the color image dominates this study, the JND estimator for color images will be designed. In this article, the wavelet-domain JND of each coefficient in luminance and chrominance components of color images are estimated in a locally adaptive fashion based on the wavelet decomposition. For the coefficient in luminance component, its visual tolerance is measured by using the visual masking effects given coefficient by coefficient by taking into account the luminance content and the texture of grayscale images. On the other hand, for the coefficient in chrominance components, its visual tolerance is measured by combining not only the visual masking effects, but also the effect given by the variance within the local region of the target coefficient while considering that the HVS is insensitive to chrominance than to luminance. The preprocessor is then designed by adjusting an appropriate quantity regulated by the JND profiles to shape the prediction error signals such that the perceptual distortion of the reconstructed color image can be reduced. Furthermore, for any standard color image coding scheme, the proposed preprocessor that is independent of image cod-ers can be also used to preprocess the input color image such that the processed signal can be coded with higher performance. The rest of this article is organized as follows. In Section 2, the estimation of subband JND profiles for color images is described. The proposed prediction error preprocessor based on the estimated subband JND profiles for color image compression is presented in Section 3. The simulation results on overall performance of the coding scheme are given in Section 4. In Section 5, the conclusions of this article are made.
2. Subband JND profiles for color images
In the application of perceptual color image compression where high visual quality of the reconstructed color image at lower bit rates would be required, the appropriate choice of a color space is important to determine the coding performance. The reduction of the correlation between color channels in the color space is expected for most image compression schemes. Since the redundancy among color channels of the color image in the YUV and YCbCr color spaces is less than that in the RGB color space, most of the compression techniques use the former color spaces for coding images. For example, the Y channel in the YCbCr color space contains almost all the luminance information while the Cb and Cr channels that may easily be down sampled have less information. The subband JND profiles for color images in the YCbCr color space are thus estimated to build the prediction error preprocessor in this article. To obtain better visual quality of the reconstructed color image, the JND of each wavelet subband coefficient in luminance and chrominance components is estimated to preprocess each coefficient signal.
2.1. Luminance-adapted base detection threshold, d O, D (λ, θ, i, j)
Base detection threshold for each subband of four-level 9/7 DWT 
Herein, the model designed for gray image is also applied to chrominance components since the human visual perception is more sensitive to luminance component than to chrominance compo-nents.
2.2. Visual Masking Adjustment, a O (λ, θ, i, j)
where dCb(λ, θ, i, j) and dCr(λ, θ, i, j) denote visibility thresholds of the coefficients at location (i, j) of the (λ, θ) subband in Cb and Cr components, respectively, and η and κ the parameters that are allowed to vary with frequency and perceptual color channel . The larger the values of η and κ are set, the greater the crossed masking effect can be given. When η and κ are set by the values of 0, no crossed masking occurs and the crossed masking adjustment is constant at 1. Through experiments, η = 1.0 and κ = 1.0 for all bands are determined in this article.
where is the local variance measured in O component. The local region that is used to calculate the local variance contains the coefficients in the same subband that lies within a window centered at the location of (i, j).
3. Perceptual color image compression scheme
In this article, a prediction error preprocessor built by utilizing the estimated JNDs of one achromatic and two chromatic components is proposed to integrate into a perceptual color image compression scheme using the DPCM technique. The JNDs mainly attempt to design a prediction error preprocessor that can shape the prediction error signals more smooth instead of investigating the adaptive prediction while the same visual quality of the reconstructed color image for lower compression bit rates is achieved. Then, it aims at varying the quantization level to constrain the quantization error under the visual tolerance for higher quality of the reconstructed image.
where φ O is the step size multiplier whose value can be chosen such that the compression distortion is uniformly distributed over the reconstructed image while a tight entropy (bit-rate) budget is required. In this article, φY = 1.0, φCb = 1.0, and φCr = 1.0 are used to achieve the perceptually lossless visual quality of the reconstructed image for the variable uniform mid-riser quantizer in the proposed compression scheme.
4. Simulation results
To evaluate the performance of the proposed compression scheme, the scheme has been implemented by incorporating the proposed prediction error preprocessor into the DPCM coder for compressing color images. A variety of color images that represent a great of diversity of visual information is used in the experiments. The size of each color image is 512 × 512 with color depth of 24 bits in the RGB color space. Since the compression performance achieved by the proposed prediction error preprocessor is emphasized, the stage of entropy coding is not further discussed in this article. In the simulation, the proposed compression scheme therefore makes use of entropies rather than bit rates to represent its performance while a specified visual quality of the reconstructed color image is obtained. Meanwhile, the subjective viewing test for assessing the visual quality of the compressed color image is conducted in the simulation.
Subjective rating criterion for visual quality of an image pair
The right one is much better than the left one
The right one is better than the left one
The right one is slightly better than the left one
The right one has the same quality as the left one
The right one is slightly worse than the left one
The right one is worse than the left one
The right one is much worse than the left one
Entropies of the proposed compression scheme with preprocessor, without prepro-cessor and the Watson's compression method at nearly the same visual quality of the reconstructed color images
Preprocessed prediction error
G 1 (%)
G 2 (%)
( γ O = 0, φ O = 1.0)
( γ O = 0.4, φ O = 0.5)
The proposed compression scheme is also compared with the compression method proposed by Watson et al.  to show the performance of compressing color images. The Watson's method uses a perceptual quantization matrix to compress color images, in which the quantization step size of each subband of the color image is determined by the base JND  within the subband. In order to make a fair comparison, the same observers took part in the subjective viewing tests. Table 3 lists the entropies of the Watson's compression method while its reconstructed color image is nearly no loss in perceptual quality at a viewing distance equals to six times the image height. The entropy gains, G1 and G2, are, respectively, evaluated while comparing the proposed schemes with the above two conditions with the Watson's method. From the G2 shown in the table, it is obvious that the proposed compression scheme achieves better performance when the prediction error preprocessor is applied. Furthermore, the G1 of the proposed compression scheme only using the perceptual quantization also shows better results, since the JNDs of the wavelet coefficients are effectively estimated to calculate the quantization step sizes for each coefficient as shown in Equation (12).
A prediction error preprocessor is presented with the goal to reduce the dynamic range of the prediction error signals of the color image to be compressed. The lower bit rate of the reconstructed image can be obtained by using the preprocessor while reaching high visual quality. For this purpose, a color JND estimator that takes into account various masking effects of human visual perception is proposed and incorporated into the preprocessor for the design of the perceptual color image compression scheme using the DPCM technique. The proposed compression scheme with the preprocessor generates consistent quality images at a lower entropy when comparing with that without prediction error preprocessor and the existing compression method. At the same quality factor of the coding standard, the preprocessor is also applied to the input color image of the JPEG and JPEG2000 coders to provide compatible bitstream and to achieve higher performance in terms of bit rate.
The study was supported by the National Science Council, Taiwan, under contract NSC99-2221-E-278-001 and the Image & Video Processing Laboratory, National Dong Hwa University, Taiwan.
- Jayant N, Johnston J, Safranek R: Signal compression based on models of human perception. Proc IEEE 1993, 81(10):1385-1422. 10.1109/5.241504View ArticleGoogle Scholar
- Safranek RJ, Johnston JD: A perceptually tuned subband image coder with image dependent quantization and post-quantization data compression. In Proc IEEE Int Conf Acoust, Speech, Signal Processing. Volume 3. Glasgow, Scotland; 1989:1945-1948.View ArticleGoogle Scholar
- Watson AB: DCT quantization matrices visually optimized for individual images. In Proc SPIE Int Conf Human Version, Visual Processing, and Digital Display-IV. Volume 1913. San Jose, CA, USA; 1993:202-216.View ArticleGoogle Scholar
- Safranek RJ: A JPEG compliant encoder utilizing perceptually based quantiza-tion. In Proc SPIE Int Conf Human Vision, Visual Processing, and Digital Display - V. Volume 2179. Bellingham; 1994:117-126.View ArticleGoogle Scholar
- Chou CH, Li YC: A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile. IEEE Trans Circ Syst Video Technol 1995, 5(6):467-476. 10.1109/76.475889View ArticleGoogle Scholar
- Bradley AP: A wavelet visible difference predictor. IEEE Trans Image Process 1999, 8(5):717-730. 10.1109/83.760338View ArticleGoogle Scholar
- Höntsch I, Karam LJ: Locally adaptive perceptual image coding. IEEE Trans Image Process 2000, 9(9):1472-1483. 10.1109/83.862622MATHMathSciNetView ArticleGoogle Scholar
- Tang CW: Spatiotemporal visual considerations for video coding. IEEE Trans Multimedia 2007, 9(2):231-238.View ArticleGoogle Scholar
- Al-Fayadh A, Hussain AJ, Lisboa P, Al-Jumeily D: An adaptive hybrid classified vector quantization and its application to image compression. In Proc European Symp Computer Modeling and Simulation. Cambridge, UK; 2008:253-256.Google Scholar
- Yang XK, Lin WS, Lu Z, Ong EP, Yao S: Just-noticeable-distortion profile with nonlinear additivity model for perceptual masking in color images. In Proc IEEE Int Conf Acoustics, Speech, Signal Processing. Volume 3. Hong Kong; 2003:609-612.Google Scholar
- Nadenau MJ, Reichel J, Kunt M: Wavelet-based color image compression: exploiting the contrast sensitivity function. IEEE Trans Image Process 2003, 12(1):58-70. 10.1109/TIP.2002.807358View ArticleGoogle Scholar
- Liu KC, Chou CH: Locally adaptive perceptual compression for color images. IEICE Trans Fund Electron Commun Comput Sci 2008, E91-A(8):2213-2222. 10.1093/ietfec/e91-a.8.2213View ArticleGoogle Scholar
- Chou CH, Liu KC: Colour image compression based on the measure of just noticeable colour difference. IET Image Process 2008, 2(6):304-322. 10.1049/iet-ipr:20080034MathSciNetView ArticleGoogle Scholar
- Watson AB, Yang G, Solomon JA, Villasenor J: Visibility of wavelet quantization noise. IEEE Trans Image Process 1997, 6(8):1164-1175. 10.1109/83.605413View ArticleGoogle Scholar
- Teo PC, Heeger DJ: Perceptual image distortion. In Proc IEEE Int Conf Image Processing. Austin, Texas, USA; 1994:982-986.View ArticleGoogle Scholar
- de Valois KK, Switkes E: Simultaneous masking interactions between chromatic and luminance gratings. J Opt Soc Am 1983, 73(1):11-18. 10.1364/JOSA.73.000011View ArticleGoogle Scholar
- Watson AB: Perceptual optimization of DCT color quantization matrices. In Proc IEEE Int Conf Image Processing. Austin, Texas, USA; 1994:100-1044.View ArticleGoogle Scholar
- Le Callet P, Saadane A, Barba D: Interactions of chromatic components on the perceptual quantization of the achromatic component. In Proc SPIE Human Vision and Electronic Imaging. Volume 3644. San Jose, CA, USA; 1999:121-128.Google Scholar
- Meng Y, Guo L: Color image coding by utilizing the crossed masking. In Proc IEEE Int Conf Acoust, Speech, Signal Processing. Philadelphia, Pennsylvania, USA; 2005:389-392.Google Scholar
- Imai FH, Tsumura N, Miyake Y: Perceptual color difference metric for complex images based on Mahalanobis distance. J Electron Imag 2001, 10(2):385-393. 10.1117/1.1350559View ArticleGoogle Scholar
- Watson AB, Solomon JA: Model of visual contrast gain control and pattern masking. J Opt Soc Am A 1997, 14(9):2379-2391. 10.1364/JOSAA.14.002379View ArticleGoogle Scholar
- Longere P, Zhang X, Delahunt PB, Brainaro DH: Perceptual assessment of demosaicing algorithm performance. Proc IEEE 2002, 90(1):123-132. 10.1109/5.982410View ArticleGoogle Scholar
- Yang XK, Lin WS, Lu ZK, Ong EP, Yao SS: Just noticeable distortion model and its applications in video coding. Signal Process: Image Commun 2005, 20(7):662-680. 10.1016/j.image.2005.04.001Google Scholar
- Berger T: Rate Distortion Theory. Prentice-Hall, Englewood Cliffs, NJ; 1971.Google Scholar
- Jia YT, Lin WS, Kassim AA: Estimating just-noticeable distortion for video. IEEE Trans Circ Syst Video Technol 2006, 16(7):820-829.View ArticleGoogle Scholar
- Wang Z, Bovik AC, Sheikh HR, Simoncelli EP: Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004, 13(4):600-612. 10.1109/TIP.2003.819861View ArticleGoogle Scholar
- Sheikh HR, Bovik AC: Image information and visual quality. IEEE Trans Image Process 2006, 15: 430-444.View ArticleGoogle Scholar
- Chandler DM, Hemami SS: VSNR: a wavelet-based visual signal-to-noise-ratio for natural images. IEEE Trans Image Process 2007, 16(9):2284-2298.MathSciNetView ArticleGoogle Scholar
- Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC: Image quality as-sessment based on a degradation model. IEEE Trans Image Process 2000, 4: 636-650.View ArticleGoogle Scholar
- Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V: On between-coefficient contrast masking of DCT basis functions. In Proc of the 3rd Int Workshop on Video Processing and Quality Metrics for Consumer Electronics. Scottsdale, Arizona, USA; 2007.Google Scholar
- JPEG2000 latest reference software (Jasper Version 1.900.1)[http://www.ece.uvic.ca/~frodo/jasper/]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.