Context-Based Defading of Archive Photographs
© V. Bruni et al. 2009
Received: 30 January 2009
Accepted: 15 September 2009
Published: 8 November 2009
We present an algorithm for the enhancement of contrast in digitized archive photographic prints. It aims at producing an adaptive enhancement based on the local context of each pixel and is able to operate without direct user's intervention. A relation between the variation of contrast at different resolutions and the local Lipschitz regularity of the image is exploited. In this way, each pixel is defaded according to its nature: noise, edge, or smooth region. This strategy provides for an algorithm that drastically reduces typical, annoying artifacts like halo effects and noise amplification.
In order to enable the researcher or the public at large to visualize an image of the faded photograph as similar as possible to the original one, digital acquisition and processing is the only possible approach. Photographic archives acquire their images using professional scanning equipment and create digital versions of their art. The latter can then undergo a process of "virtual" restoration, for example, through a proper contrast enhancement algorithm.
Contrast enhancement is a well-known and challenging problem in image processing. In general, it aims at a recovery of the original vividness of images having a suboptimal contrast. A wide range of approaches have been proposed in literature in both the spatial and transform domains. Examples in the transform domain are alpha-rooting techniques, and techniques based on scaling the DCT coefficients. Alpha-rooting was first presented in , and it has been successively modified in [4–6], since it can be combined with different transforms. A recent version of alpha-rooting is described in ; it is based on properties of a tensor representation of the DFT. A DCT-domain operation is suggested in , where all the three attributes of brightness, contrast, and color of an image are addressed. It is based on a simple and computationally efficient algorithm, that only requires scaling of the DCT coefficients—mostly by a factor which remains constant in a block.
In the spatial domain, in addition to the use of simple linear techniques which emphasize the high-frequency contents of an image (the so-called unsharp masking approach), the most famous approaches are probably the Retinex model, based on Land's studies , and histogram equalization . A set of modifications has been proposed for the improvement of these methods. In particular, it is interesting to note that both methods have evolved to include a multiscale (i.e., multiresolution) version, based on convolution with smoothing kernels. The evolution of the methods has incorporated the estimation of a context, based on a global measure in a suitable neighborhood, allowing adaptive enhancement [11–14]. In fact, there is a general agreement about the fact that these two factors greatly improve the performance of any contrast-enhancement framework . However, they are also responsible for unavoidable undesired artifacts like oversmoothing (with a loss of details) or excessive enhancement (with a resulting amplification of noise and/or halo effects) . Even though some sophisticated approaches have been proposed for their reduction [17, 18], these artifacts remain an aspect to be considered in the design of any contrast-enhancement framework. The situation is even more difficult when scanned antique photographic prints are processed. In this case, the presence of defects in the original art may introduce specific artifacts in the digital item, which in turn produce particularly annoying effects if conventional enhancement techniques are applied.
In this paper we present an adaptive enhancement tool that tries to overcome the above-mentioned problems. It is based on a multiscale approach that exploits the local context. In particular, it exploits the link between the change of contrast (as the resolution is increased) and the local Lipschitz regularity of the image [19, 20]. Such a link can be used for asserting the (possibly) noisy nature of each pixel, avoiding convolutions with kernels that would introduce the aforementioned artifacts. On the other hand, a measure of contrast at different resolutions allows to exploit visibility laws, such as the Weber-Fechner law; they are used in the assessment of the importance, and then the enhancement of each pixel of the image under study.
After the pixels have been classified (edge, noise, or smooth region), their contrast is changed appropriately. Then, at a successive stage, an optimal (global) gamma correction tool that exploits the results in  is performed. The proposed framework has been tested on various digitized historical photographic prints subjected to fading. Experimental results show good results in terms of subjective quality and a good efficiency even in critical cases. To make a more objective evaluation of the results, comparisons with representative contrast enhancement methods have been introduced. Moreover, several quality measures have been used to quantify the visual appearance of the restored images.
The paper is organized as follows. Section 2 presents the proposed model; it includes the detailed algorithm and a description of each of its three phases. Section 3 contains some experimental results and comparative studies. Finally, some discussions, conclusions, and guidelines for future research are the topic of Section 4.
2. The Proposed Model
The proposed method, initially explored in , consists of three main stages. In the first one, the image is preprocessed and its pixels are classified according to the inferred type of damage suffered. In particular, we check if a pixel belongs to a blotch (a common fault in antique photos) in the image. This operation allows for a more appropriate estimation of the parameters in the two remaining stages. In the second stage, the link between the local Lipschitz regularity and the change of contrast of the image across scales is exploited; after this stage, adaptive contrast enhancement can be performed on the faded image. The aim of the second stage is to differentiate the type of defading to be applied to each pixel according to its nature (edge, noise, or flat region). In the third stage, the image is defaded using a contrast-enhancement tool that is based on the classical characteristic curve , with (as in gamma correction). In order to automatically estimate an optimal value of , we exploit the results presented in  that are based on the following observation: visually pleasant images show a sort of orthogonality between the local first moment and the local second central moment of the distribution of the luminance values. It is interesting to note that  reports a statistical independence between luminance and contrast in natural images. (Mante et al. use the weighted sums and to measure local contrast and luminance, resp., where is the pixelwise luminance, and the weights decrease with the distance from the center of the context.) In the following, the aforementioned stages are described in detail.
In the first stage, roughly called deblotching, the regions with a color that is stronger than the more common (faded) colors in the remaining parts of the image are detected. We use the term "strong" here since, for achromatic images, to say that a region is saturated black or white is perhaps misleading. Observing such dark and bright blotches in Figure 1, it can be seen that there are two main reasons for performing deblotching. First, blotches would increase their appearance after any contrast enhancement operation with the result that the defaded image would be conspicuously spotted, compromising its global visual quality. The second reason is that blotch pixels have statistical properties that are different from those in the rest of the image. Hence, to ignore blotch pixels allows an improved estimation of the parameters in the remaining stages.
2.2. Lipschitz-Based Contrast Enhancement
The phenomenon of fading is often accompanied by noise resulting from a chemical degradation of the photographic emulsion. The aim of this stage is then to produce an image where the contrast of each pixel is changed depending on whether it is part of a noisy, an edge, or a flat region. The analysis carried out in this section is local; global corrections are addressed in the third phase. We are interested here in analyzing the link between the pointwise Lipschitz regularity and the variation of contrast of the image. It is well-known that the Lipschitz coefficient gives information about the (possibly) noisy nature as well as the regularity of each point .
We assume that in a neighborhood of the pixel the image is locally smooth. This means that it can be locally approximated by a polynomial of degree in the variable . It turns out that the local background of the pixel at is still a polynomial function. In fact, it is the mean value of in the region . More precisely,
where the integral is a polynomial function whose degree does not exceed , as proved in the appendix. It turns out that is a polynomial function with respect to where while Hence ( means that has the same order of ).
As a result, the contrast variation can be linked to the Lipschitz regularity as
Integrating by separation of variables,
It is important to notice that the result above permits to impose some constraints on choices usually made by hand in other methods proposed in literature. First of all, only two scale levels are required for the discrimination between noisy and uncorrupted points of the faded image. Indeed, taking into account the pointwise nature of the noise, two levels among all the possible ones can be selected. Furthermore, no additional thresholding is required for discriminating the nature of each pixel and selecting the corresponding enhancement function. Finally, the size of the context used for the computation of the contrast coincides with the support of the regularizing function, and the mean can be seen as the convolution between the image and a Haar basis function at a given scale. It is obvious that the aforementioned considerations are valid just in case of contrast enhancement under noise and not in general. In the latter case, the parameters above have to take into account the local frequency information of the image as well; consider, for example, textures. This would imply the use of a more sophisticated measure of contrast that would take into account not only the spatial information (local mean) but also the frequency (in terms of dominant frequency values) in the same region.
2.3. Defading and Image-Quality Measure
To complete the defading process, a global (i.e., uniform in the image) luminance mapping is applied. It is based again on a power-law function, . This mapping depends on the choice of the parameter which is made using an image quality measure. The distribution of the local standard deviation with respect to the local average of the luminance has been recently used in order to define a figure of merit that was used in a restoration algorithm applied to faded images . It has been shown that these two statistical parameters live constrained in a bell-shaped region of the plane ( , ) . We use here the same approach, in order to get an estimate of the optimal values of the parameters used in the algorithm described above.
Let us suppose that we acquire a digital image from a given real-world scene using an ideal linear device and consider only its luminance values for simplicity. We subdivide the image into adjacent blocks, and calculate the standard deviation and the average of the luminance or gray level within each block. In the ( , ) plane each block is then represented by a point. If we imagine to repeat this procedure for a huge set of scenes with all sorts of conceivable contents, and to display the corresponding values ( , ) in a single plane, we will probably get a cloud of points showing no correlation between and . There is no reason indeed why the average of the luminance of an object in the real world should influence the standard deviation of the same luminance. Notice that this consideration does not contradict Weber's law, which is related to our perception of the scene, and is not a property of the scene itself. The situation is different if, as it happens in practice, the dynamic range of the acquisition device is limited; in this case, very dark and very bright blocks present a limited deviation. In fact, it can be demonstrated that the values of lie now in a limited range bounded above by a bell-shaped function of the average; the function takes its maximum value when the average is half the available range and falls to zero when the average corresponds to the minimum or the maximum of the luminance range .
tends to assume negative values for the degraded picture. After the processing, the shape of the cloud of points in the ( , ) plane corresponds to values of close or equal to zero. Thus, we use closeness of to 0 as a quality criterion for the choice of the parameter in Phase 3, as it will be shown in the following section and in Figure 6.
It is worth outlining that image quality measurement is of course a complex subject. The total amount of contrast in an image is sometimes considered as a measure of image quality since, quite often, the larger the total contrast, the better the image. In fact, for the restoration of faded prints, gamma correction increases the average value of . In addition to our Weber-related definition of contrast, and that in , one further definition is the well-known Michelson contrast :
In LIP arithmetic (assuming the bounded range [ ] for the intensity magnitude) one has, for f and g intensity values and a real scalar, ; ; , and . LIP arithmetic has the important advantage of respecting the bounded luminance range, for example, [ ], of an image; also, Weber's law can be expressed in LIP arithmetic. Thus, LIP arithmetic is advisable when the result of the operation is to be used as an intensity value, and perhaps also in the present case since LIP arithmetic is related to human visual perception issues. The entropy version AME2 stresses the importance of uniformly distributed local contrast. The mentioned quality indicators will be considered in the experiments described in Section 3.
2.4. The Algorithm
It is worth stressing that sepia images are the input of the proposed algorithm. For this reason, only their luminance component has been processed and is shown; the two chrominance components can be kept unchanged if desired.
3. Experimental Results
The proposed framework has been tested on various images coming from the Fratelli Alinari Archive in Florence, Italy. In this paper we consider the two images shown in Figure 1 and the ones on the left side of Figure 8.
values and quality metrics of the corresponding corrected image, as depicted in Figure 9.
The restoration application we address is not characterized by real-time needs; nonetheless, the operations performed by the proposed algorithm are very simple and the required computing time is comparable to the ones required by the mentioned competing approaches.
4. Discussion and Conclusions
In this paper we have presented a framework aimed at giving faded images their original vividness. After the application of an adaptive technique of contrast enhancement that exploits the link between local Lipschitz image regularity and the change of contrast, a global power-law correction is performed. The proposed model allows for a gradual enhancement of the image that avoids drawbacks like halo and noise amplification. In a forthcoming paper we explore further the theoretical framework presented in Section 2.2, using more sophisticated bases such as those in . For the specific usage on faded photographic prints, the experiments we have performed indicate that the proposed method gives a satisfactory performance. However, a few issues should be addressed in future works. First of all we observe that the estimate we use for the Lipschitz regularity is slightly noisy; this affects in particular quasihomogeneous areas where the contrast is very low. An improved definition of contrast that permits a stronger dependence of the power-term correction on the local characteristics of smooth image areas should be devised. Finally, it would be convenient if an optimum balance between the local and the global correction stages could be automatically attained, since the ( , ) method does not yield a satisfactory input for this purpose. For pictures having a nonuniform exposure to light, it would be more reasonable to differently treat two or more portions of the image itself. In this case, some user intervention would be required.
This work has been supported by the Italian Ministry of Education as a part of the Firb Project no. RBNE039LLC. The authors wish to thank F. lli Alinari SpA for providing the pictures used in the experiments.
- Reilly JM: The question of permanence. In The Albumen & Salted Paper Book: The History and Practice of Photographic Printing, 1840–1895. Light Impressions, Rochester, NY, USA; 1980.Google Scholar
- Wilhelm H, Brower C: The Permanence and Care of Color Photographs. Preservation, Grinnell, Iowa, USA; 1993.Google Scholar
- Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, Upper Saddle River, NJ, USA; 1989.MATHGoogle Scholar
- Aghagolzadeh S, Ersoy OK: Transform image enhancement. Optical Engineering 1992,31(3):614-626. 10.1117/12.56095View ArticleGoogle Scholar
- Agaian SS, Silver B, Panetta KA: Transform coefficient histogram-based image enhancement algorithms using contrast entropy. IEEE Transactions on Image Processing 2007,16(3):741-758.View ArticleMathSciNetGoogle Scholar
- Panetta KA, Wharton EJ, Agaian SS: Human visual system-based image enhancement and logarithmic contrast measure. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2008,38(1):174-188.View ArticleGoogle Scholar
- Turkay Arslan F, Grigoryan AM: Fast splitting alpha-rooting method of image enhancement: tensor representation. IEEE Transactions on Image Processing 2006,15(11):3375-3384.View ArticleGoogle Scholar
- Mukherjee J, Mitra SK: Enhancement of color images by scaling the DCT coefficients. IEEE Transactions on Image Processing 2008,17(10):1783-1794.View ArticleMathSciNetGoogle Scholar
- Land EH, McCann JJ: Lightness and retinex theory. Journal of the Optical Society of America 1971,61(1):1-11. 10.1364/JOSA.61.000001View ArticleGoogle Scholar
- Hummel R: Image enhancement by histogram transformation. Comput Graphics Image Process 1977,6(2):184-195. 10.1016/S0146-664X(77)80011-7View ArticleGoogle Scholar
- Jobson DJ, Rahman Z-U, Woodell GA: A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing 1997,6(7):965-976. 10.1109/83.597272View ArticleGoogle Scholar
- Tao L, Asari VK: Modified luminance based MSRCR for fast and efficient image enhancement. Proceedings of the 32nd IEEE Applied Imagery Pattern Recognition Workshop (AIPR '03), 2003, Washington, DC, USA 174-179.Google Scholar
- Pizer SM, Zimmerman JB, Staab EV: Adaptive grey level assignment in CT scan display. Journal of Computer Assisted Tomography 1984,8(2):300-305.Google Scholar
- Jin Y, Fayad LM, Laine AF: Contrast enhancement by multiscale adaptive histogram equalization. Wavelets: Applications in Signal and Image Processing IX, 2001, Proceedings of SPIE 206-213.View ArticleGoogle Scholar
- Yoon B-W, Song W-J: Image contrast enhancement based on the generalized histogram. Journal of Electronic Imaging 2007.,16(3):Google Scholar
- Tao L, Asari VK: Adaptive and integrated neighborhood-dependent approach for nonlinear enhancement of color images. Journal of Electronic Imaging 2005.,14(4):Google Scholar
- Pattanaik SN, Ferwerda JA, Fairchild MD, Greenberg DP: A multiscale model of adaptation and spatial vision for realistic image display. Proceedings of the Annual Conference on Computer Graphics (SIGGRAPH '98), July 1998, Orlando, Fla, USA 287-298.Google Scholar
- Tumblin J, Turk G: LCIS: a boundary hierarchy for detail-preserving contrast reduction. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99), 1999 83-90.View ArticleGoogle Scholar
- Mallat S: A Wavelet Tour of Signal Processing. Academic Press, San Diego, Calif, USA; 1998.MATHGoogle Scholar
- Mallat S, Hwang WL: Singularity detection and processing with wavelets. IEEE Transactions on Information Theory 1992,38(2):617-643. 10.1109/18.119727View ArticleMathSciNetMATHGoogle Scholar
- Ramponi G: Adaptive contrast improvement for still images and video frames. Proceedings of the IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing (NSIP '07), September 2007, Bucharest, RomaniaGoogle Scholar
- Bruni V, Ramponi G, Restrepo A, Vitulano D: Restoration of faded images without noise amplification. Proceedings of the 16th IEEE-EURASIP European Signal Processing Conference (EUSIPCO '08), 2008 SS126.96.36.199-SS188.8.131.52.Google Scholar
- Mante V, Frazor RA, Bonin V, Geisler WS, Carandini M: Independence of luminance and contrast in natural scenes and in the early visual system. Nature Neuroscience 2005,8(12):1690-1697. 10.1038/nn1556View ArticleGoogle Scholar
- Gutierrez J, Ferri FJ, Malo J: Regularization operators for natural images based on nonlinear perception models. IEEE Transactions on Image Processing 2006,15(1):189-200.View ArticleMathSciNetGoogle Scholar
- Restrepo A, Ramponi G: Filtering and luminance correction for aged photographs. Image Processing: Algorithms and Systems VI, January 2008, San Jose, Calif, USA, Proceedings of SPIE 6812: 26-31.Google Scholar
- Restrepo A, Ramponi G: Word descriptors of image quality based on local dispersion-versus-location distributions. Proceedings of the 16th European Signal Processing Conference (EUSIPCO '08), August 2008, Lausanne, SwitzerlandGoogle Scholar
- Velde KV: Multi-scale color image enhancement. Proceedings of the IEEE International Conference on Image Processing (ICIP '99), October 1999, Kobe, Japan 3: 584-587.View ArticleGoogle Scholar
- Peli E: Contrast in complex images. Journal of the Optical Society of America A 1990,7(10):2032-2040. 10.1364/JOSAA.7.002032View ArticleGoogle Scholar
- Pinoli J-C, Debayle J: Logarithmic adaptive neighborhood image processing (LANIP): introduction, connections to human brightness perception, and application issues. EURASIP Journal on Advances in Signal Processing 2007, 2007:-22.Google Scholar
- Agaian SS, Lentz KP, Grigoryan AM: A new measure of image enhancement. Proceedings of the of IASTED International Conference on Signal Processing and Communications, 2000Google Scholar
- Gobbers JF, Vandergheynst P: Directional wavelet frames: design and algorithms. IEEE Transactions on Image Processing 2002, 11: 363-372. 10.1109/TIP.2002.999670View ArticleMathSciNetGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.