Multispectral Acquisition of Large-Sized Pictorial Surfaces

Abstract

Multispectral acquisition of artworks has recently received considerable attention in the image processing community. Quite understandably, so far this attention has mainly focused on paintings, given their predominant role in museum collections. It is worth pointing out that the instrumentation and procedures used for acquiring regular paintings are not suited for the multispectral acquisition of large-sized painted surfaces such as frescoed halls and great paintings. Given the relevance of such artifacts, and their widespread presence in churches or historical buildings due to their social function, the problem of finding suitable techniques for their acquisition is certainly worth addressing. This paper focuses on multispectral acquisition of large-sized pictorial surfaces, systematically addressing the practical issues related to the acquisition equipment and procedure. Given the crucial role played by the illumination in this application, special attention is given to this issue. The proposed approach is supported by experimental results.

1. Introduction

In the recent years, the acquisition of multispectral images of paintings has become quite popular in the imaging community dealing with cultural heritage applications [1]. A first reason is that acquiring the spectral reflectance is the most reliable asset for faithful color reproduction, where faithfulness is measured in terms of independence from illumination and acquisition devices. Indeed, representation depends both on the acquisition device and the environmental conditions [2]. Independence from the former can be achieved by means of device-independent color spaces but, in any case, the acquisition will be influenced by the environment, mainly by lighting [3]. For this reason, a number of cultural heritage institutions have carried out the construction of "digital museums'' [3], that is, the digitization of their collections, using multispectral sensors. A second advantage of multispectral measurements is that they permit monitoring of the conservation status of paintings. Indeed, multispectral color acquisition is objective, hence repeatable and furthermore, digital images are virtually eternal, since they do not degrade over time. A third important consideration is that multispectral imaging allows material identification [4]. Moreover, the nonvisible parts of the spectrum can render valuable service both in the noninvasive diagnosis of the conservation status of a work of art and in the reconstruction of its history. For example, infrared (IR) reflectography has been used for some decades for the detection of underdrawings, while ultraviolet (UV) fluorescence highlights former restoration interventions. Spectral signatures of color pigments used in the painting process can also be found in the nonvisible spectrum. Unobtrusive diagnosis can therefore lead to virtual restoration planning. The VASARI, MARC, and CRISATEL projects were pioneering projects funded by the European Commission which made use of multispectral data for the acquisition and monitoring of paintings [5–22].

VASARI and MARC considered the multispectral acquisition of paintings in order to derive reliable CIELAB coordinates [7, 10]. A remarkable multispectral imaging project involving the CRISATEL scanner is the "Mona Lisa project''. In 2004, an unprecedented number of techniques were applied to the analysis of the Mona Lisa [23]. The methodologies adopted included radiography, X-ray fluorescence, Raman spectrometry, digital photography under raking light, infrared reflectography, multispectral imaging, and 3D reconstruction. The study was aimed at assessing and recording the conservation status of the painting and at gaining an insight into Leonardo's pictorial technique. Multispectral digitization was performed by the French Center of Museum Research with the camera developed within the CRISATEL project [24]. This multispectral camera comprises 13 filters with a bandwidth of 40 nm spanning the spectrum from 380 nm to 1050 nm. Multispectral measurements were used to identify the various colors of the painting through luminance and color segmentation, to relight the painting with different illuminants and to identify restorations from the most superficial layers (near UV) to the deepest ones (near IR).

Several techniques are currently employed to perform noninvasive spectroscopy in cultural heritage applications. Three of the most common are Fiber Optics Reflectance Spectroscopy (FORS), Imaging Spectroscopy (IS) [25], and Narrow Band Illumination (NBI) spectroscopy. The first approach uses fiber optics to convey light (typically exiting a monochromator) to the target object. Reflectance information is recovered in two possible ways: when a dual-beam approach is used, the incident light beam is split before being directed onto the object, and the unreflected beam is used as a reference to calculate the reflection coefficient. In the single-beam approach, the absolute reflected light intensity is measured. In this case, reflectance can be calculated after acquiring a reference white object. Dual-beam fiber optics spectrometers are very accurate but not easily portable. On the contrary, single-beam fiber optics spectrometers are more compact and easily transportable but can lead to greater measurement errors.

FORS instrumentation operates in a pointwise fashion. On the contrary, IS determines spectral reflectance data for each pixel in a spatial image. Imaging spectrometers can be divided into two classes according to the way multispectral information is recovered. A first approach consists in putting a set of filters in front of the detector [5, 24, 26–28]. The number of filters used in state-of-the-art solutions varies from 7 to 32, and the spectral resolution from 10 nm (narrow-band filters) to 40 nm (wide-band filters) [1]. Another approach consists in using a dispersive element to separate the different light components, which are then detected by a sensor (typically a CCD). In this case, higher spectral resolution (1 nm) can be achieved, but the received signal intensity is considerably lower than in the filter-based approach.

Imaging spectroscopy allows the recovery of the spectral reflectance of the whole target object (or parts of it). Therefore, not only can a faithful color reproduction of the object be achieved, but also more interesting tasks, such as the comparison of the spectral content of different parts of the object, or image segmentation based on spectral features, can be performed. IS methods are generally less spectrally accurate than FORS ones, but allow a more exhaustive inspection of the spectral content of the target. It could be claimed that IS methods are sufficiently accurate if they allow the detection of significant spectral features in localized parts of an object, to which more accurate analysis can be restricted. In this logic, IS and FORS can be considered as complementary. The reliability of the spectral content analysis clearly depends on the accuracy of the spectral reflectance reconstruction, which in turn is determined by several factors, such as the instrument's spectral resolution, the measurement noise, and the choice of the illumination source.

NBI spectroscopy is similar to IS, with the difference that wavelength selection is performed directly at the illumination source by using LEDs, or by placing a monochromator or a set of filters in front of a broadband source. The use of NBI avoids the degradation of the image resolution due to the use of filters in the optical path between the target and the camera in filter-based IS [29]. However, this is not an issue when using a dispersive element to separate wavelength components. NBI spectroscopy also has the advantage of limiting the exposure of the target surface to light, which may be desirable for conservation purposes. As for spectral resolution, the use of LEDs and filters presents the same disadvantages as filter-based IS, while higher spectral resolution (10 nm) can be obtained by using a monochromator. However, the use of a monochromator may be impractical when deploying extended illumination sources to illuminate large surfaces.

All throughout history, frescoed halls or wall-sized paintings have represented a relevant aspect of artistic expression together with regular-sized paintings performed on wood or canvas. Their large dimensions have made them particularly suitable to effectively convey political, religious, or social messages, a function that is similar to that of contemporary large billboards or movie screens. Therefore, size has always been considered as an important means of communication by artists, historians, restorers, and public.

An important and often underestimated consideration regarding the multispectral acquisition of large-sized painted surfaces is that performing it using the instruments currently employed for the multispectral acquisition of paintings [5, 24, 30, 31] is not an optimal choice. First of all, in those instruments the camera is placed close to the painting surface, and a full image is acquired moving the camera by means of linear translators. This solution is unpractical when acquiring surfaces of several squared meters; whereas the use of a rotating stage allows more compactness. Most of all, state-of-the-art instruments for the multispectral scanning of paintings feature illumination sources positioned near the camera so as to illuminate the small area under consideration, generally using halogen lamps due to their broad emission spectrum [5, 30, 31]. In the case of the acquisition of large scenes, one needs the illumination sources not to be integral with the scanner, but of standalone type and positioned several meters far from the scanned surface. This particular setup forces nonstandard choices for the illumination sources. As we will see, the emission of broad spectrum lamps such as halogen and incandescence lamps has a drop in the blue region. While this may not be a problem when the lamps are positioned very close to the target, it turns out that when they are far from the acquired surface the signal in the blue region possibly falls close to or below the dark signal (we recall that illumination intensity is inversely proportional to the square of distance).

In this paper, we take into consideration fundamental issues related to the multispectral acquisition of large-sized surfaces such as those found in frescoed halls or large paintings. The required characteristics of the acquisition equipment and a suitable measurement procedure are defined. Given the critical role of illumination, especially as concerns the use of distant light sources, particular attention is devoted to illumination issues. The considered illumination setups are evaluated according to the corresponding reflectance reconstruction results. In the literature, the latter are usually evaluated by showing qualitative spectral reflectance reconstruction results (e.g., a figure comparing the reference and measured spectral reflectance), and presenting quantitative results concerning the reference and reconstructed or coordinates [30–33]. An important contribution of this work is the proposal of a performance characterization metric based on spectrally dependent errors and error uncertainties, which allows a local evaluation of the reconstruction performance along the considered spectral window. The error average and standard deviation are derived from those of the input signals through error propagation, in conformity with the uncertainty evaluation procedures recommended by metrological standards [34]. A review of the related literature will be done in Section 3.1.

The paper is organized as follows. Section 2 introduces the imaging tools needed for the acquisition of large-sized painted surfaces and a suitable measurement procedure. Section 3 describes the adopted spectrograph performance characterization procedure. Section 4 relates on the application of this procedure to the choice of the illumination sources. Section 5 presents the validation of two illumination setups in laboratory and on-field acquisitions of cultural heritage artifacts. Lastly, Section 6 draws the conclusions.

2. Imaging Equipment and Acquisition Procedure for Large-Sized Pictorial Surfaces

In order to acquire large-sized surfaces, the imaging equipment will feature two main ingredients, namely, a multispectral IS acquisition device, which can be considered as the core of the instrument, and a rotation mechanism for easily covering large surface areas.

As for the multispectral camera, our choice has fallen on the imaging spectrograph Imspector V10 by SPECIM. ImSpector is a direct sight imaging spectrograph provided with a dispersive element that can be quickly combined with a broad range of industrial and scientific monochrome area cameras to form a spectral camera. Compared to conventional color cameras and other filter-based imaging systems, ImSpector produces full contiguous spectral information with high quality spectral and spatial resolution. It can cover a broad spectral range over which it enables flexible wavelength selections via software.

A schematic drawing of the imaging spectrograph is shown in Figure 1. The objective lens focuses the image of the target to be acquired on the plane of the input slit of the spectrograph. The light coming from a rectangular narrow strip conjugated with the slit enters into the spectrograph is dispersed by a dispersive element and focused on the plane of the 2D detector. With reference to Figure 1, the horizontal axis of the camera is the spatial axis, while the vertical axis is the spectral axis. The light coming from the same point on the target but with different wavelengths is focused on the sensor on the same column (i.e., the same spatial position) but in different rows (i.e., different spectral positions). A 2D spectral image can be recovered using a series of monochromatic images of a 2D region on the target obtained by scanning it in the direction perpendicular to the slit [36]. The choice of the objective focal length is imposed by the required field-of-view (FOV) in the direction parallel to the slit. Several objectives are currently available. We normally use a Canon  mm, /# 1.4, that matches the /number of the spectrograph, or a Canon  mm, /# 1.4 objective. The nominal spectral window of the spectrograph is 400 to 1000 nm. The entrance slit size is m. The sensor is the Hamamatsu C8484-05G. It is a progressive scan interline CCD with microlenses, 1024 (spatial) 1344 (spectral) pixel format, pixel size, active area. The CCD is moderately cooled to reduce the thermal noise and increase the sensitivity. The camera has 10  rms (effective value of alternating voltage, expressed in electrons) readout noise, dynamic range of , and quantum efficiency in the 0.35–0.70 range in the 400–750 nm spectral interval. Its performances are significantly higher than a conventional TV camera. The FOV in the direction parallel to the slit is limited by the number of pixels in the spatial direction. A spatial sampling of 2 mm gives a FOV of about 2 m at a distance of about 5 m with the  mm objective.

Figure 1

The SPECIM Imspector V10 spectrograph [ 35 ].

Several solutions are possible to move the spectrograph in the direction perpendicular to the slit, so as to acquire a full 2D image. As previously mentioned, the one that allows greater compactness when scanning large surfaces is a rotating stage. In our setup, we have chosen to use the Physik Instrumente M-062. The scanning in the direction perpendicular to the slit is performed at constant angular steps. This means that the spatial pixel resolution increases with the incident angle between surface normal and scanning direction. However, in practice the maximum incident angle does never get large enough for these variations to be significant. For example, working at a distance of 5 m, with an angular step of 0.5 mrad, and performing 500 acquisitions symmetrically with respect to the surface normal (these were the acquisition parameters used to acquire the fresco in the Castello del Buonconsiglio; see Section 5.2), we reach a maximum incident angle of 125 mrad. This means the spatial pixel resolution varies from 2.5 to 2.54 mm.

The variation of the incident angle also influences the acquired spectral reflectance, as the amount of light reflected by a surface depends both on the lighting and sensing directions [37]. We implicitly assume that all the acquired surfaces are approximately Lambertian, and thus that the reflectance does not vary with the observation direction. From a practical point of view, this means we should be careful when acquiring, for example, frescoes with extended gold leaf coverage. As for possible reflections from the surrounding surfaces, we assume these are negligible in comparison with the light from the illumination sources. What is not accounted for is the dependence of the spectral reflectance on the incident light direction. Therefore, in order to compare different acquisitions for monitoring purposes, the operator should place the illumination sources in matching positions.

A suitable acquisition procedure can be devised remembering that our imaging spectrograph measures absolute light intensities. Therefore, in order to recover spectral reflectance it is necessary to acquire a reference white signal under the same illumination conditions as the target object. This is generally performed by using a white screen of known reflectance superimposed to the target surface. While this is a straightforward task when acquiring a regular painting, several challenges arise when acquiring a larger surface. First of all, the dimensions of a screen covering the whole surface can become prohibitive. Moreover, some parts of the surface, consider, for example, the topmost parts of a wall or of ceilings, may not be reachable. A possible solution is therefore that of acquiring the white signal in a piece-wise manner as far as the target surface can be reached, performing some kind of interpolation during the processing step. A moving support for the screen must therefore be available. It is also worth pointing out that if more complex surfaces are scanned, for example, frescoed vaults, the subtle changes in illumination may not be accurately measurable at all.

The spectral reflectance of the target for any pixel within the sensor is calculated by

(1)

where () spans the spatial dimension and () spans the spectral dimension, is the color signal (the raw signal collected by the sensor), is the reference white signal and is the reference white albedo (the albedo is defined as the spectral reflectance of a Lambertian surface). The symbols and represent the signal-dependent offsets for the color signal and the reference white, respectively. They are calculated as the mean of the first four bands of the acquired signal (lying in the blue region: 393–409 nm), where no useful signal is present due to the low sensitivity of our CCD. Formalizing, we have

(2)

where , . The choice of signal-dependent offsets or dark signals allows to compensate for possible nonlinearities of the camera, which cause the dark current signal to be dependent on the useful signal. Dark current values are always signal and pixel dependent [38]. The most correct procedure would be that of subtracting to each pixel the corresponding dark current value [30, 31], acquired at the same temperature conditions as the active frame. As this would require doubling the number of acquisitions for each frame, a good compromise [38] is that of using inactive pixels to compute a mean value for the dark current. Averaging the offset along several samples allows to isolate the contribution of the dark current from that of the dark current random noise term.

As a last observation, we would like to mention that the choice of this setup for the multispectral camera allows to easily integrate it with other scanning instruments. For example, our imaging spectrograph is coupled with a time-of-flight laser scanner acquiring 3D information. In practical situations, the use of 3D data is particularly useful when dealing with large painted areas, since they are often piece-wise planar rather than planar (e.g., frescoed rooms), and sometimes present a more complex structure (e.g., painted vaults). The architecture of the instrument can be seen in Figure 2. Such an instrument has been employed for the acquisition of small frescoed architectural volumes [32, 39]. However, the illumination setup chosen at the time, featuring two metallic iodide lamps, offered unsatisfactory results in the reconstruction of spectral reflectance, and eventually motivated the presented study.

Figure 2

Schematic (a) and picture (b) of the instrument coupling the multispectral camera with a time-of-flight laser scanner.

2.1. Spectral Calibration

The spectral calibration of an imaging spectrograph consists in determining the correspondence between pixel indices and wavelengths. The calibration has been performed by measuring the spectra of low pressure gas lamps which emit very narrow spectral lines characteristic of the gas in the lamp. The correspondence between pixels and wavelengths has been found to be nearly linear (see Figure 3). The spectral dispersion is 72.1 nm/mm and the spectral resolving element is 0.465 nm/pixel.

Figure 3

Spectral calibration curve.

3. Performance Characterization for the Imaging Spectrograph

3.1. Related Work

An extensive literature has addressed the problem of characterizing the performance of multispectral measurement systems. Some works [40, 41] have addressed the problem of diagnosing and correcting systematic spectrophotometric errors. In particular, Berns and Petersen [41] have defined a method for identifying and correcting the effects of seven types of spectrophotometric errors on multispectral measurements. We do not consider the effects of spectrophotometric errors in our analysis but concentrate on the influence of the illumination. Spectrophotometric errors clearly affect the systematic (average) errors found in our performance characterization procedure. However, since all measurements have been performed using the same instrument under the same experimental conditions, they do not affect the comparison between different illumination sources.

Some works have also studied the propagation of random errors to color measurements [42–44]. However, their analysis is mainly concerned with propagation to colorimetric coordinates. Fairchild and Reniff [43] have calculated wavelength-dependent errors on reflectance as an intermediate step to propagation to and coordinates. However, they consider all input quantities to be uncorrelated, and the dark current term to have zero uncertainty. Our analysis can be considered an extension of this method, accounting for correlated measurement errors and error-affected dark currents. Burns and Berns [44] have also extended the method in [43] to account for correlated measurement errors. However, they have only considered error propagation to CIELAB coordinates, without explicitly considering errors on spectral reflectance. Moreover, their analysis has been performed by considering uncorrelated instrument errors, and only accounting for correlation between color matching functions.

The effects of noise on multispectral measurements under different illumination sources have been addressed by Vrhel and Trussell [45]. Their purpose is to select the set of transmittance filters which minimizes trichromatic color measurement errors wrt a set of a set of fixed illuminants. On the contrary, our aim is that of selecting the illumination source which minimizes wavelength-dependent errors on spectral reflectance.

3.2. Performance Analysis

As previously mentioned, we have chosen to characterize the reflectance reconstruction performance by the error and the error standard deviation calculated as functions of wavelength. We have performed five multiple independent acquisitions of a set of colored calibrated tiles (Spectralon, LabSphere [46]), with spectral reflectance functions specified by the manufacturer, and compared the tabulated and measured spectral reflectance functions. The error standard deviation (uncertainty) has been derived from the statistics of the input signals. The tile set comprises eight colored tiles spanning the visible spectrum (red, orange, yellow, green, cyan, blue, violet, and purple), with spectral reflectance tabulated in the 380–830 nm interval. As a reference white, we have used a Labsphere white tile with a flat 80% spectral reflectance over the interval 250–2500 nm.

The spectral reflectance for the th tile has been computed by considering the color and white signals averaged over the repeated independent acquisitions, so as to decrease the input signal noise. Equation (1) then becomes

(3)

where is the tile index and , , the repetition index. We recall that the index spans the spectral dimension, while spans the spatial dimension. Although the CCD sensor acquires 1344 spectral values for each spatial pixel, we have considered bands averaged over 12-sample bins to lower the acquisition noise. Therefore, from now on we will consider , . Moreover, in the following we shall drop the spatial index , with the understanding that we are considering the central pixel of each tile. Introducing the vector notation , (3) can be rewritten as

(4)

We now define . Equation (4) then becomes

(5)

The reflectance error for each of the eight colors can now be defined as

(6)

where is the tabulated spectrum of the th tile.

We want to calculate the variance of the error from the variances of the experimentally observed variables through error propagation [34]. The observed variables are and , but it will be useful to consider also the newly defined variables and . For the sake of simplicity, we would like to apply the error propagation formula to the derived variables , . We thus need to derive the variance and covariance of , from those of , , , and . In accordance with (4), we have to consider the covariance of the mean vectors , and of the mean variables and , which can be calculated as

(7)

where and , , , . The products in (7) are intended element-wise. The underlying model to these equations is that of an observation given by the sum of the "true'' spectrum with a random noise.

From the definition of , we have that

(8)

where is a vector storing unitary values.

The variance of the derived variable can finally be calculated as (see [34])

(9)

where the partial derivatives can be easily computed from (5) as

(10)

(11)

The variable is the variance of the tabulated reflectance. As it is less than (from the calibration certificate of the Spectralon dataset [46]), it can be neglected in the computation of .

Once we have obtained the error and error variance for each tile, we can average them over the eight tiles to obtain the average error (AE) and the average error variance as

(12)

However, as the standard unit of uncertainty is standard deviation [34], we will eventually use the average error standard deviation (AESTD) instead of the variance. The AESTD is given by

(13)

which is close to, but does not coincide with the square root of the variance expressed in (12).

4. Illumination Selection for Distant Light Sources

The light reflected by an object depends both on the object's surface reflectance characteristics and on the composition of the light illuminating the object. An ideal illumination source for the measurement of an object's spectral reflectance should thus exhibit a uniform spectrum over the whole desired wavelength range. Unfortunately, no existing illumination apparatus satisfies this condition over the visible and near-infrared spectrum, so that suboptimal solutions must be adopted. As previously mentioned, state-of-the-art solutions for the multispectral acquisition of paintings mostly employ halogen lamps due to their smooth and broad spectrum. However, in our case, the inverse-square law for light intensity, combined with the low sensitivity of the CCD in the blue region and with the emission drop of halogen lamps for those wavelengths, causes the white signal to fall close to the dark signal in the blue region. This consideration has suggested the use of two different lamps, one with a broad, smooth spectrum covering the green-red region, and one with a strong emission in the blue region.

Three different lamps have been considered in our experiment:

• (1) a Disano 250 W metallic iodide lamp (Figures 4(a) and 4(b));

• (2) a Cixi Zhongfa Lamps 500 W halogen lamp (Figure 4(c));

• (3) an Osram 100 W incandescence lamp (Figure 4(d)).

Figure 4

Emission spectrum of the three lamps used in our study: (a) metallic iodide lamp, (b) metallic iodide lamp (blue region), (c) halogen lamp, and (d) incandescence lamp.

All the lamps are divergent directional sources. Figure 4 shows the emission spectrum of the lamps. These spectra have been measured by acquiring a white target of known reflectance illuminated by each source. The pictures represent the spectral power density of the lamps, expressed in CCD counts.

During the acquisitions, we have set the camera exposure time and shutter aperture so as to have a strong signal in the blue region when using the metallic iodide lamp. This has caused the signal to saturate in the 530–680 nm interval (Figure 4(a)). However, as we will see, this spectral window will be discarded for the metallic iodide lamp. A zoom of the emission spectrum of the same lamp in the blue region can be seen in Figure 4(b).

As shown in Figures 4(c) and 4(d), the halogen and incandescence lamps mainly emit in the red-NIR spectrum. The two lamps exhibit similar spectra. However, it can be noted that the incandescence lamp has a stronger emission in the NIR region and that its spectrum is slightly smoother.

Four illumination setups have been considered.

• (1) Halogen lamp alone. This setup has been considered for comparison with the results presented in the literature.

• (2) Incandescence lamp alone.

• (3) Metallic iodide lamp and halogen lamp in a sequence. The reflectance measurement has been obtained by juxtaposing the spectral reflectance measured with the metallic iodide lamp in the 420–500 nm interval and that obtained with the halogen lamp in the 500–800 nm interval.

• (4) Metallic iodide lamp and incandescence lamp in a sequence, obtaining the spectral reflectance as in the previous case.

The two combined setups can be considered as a particular kind of bracketing [47], a high dynamic range imaging technique where acquisitions taken at different exposure times are combined to achieve better reconstruction results. The use of two different lamps for the acquisitions (first the saturated metallic iodide lamp, then the halogen or incandescence lamp) is preferable both to the use of two different exposure levels for the metallic iodide lamp, and to a double acquisition with one of the broad spectrum lamps. Obtaining a sufficiently strong signal in the red and blue regions, respectively, would require very long exposure times. As a consequence, the dark current contribution would be increased [48], and the acquisition times would become too long.

When using the combined setups, a scaling has been performed in order to eliminate the discontinuity between the reflectance acquired with the metallic iodide lamp and that measured with the other lamp. This scaling is always performed when measuring real reflectances, therefore the corresponding performance characterization results are meaningful.

For each of these setups, the AE and AESTD have been computed as described in Section 3. The performance characterization results are discussed in the following.

Figure 5 shows the performance characterization results for the four illumination setups described above. In Figure 6, the AESTD is separately shown. In Figure 5(a), the halogen lamp is considered. It can be noted that in the blue region, the AE presents a variable behavior, that tends to get worse the smaller the wavelength. Most of all, the AESTD greatly increases below 480 nm (Figure 6(a)). The reason is that in this region the illumination signal falls close to the dark signal, so that the propagation coefficients of (10) and (11) become larger (note that the absolute value of the coefficient of (11) grows like , where the division is intended component-wise).

Figure 5

AE ± AESTD for (a) the halogen lamp alone, (b) the incandescence lamp alone, (c) the metallic iodide lamp and the halogen lamp used in a sequence, and (d) the metallic iodide lamp and the incandescence lamp used in a sequence.

Figure 6

AESTD for (a) the halogen lamp alone, (b) the incandescence lamp alone, (c) the metallic iodide lamp and the halogen lamp used in a sequence, and (d) the metallic iodide lamp and the incandescence lamp used in a sequence.

A similar consideration can be applied when considering the AESTD of the incandescence lamp used alone (Figure 5(b)). As shown in Figure 6(b), in this case the AESTD is even greater (0.1), probably due to the fact that the emission spectrum of this lamp is slightly translated towards the red-NIR region with respect to that of the halogen lamp (see Figure 4). However, for  nm the incandescence lamp outperforms the halogen lamp, with an AE which is nearly zero between 500 and 700 nm, and below 0.02 (in absolute value) from 700 to 800 nm.

Figure 5(c) shows the results of juxtaposing the reflectance acquired with the metallic iodide lamp between 420 and 500 nm (the exact transition wavelength is  nm), and that measured with the halogen lamp from 500 to 800 nm. In the blue region, the AE is fairly constant and equal to about 0.01 in absolute value. As seen in Figure 6(c), the AESTD is greatly improved (0.01), showing no significant variation with wavelength. The best results are shown in Figures 5(d) and 6(d), when the combined advantages of the metallic iodide lamp in the blue region and of the incandescence lamp in the green-red-NIR regions are benefited. In this case, the AE is nearly zero between 420 and 700 nm, and below 0.02 (in absolute value) between 700 and 800 nm. The AE for –500 nm looks different in Figures 5(c) and 5(d) because of the aforementioned scaling. Observe that the scaling has no effect on the AESTD.

To emphasize the comparison between the use of the incandescence and the halogen lamps, the tabulated and reconstructed spectra for the two combined setups are shown in Figure 7. Although both configuration lead to good qualitative matching between true and measured spectra, the incandescence lamp outperforms the halogen lamp in the red-NIR region.

Figure 7

Tabulated and reconstructed spectra using the metallic iodide+halogen and metallic iodide+incandescence illumination setups for the (a) red, (b) orange, (c) yellow, (d) green, (e) cyan, (f) blue, (g) violet, and (h) purple tiles.

Summarizing, the illumination setup featuring the metallic iodide lamp and the incandescence lamp seems the best to be employed for the acquisition of large painted surfaces. However, it must be considered that halogen lamps have a number of practical advantages over incandescence bulbs, as they are more energy-efficient, produce a higher light intensity with the same power, and last more. For all these reasons, in on-field applications it is more feasible to use halogen lamps. We have therefore considered the combination of the metallic iodide and the incandescence lamps a sort of ideal case, which has been validated in the acquisition of three paintings performed in a controlled environment (see Section 5.1). On the contrary, for on-field acquisitions we have chosen to use the metallic iodide lamp in combination with the halogen lamp.

5. Validation in Cultural Heritage Applications

We are now going to present some results obtained by employing two of the considered illumination setups for the acquisition of paintings. The acquired works of art have been chosen in order to offer a wide variety of pictorial techniques and surfaces. Section 5.1 shows the results obtained acquiring three paintings by Italian contemporary artists. This first set of acquisitions has been performed in one of our laboratories, that is, in a confined, controlled environment. It has been designed to validate our choice of the illumination setup for distant light sources. Therefore, in this particular case, the size of the acquired surfaces is of secondary importance. Section 5.2 describes an on-field validation of our measurement procedure, consisting of the acquisition of a large portion of a fresco in the Castello del Buonconsiglio in Trento (Italy).

For all the acquisitions, we have used as a reference white a white screen of known reflectance and approximately Lambertian behavior. As multispectral measurement achieves independence from the illumination source, an illuminant must be chosen to display the acquisition results in the usual trichromatic coordinates. All the reconstructions presented in this section have been obtained by choosing as illuminant the CIE standard illuminant D65 [49], which approximates daylight illumination conditions. The coordinates for each measured spectral reflectance have been computed using the CIE standard 1931 standard observer. A conversion from to sRGB coordinates has finally been performed.

5.1. Acquisitions in a Controlled Environment

For the laboratory acquisitions, we have chosen the best-performing illumination setup, consisting of the metallic iodide lamp and the incandescence lamp used in a sequence.

The first acquired painting, "Tulips'' is an oil on wood by the contemporary Italian painter and engraver Matteo Massagrande (Padua, 1959). The author's interests lie in the study of ancient pictorial techniques, of engraving and in the art of restoration. His frequent journeys abroad are often occasions to develop pictorial cycles and great compositions. His first exhibition dates back to 1973 and since then he has exhibited his works in several collective and personal exhibitions, and has been awarded several prizes. He currently lives in Padua and works between his studies in Padua and Hajos (Hungary).

For the acquisition of "Tulips'' the two lamps have been positioned at approximately with respect to the object plane, so as to minimize the reflections. The reconstruction of "Tulips'' can be seen in Figure 8. The quality of the reconstructed colors is very good. The resolution is limited along the horizontal axis by the rotating stage acquisition step.

Figure 8

"Tulips'' by Matteo Massagrande (oil on wood).

Figure 9(b) shows the spectral reflectance of four points indicated in Figure 9(a). Point 2 has the typical behavior of a red, with a high reflectance over the 620–800 nm interval and almost no light reflected in the blue region, can be recognized. The grey points 1 and 4 have a constant spectral reflectance over most of the visible spectrum, with an increasing albedo from darker to lighter shades. Point 3 is a composite color with characteristics of both green and yellow: a bell-shaped spectral reflectance in the central part of the visible spectrum (typical of green) is mixed with a flat, high spectral reflectance for long wavelengths (yellow).

Figure 9

(b) measured spectral reflectance of the points indicated in (a) for the painting "Tulips''.

"Peach'' is another oil on wood by Matteo Massagrande. It has been acquired under the same illumination conditions and using the same setup as for "Tulips''. Its reconstruction can be seen in Figure 10.

Figure 10

"Peach'' by Matteo Massagrande (oil on wood).

Figure 11(b) shows the spectral reflectance of four points (Figure 11(a)) chosen on this painting. The sky point (Point 1) shows the behavior typical of blue (a bell-shaped bump in the 450–550 nm interval), but also reveals the presence of an orange-red component. Point 2 is a typical pink, with light reflected along the whole spectrum, but especially around 600 nm and in the orange-red region. Point 3 is a very dark green, while Point 4 exhibits the characteristics of a dark brown, with a behavior similar to that of orange-red but with a smoother transition from low to higher reflectance values.

Figure 11

(b) measured spectral reflectance of the points indicated in (a) for the painting "Peach''.

The last painting we have measured is "Bullfighter'', an oil on canvas by another contemporary Italian painter, Vittorio Buzzanca. The main difference between this painting and the other two lies in the presence of a protective glass over the canvas. The glass introduces effects of reflection and dispersion, which can lead to a blurred reconstruction. The acquisition configuration has thus been slightly modified in order to obtain better results. The illumination sources have been positioned at a more grazing angle with respect to the painting's surface, and the painting itself has been brought closer to the camera. The reconstruction obtained with this setup is shown in Figure 12.

Figure 12

"Bullfighter'' by Vittorio Buzzanca (oil on canvas).

Figure 13(b) shows the spectral reflectance of the points indicated in Figure 13(a). In this case, it is very easy to recognize the spectral reflectance of a lighter and darker blue (Points 1 and 4, resp.) and that of the two red spots (Points 2 and 3).

Figure 13

(b) measured spectral reflectance of the points indicated in (a) for the painting "Bullfighter''.

5.2. On-Field Validation

An on-field validation of our measurement procedure has been performed by acquiring a portion of a fresco painted by Girolamo di Romano, known as "Romanino'' in the Castello del Buonconsiglio in Trento (Italy). Romanino was born in 1485 in Brescia (Italy), a city in which one of the most important north Italian schools of Renaissance painting had developed. During his youth, he fell under the spell of Venetian painting, especially that of Giorgione and Titian, and of Milanese painting. As he matured, however, he developed a very personal style, drawing inspiration especially from the very dramatic pictorial style of German art, as demonstrated in the magnificent cycle of paintings for Cremona cathedral (1519). Romanino was a versatile artist; he painted on panels and on canvas, but he favored the technique of fresco, the means of expression which he found most congenial. In addition, Vasari counted him among the most capable draughtsmen of his time. In 1531 he was offered to decorate the Castello del Buonconsiglio for the Prince Bishop of Trento, Bernard Cles, and completed a large cycle of paintings, with secular themes, in the castle.

The acquired scene, an rendition of which is shown in Figure 14, is a portion of a fresco representing a rest after the hunt, with a servant (on the right) leaning towards his master, holding a dove on his arm. We will call this acquired portion "The dove''. The fresco is situated in a vault of the "Volto sotto la Loggia'', one of the rooms of Magno Palazzo, and its dimensions are approximately . The acquired area is positioned at a height of approximately 2.5 m. A picture of the acquisition setup can be seen in Figure 15. It can be noted that the illumination sources are placed several meters far from the target surface, also because of the presence of the staircase. Therefore, two metallic iodide lamps and two halogen lamps have been used to illuminate the scene. Figure 15 shows the two metallic iodide lamps symmetrically mounted on a tripod.

Figure 14

"The dove'' by Romanino.

Figure 15

Acquisition setup in the Castello del Buonconsiglio, Trento (Italy).

As previously mentioned in Section 2, the acquisition of the white signal is often difficult when acquiring large scenes, and particularly difficult when acquiring a vault. As it often happens, in this case we did not possess a sufficiently large white panel to cover the whole scanned area. We have therefore acquired the white signal in two different steps, placing the white panel (mounted on a tripod) in different, but partially overlapping, positions. The final reference white has been obtained by exploiting a property of our illumination system, that is, that the spectral content of the illumination function (considered as the juxtaposition of the spectra of the two lamps) is constant [50]. This allows to express the illumination function as

(14)

where are the spatial coordinates, is a spatially dependent coefficient accounting for the illumination intensity, and is a known spectrally dependent function.

To recover the white signal, we have calculated on the two panels, and fitted a plane in the least-square sense to the two resulting surfaces. The interpolation of the plane on the whole image support has been taken as the reference white signal. Figure 16 shows the input and output of the interpolation process. It is worth pointing out that is clearly unrealistic to assume that of Figure 16(c) is the actual illumination of the left part of the vault, which lies on on a different, nonplanar surface. However, as no paintings are present in that region, we have given up the acquisition of the actual white signal for the presented study, and used one single interpolated surface for the illumination coefficients . As the interpolated white signal satisfies the physical constraints for the illumination on all its support, the corresponding reflectance values are feasible, if not correct.

Figure 16

(a) k coefficient corresponding to the white panel in position 1; (b) k coefficient corresponding to the white panel in position 2; (c) interpolated planar surface for the coefficients k (p).

Figure 17(b) shows the spectral reflectance of the points shown in Figure 17(a). It can immediately be observed that all the colors are quite dark, probably due to aging. Nonetheless, the behavior of blue (Point 1), red (Point 2), yellow (Point 3), and green (Point 4) is still perfectly recognizable.

Figure 17

(b) measured spectral reflectance of the points indicated in (a) for the fresco "The dove''.

6. Conclusions

Large-sized pictorial surfaces such as frescoed rooms or great paintings represent a relevant aspect of artistic expression. Their acquisition poses different challenges from those encountered when acquiring regular paintings, the most relevant dissimilarity being that in the former case the illumination sources must be placed far from the acquired scene. Therefore, the instruments and procedures used for the multispectral acquisition of regular-sized paintings are not suited to the acquisition of large surfaces. This paper systematically addresses the issues involved in multispectral acquisition of large-sized painted surfaces. Suitable acquisition instrumentation comprising an IS multispectral camera mounted on a rotatory device is exemplified. As illumination selection turns out to be a rather critical task, the issue is discussed in detail. We have considered four illumination setups: a halogen lamp alone (the illumination source most commonly used for the multispectral acquisition of paintings), an incandescence lamp alone, and two composite setups comprising a metallic iodide lamp for the measurement of reflectance between 420 and 500 nm, and the halogen or incandescence lamp for recovering the reflectance between 500 and 800 nm. These illumination configurations have been tested on the acquisition of the spectral reflectance of a set of calibrated colored tiles. We have also examined the delicate issue of performance evaluation and defined the error as a function of wavelength, using a metrological procedure to infer the uncertainty of the computed error from the statistics of the measured variables. To quantify the performance of reflectance measurements, we have used the average error (AE) and the average error standard deviation (AESTD), calculated for each illumination setup and averaged over the eight-tile set. Although both the combined illumination setups yield a good performance, the best results have been obtained using the metallic iodide lamp in combination with the incandescence lamp (, ). However, considerations of practical nature suggest that in on-field applications, the employment of halogen lamps is preferable to that of incandescence bulbs. Both illumination configurations have therefore been validated in laboratory and in-field conditions. The combination of metallic iodide and incandescence lamps has been deployed for the acquisition of three paintings from Italian contemporary artists in a controlled environment. The setup featuring the metallic iodide and halogen lamps has then been used for the on-field acquisition of a large portion of a fresco of the Castello del Buonconsiglio in Trento (Italy).