Edge Adaptive Color Demosaicking Based on the Spatial Correlation of the Bayer Color Difference

Abstract

An edge adaptive color demosaicking algorithm that classifies the region types and estimates the edge direction on the Bayer color filter array (CFA) samples is proposed. In the proposed method, the optimal edge direction is estimated based on the spatial correlation on the Bayer color difference plane, which adopts the local directional correlation of an edge region of the Bayer CFA samples. To improve the image quality with the consistent edge direction, we classify the region of an image into three different types, such as edge, edge pattern, and flat regions. Based on the region types, the proposed method estimates the edge direction adaptive to the regions. As a result, the proposed method reconstructs clear edges with reduced visual distortions in the edge and the edge pattern regions. Experimental results show that the proposed method outperforms conventional edge-directed methods on objective and subjective criteria.

1. Introduction

Single chip CCD or CMOS imaging sensors are widely used in digital still cameras (DSCs) to reduce the cost and size of the equipments. Such imaging sensors obtain pixel information through a color filter array (CFA), such as Bayer CFA [1]. When the Bayer CFA is used in front of the image sensor, one of the three spectral components (red, green, or blue) is passed at each pixel location as shown in Figure 1(a). In order to obtain the full color image, the missing color components should be estimated from the existing pixel information. This reconstruction process is called color demosaicking or color interpolation [2–25]. Generally, the correlation between color channels is utilized by assuming the smoothness color ratio [3, 4] or smoothness color difference [5–7]. These methods produce satisfactory results in a homogeneous region, while visible artifacts (such as zippers, Moiré effects, and blurring artifacts) are shown in edge regions.

Figure 1

(a) The Bayer CFA pattern and (b) the down sampled low resolution images.

In order to reduce interpolation errors in these regions, various approaches have been applied to color demosaicking. In [8–12], various edge indicators were used to prevent interpolation across edges. Gunturk et al. decomposed color channels into frequency subbands and updated the high-frequency subbands by applying a projection onto convex-sets (POCS) technique [13]. Zhang and Wu modeled color artifacts as noise factors and removed them by fusing the directional linear minimum mean squares error (LMMSE) estimates [14]. Alleysson et al. proposed frequency selective filters which adopt localization of the luminance and chrominance frequency components of a mosaicked image [15]. All of these approaches show highly improved results on the edge regions. However, the interpolation error and smooth edges in edge patterns or edge junctions are challenging issues in demosaicking methods.

As an approach to reconstruct the sharp edge, edge directed color demosaicking algorithms were proposed which aimed to find the optimal edge direction at each pixel location [16–25]. Since the interpolation is performed along the estimated edge direction, the edge direction estimation techniques play a main roll in these methods. In some methods [20–22], the edge directions of missing pixels are indirectly estimated in aid of the additional information from the horizontally and vertically prereconstructed images. Wu and Zhang found the edge direction based on the Fisher's linear discriminant so that the chance of the misclassification of each pixel is minimized [20]. Hirakawa and Parks proposed a homogeneity map-based estimation process, which adopted the luminance and chrominance similarities between the pixels on an edge [21]. Menon et al. proposed the direction estimation scheme using the smoothness color differences on the edges, where the color difference was obtained based on the directionally filtered green images [22]. In these methods, the sharp edges are effectively restored with the temporally interpolated images. However, the insufficient consideration for the competitive regions results in outstanding artifacts due to the inconsistent directional edge interpolation.

Recently, some methods that directly deal with the CFA problems such as CFA sampling [23–25], CFA noise [26] or both of the problems [27] were proposed. These methods studied the characteristics of the CFA samples and reconstructed the image without the CFA error propagation and the inefficient computations due to the preinterpolation process. Focusing on the demosaicking directly on the CFA samples, Chung and Chan studied the color difference variance of the pixels located along the horizontal or the vertical axis of CFA samples [23]. Tsai and Song introduced the concept of the spectral-spatial correlation (SSC) which represented the direct difference between Bayer CFA color samples [24]. Based on the SSC, they proposed heterogeneity-projection technique that used the smoothness derivatives of the Bayer sample differences on the horizontal or vertical edges. Based on the Tsai and Song's method, Chung et al. proposed modified heterogeneity-projection method that adaptively changed the mask size of the derivative [25].

As shown in [24, 25], difference of the Bayer samples provides key to directly estimate the edge direction on the Bayer pattern. In the conventional SSC-based methods, the smoothness of the Bayer color difference along an edge is examined, and the derivative of the differences along the horizontal or vertical axis is adopted as a criterion for edge direction estimation. However, in the complicated edge region, such as edge patterns or edge junctions, the edge direction is usually indistinguishable since derivatives along the line are very close to the horizontal and vertical directions. To carry out more accurate interpolation on these regions, region adaptive interpolation scheme which estimates the edge direction adaptive to the region types with the given directional correlation on Bayer color difference is required.

In this paper, a demosaicking method that estimates the edge direction directly on the Bayer CFA samples is proposed based on the spatial correlation of the Bayer color difference. To estimate the edge direction with accuracy, we investigate the consistency of the Bayer color difference within a local region. We focus on the local similarity of the Bayer color difference plane not only along the directional axis but also beside the axis within the local region. Since the edge directions of the pixels on and around the edge contribute to the estimation simultaneously, the correlation adopted in the proposed method is a stable and effective basis to estimate the edge direction in the complicated edge regions. Based on the spatial correlation on the Bayer color difference plane, we propose an edge adaptive demosaicking method that classifies an image into edge, edge pattern, and flat regions, and that estimates the edge direction according to the region type. From the result of the estimated edge direction, the proposed method interpolates the missing pixel values along the edge direction.

The rest of the paper is organized as follows. Using the difference plane of the down sampled CFA images, the spatial correlation on the Bayer color difference plane is examined in Section 2. Based on the examined correlation between the CFA sample differences, the proposed edge adaptive demosaicking method is described with the criteria for the edge direction detection and the region classification in Section 3. Also, the interpolation scheme along the estimated edge direction is depicted, which aims to restore the missing pixels with reduced artifacts. Section 4 presents comparisons between the proposed and conventional edge directed methods in terms of the quantitative and qualitative criteria. Finally, the paper is concluded with Section 5.

2. Spatial Correlation on the Bayer Color Difference Plane

In the proposed method, the region type and the edge direction are determined directly on the Bayer CFA samples based on the correlation of the Bayer color difference. For the efficient criteria for these main parts of the proposed demosaicking method, the Bayer color difference is reexamined on the down sampled low-resolution (LR) Bayer image plane so that the direction-oriented consistency of the Bayer color differences is emphasized within the local region of an edge.

The Bayer color difference is a strong relation between the CFA samples on a horizontal or vertical line [24], followed as

(1)

where the , and are Bayer CFA samples of red and green channels in pixel location, respectively, is a missing sample of green channel, and and are the Bayer color difference on the horizontal and vertical directional lines, respectively. The Bayer color difference is assumed piecewise constant along an edge since it inherits the characteristics of spectral and spatial correlations [24].

From the relation between the CFA samples on a line, we expand the CFA sample relation into the Bayer color difference plane which is defined by the difference of Bayer LR images. When we consider the down sampling of the Bayer CFA image as shown in Figure 1, each of the LR image is obtained according to the sampling position of each color channel, given as

(2)

where represent the Bayer CFA samples at pixel index and the LR image channel is green, red, blue, and green channels according to the sampling index , respectively. Therefore, we obtain four LR images , and each of them has full spatial resolution in LR grid as shown in Figure 1(b). Using the defined LR images, the Bayer color difference plane is defined as the difference between the LR images,

(3)

where is the Bayer color difference plane given the different Bayer LR images, . Note that, the correlation between the sampling positions are simultaneously considered with the inter channel correlation in (3).

To describe the local property of , we consider the directional components of LR images. When we use the undecimated wavelet transform, a LR image can be decomposed into low-frequency, horizontal, vertical directional and the residual high frequency components [13]. As shown in Figure 2, the two-staged directional low-pass and the high-pass filters, and , respectively, make the low-pass and directionally high-pass filtered images. Given the directional forward filter banks, a Bayer LR image is represented as the sum of four frequency components, such as,

(4)

where the upper letters represent the low frequency, vertical and horizontal directional high frequencies, and the residual components of , respectively, and they are described as , and . In (4), it is assumed that the most of the high-frequencies of an image is concentrated on the vertical and horizontal directional components, so that the residual parts are not considered in the following discussion. Also, the directional high frequency components are assumed to be exclusively separated in the horizontal and vertical directions, since an image has strong directional correlation along the sharp edges. Therefore, (or ) is approximately zero in the vertical (or horizontal) sharp edge region in (4). Based on these assumptions, the Bayer color difference plane in (3) is reorganized as follows,

(5)

where represents the spectral correlation between the Bayer LR images [7], and indicates the LR image shift direction where the value 1 for represents no shift, and 0 for represents the shift toward the direction. Note that, the horizontal (or vertical) directional frequency components are paired with the vertical (or horizontal) directional shifting indicator. The cross-directional pair of shift indicator and the directional frequencies shows the relation between the global LR image shifting direction and the local edge direction: the Bayer color difference is highly correlated in a local region when the global shift and the local edge directions are corresponded to each other. We call it as the spatial correlation of the Bayer color difference.

Figure 2

Undecimated 2D wavelet transform with filter banks and spectral components of G₀₀.

In Figure 3, a vertical edge region is shown as an example of the relation between the global and the local directions. When the vertical region in the local region of Bayer pattern in Figure 3(a) is down sampled, the corresponding LR images in Figure 3(b) show different edge locations according to the sampling location. When the global shift direction coincides with the vertical local direction, Bayer LR images show similar edge location. Otherwise, the edges in each image are dislocated. From (5), the Bayer color difference planes that is obtained by and horizontally and vertically shifted images and , respectively, are given as follows:

(6)

In (6), the difference of vertical high frequency components are remained in the difference of horizontally shifted LR images, while they are disappeared in the difference of vertically shifted LR images. In the real images, the spatial correlation on the Bayer color difference plane can be shown as depicted in Figure 4. In the strong vertical edge region in Figure 4(a), the difference plane obtained from the vertically shifted LR images is smooth planes, while the difference obtained from the horizontally shifted images shows overstated details. In the edge pattern region in Figure 4(b), the aliasing effect of the LR images makes pattern in the difference plane from the horizontally shifted images. However, the aliasing effects are disappeared in the difference plane of the opposite case. From these examples, the strong connection of the global shift direction and the local edge direction is described by the spatial correlation of Bayer color difference. In the following section, we describe the detailed method to use the spatial correlation of the Bayer color difference in the edge direction estimation and the region classification.

Figure 3

Vertical edge region of (a) Bayer CFA samples, (b) Bayer LR images, and (c) the Bayer color difference planes.

Figure 4

Examples of the Bayer color difference planes of R ₀₁ and G ₀₀ and R ₀₁ and G ₁₁ (a) edge and flat regions (b) vertical edge pattern region.

3. Proposed Edge Directed Color Demosaicking Algorithm Using Region Classifier

In the proposed edge adaptive demosaicking method, the edge directions are optimally estimated according to the region type. Based on the spatial correlation of the Bayer color difference, the proposed method classifies an image into three regions, such as edge, edge pattern, and flat regions. In each of the regions, we classify the edge direction type () as the horizontal () or vertical () direction. When the direction is not obviously determined, we decide the direction as nondirectional (). Therefore, the final types of the edge direction are . In the proposed edge direction estimation, the diagonal directional edge is considered as the combination of the horizontal and vertical directional edges. According to the determined edge direction, the missing pixels are interpolated with weighting functions. Following the edge types and the edge directions, we present the way to classify the region and to estimate the edge direction based on the spatial correlation on the Bayer color difference plane. To utilize the correlation, we describe the details of the interpolation process as the restoration of missing channels of LR images. Given the obtained LR images in Figure 1(b), the missing channels of each LR color images are . By considering the sampling rate of the green channel, the proposed method first interpolates the missing green channels, than the red and blue channels are interpolated by using the fully interpolated green channel images. This is helpful to improve the red and blue channel interpolation quality, since the green channel has more edge information than the red and blue channels. Since the Bayer LR images are shifted to each other, they are interpolated in the same way for each channel. Once all of the missing channels are reconstructed at each sampling position, the full-color LR images are upsampled and they are registered according to the original position in the HR grid. The overall process of the proposed adaptive demosaicking method is depicted in Figure 6, where the process is composed of estimating Bayer color difference plane, the region classification, the edge direction estimation, and the directional interpolation for each green and red/blue channel interpolation. In the following subsections, the way of interpolating the missing pixels in and are described as a representative of green and red(blue) channel interpolations.

3.1. Green Channel Interpolation

3.1.1. Region Classification: Sharp Edges

In the proposed demosaicking method, the modified notation for the sampling index is used to emphasize the relation between the global shift direction and local edge direction in LR images. When we consider the interpolation of the missing green channel of position, we set the red pixel position as the center position, that is,

(7)

According to the center position, the four neighborhood positions are defined as

(8)

where represents the position of the pixels in the LR images in the north, south, east, and west from the center position. Note that the notation inherits the relative pixel position in Bayer CFA samples from the center pixel position.

Using the modified notation, the Bayer color difference in (3) is defined as

(9)

where . From the spatial correlation on the Bayer color difference plane in (5), is highly correlated in the local region when the shifting direction coincides with the local edge direction. As an estimator for the spatial correlation, the local variations of the difference is estimated, such as

(10)

where . In Figure 5, the window mask on the Bayer pattern and the corresponding Bayer color difference planes are described. When the local variations of each position are determined, the maximum and the minimum variations of horizontal shifting direction are defined as:

(11)

Also, and are determined as the same way in (11) by changing to . The edge direction is clearly determined owing to the group with smaller variations, since the maximum of local variations along the edge direction is smaller than the minimum of local variations across the edge direction in the strong edge region.

Figure 5

A 7 × 7 window of Bayer CFA pattern and its four neighboring Bayer color difference planes for local variation criterion.

Figure 6

Flowchart of the proposed edge adaptive color demosaicking algorithm.

In addition, the spatial similarity between the green channels is estimated for the restrict decision of the edge direction. Defining the difference plane of green channel,

(12)

where is a pair of the horizontally or vertically located LR image positions. By applying the discussions in (5), the spatial correlation of is estimated by the local similarity for the horizontal and the vertical directions, such as,

(13)

where and represent the local average of the differences between the horizontally and vertically shifted green images, respectively. The local similarity becomes small when the global shift and the local edge directions are coincided.

With the measured local variation and local similarity criteria, the of each pixel is determined by,

Classification 1

Sharp edge region

(14)

where and represents the sharp edges along horizontal or vertical directions, respectively. When the direction is not determined, the region is considered as a nonsharp edge region and these regions are investigated again in the following region classification step: Classification 2.

3.1.2. Region Classification: Edge Patterns

The regions of which edge types are not determined in (14) belong to the flat or the edge pattern region. The edge pattern region represents the region in the HR image that contains high-frequency components above the Nyquist rate of the Bayer CFA sampling. When the image is down sampled, the high frequency components that exceed the sampling rate are contaminated due to the aliasing effect. Therefore, the edge pattern region appears as locally flat in the LR image as shown in Figure 4(b). In this section, we derive the detection rule for the edge pattern region (pseudoflat region in the LR grid) and estimate the edge direction of the edge pattern.

To distinguish the pseudoflat region from the flat region, we use the characteristics of aliasing effect in the LR images. As shown in Figure 4(b), the fence region of and are flat for each images. This phenomenon is caused by the CFA sampling above the Nyquist rate in these regions and the high frequencies in HR image is blended into the low frequency by the down sampling. However, they are not the same flat when we compare the intensity of them at the same pixel location since the frequency blending cannot contaminate the intensity offset between the adjacent edges. Therefore, we use two criteria to classify the pseudoflat region from the normal flat region: the intensity offset and the smoothness restriction. The intensity offset is estimated by

(15)

where is the difference between averages of the horizontally and vertically located LR images, and represents the low frequency of at pixel location. In addition to intensity offset, we restrict the condition with the pixel smoothness in respective LR images. Since we deal with the flat (and also the pseudoflat) region, the local variation values, which mean the fluctuation on each of the difference images, should be similar to each other. The similarity between the local variation values is estimated by the standard deviation of the local variations, given by:

(16)

where is a variation of and is the average of local variations.

With the intensity offset and the restrictive condition, the pseudoflat region (edge pattern region) is classified from the nonsharp edge region, such as

Classification 2

Edge pattern or Flat region

(17)

where and represent that the region is determined as the edge pattern region and a flat region in this classification, respectively, and and are thresholds that control the accuracy of the classification. If is larger (and is smaller) than the threshold, the pixel at is considered as being in the edge pattern region and the direction of the edge pattern is determined by the following criteria.

For pixels classified into the edge pattern region, the pattern edge direction is estimated using the modified local variation values in (10) with the extended range . The edge direction of the edge pattern region is estimated as

(18)

where and represent that the edge pattern is horizontally or vertically directed, respectively, and represents the region of which the edge direction is not clearly determined. Once the edge type of the edge pattern region is determined, the statistics of neighboring edge directions, such as the horizontal or vertical direction, are compared within a neighborhood. Following the majority of the directions, the consistency of the edge directions in the region is improved.

3.1.3. Edge Directed Interpolation

After the edge types of all pixels are categorized with the classified region types, edge directed interpolation is performed. If the edge types are clearly determined as or , the missing pixels are interpolated toward the direction. When the edge direction is determined as , it is considered as the flat region or the region where the edge direction is not defined. In this case, the missing pixels are interpolated by the weighted average of neighboring pixels. Therefore, the missing green channel LR image is interpolated according to the edge types, such as,

(19)

where represent a weight function, and is a color difference domain value obtained from four green LR image locations. The weighting function used in the interpolation process is a reciprocal of gradient magnitude values [10]:

(20)

where , and represent the gradients of the pixels in the center image, in the LR images that are shifted corresponding to the considering direction , and in the other LR images, respectively. For example, the weighting function in the north direction is calculated from , , and . The values of each LR image are obtained as followed by using the definition of the difference between the red and green channels [7]:

(21)

where is for , respectively.

3.2. Red and Blue Channel Interpolation

Similar to the green plane interpolation, the missing red and blue channel LR images are interpolated along the edge direction by the region classification and the edge direction estimation. The fully interpolated green channels which have much information on edges are utilized to improve interpolation accuracy of the red and blue channels. To compensate insufficient LR images, the diagonally shifted LR images of are estimated using linear interpolation on the color difference domain [7]. In this section, the missing red and blue channels are found in aid of the sampled images and the interpolated images .

To interpolate the red LR image in sampling position, is used as the center image, thatis, , and the four neighboring red and green images at each side are used. The red and green images at each sampling position are defined as and where , respectively, and for each position is defined as follows:

(22)

Considering the four neighboring red and green images of , the local variation and local similarity criteria are estimated as the same way in (10) and (13) by using the newly defined . When the edge direction is estimated by (14) and (17) with the process of region classification, is directionally interpolated, given as:

(23)

where . The weight function is computed as the same way in (20), but the gradient values are calculated in the green LR images.

4. Experimental Results

To study performance experimentally, the proposed and other existing algorithms were tested with Kodak PhothCD image set and Bayer CFA raw data shown in Figure 7. For comparison, three groups of conventional methods were implemented: nonedge directed (nonED) methods proposed by Pei and Tam [7], by Gunturk et al. [13], and by Zhang and Wu [14], the indirect edge directed (indirect ED) methods such as primary-consistency soft-decision (PCSD) method [20], the homogeneity-directed method [21], and the a posteriori decision method [22], and the direct edge directed (direct ED) methods such as the variance of color differences method [23], and the adaptive heterogeneity-projection method [25]. They were implemented following the parameters given in each paper or using the provided source code [14]. Also, we implemented each of the methods without the refining step [21–23, 25] so that the performances of the methods were compared fairly.

Figure 7

(a) Kodak PhotoCD image set and (b) Bayer raw data.

The peak signal-to-noise ratio (PSNR) and the normalized color difference (NCD) were used for quantitative measurement. The PSNR is defined in decibels as , where MSE represents the mean squared error between the original and the resultant images. The NCD is an objective measurement of the perceptual errors between the original and the demosaicked color images [11]. This value is computed by using the ratio of the perceptual color errors to the magnitude of the pixel vector of the original image in the CIE Lab color space. A smaller NCD value represents that a given image is interpolated with a reduced color artifact. In Tables 1 and 2, PSNR and NCD values of each algorithm were compared. Among the conventional methods, nonED methods, such as DLLMMSE [14] and POCS [2], show high performance in terms of the numerical values. Also, the recent edge directed techniques [21–23, 25] show high PSNR and NCD performance among the conventional edge directed techniques, especially in the images with fine texture patterns, such as Kodak 5, 6, 8, 15, and 19. The proposed method outperforms the conventional edge directed methods in the majority of the images including those challenging images with  dB and improvements of the averaged PSNR and NCD values, respectively.

Table 1 The PSNR comparison of the conventional and proposed methods using the average of the three channels (dB) on the 24 test images in Figure 7(a).

Table 2 The NCD comparison of the conventional and proposed methods on the 24 test images in Figure 7(a).

To show the performance of each methods in edge patterns and edge junctions, the resulting images are shown in Figures 8–11 that contain fine textures of Kodak 19, 15 and real images, respectively. At first, the competitive regions of Kodak 19 are shown in Figure 8. In each of the image crop, the vertically directed line edge pattern of the fence and the edge junctions of the window are depicted. In spite of the high PSNR performance, POCS method shows the Moiré pattern and the zipper artifacts in Figure 8(c). In Zhang's method and the edge directed methods in Figures 8(d)–8(i), the fence regions are highly improved with reduced errors. However, visible artifacts were remained on the vertical edges of the high frequency region or boundaries between the fence and the grass. Moreover, the zippers and disconnection were shown in the edge junctions in the upper image crop in Figures 8(b)–8(i). In Figure 8(j), the resultant image of the proposed algorithm shows better results in terms of the clear edges and the reduced visible artifacts. The resultants of the methods in the textures with diagonal patterns or diagonal lines are shown in Figure 9. While the artifacts were produced along the ribbon boundary in Figures 9(b)–9(i), the proposed method produced consistent edges with accurate edge direction estimation.

Figure 8

The partially magnified images of Kodak 19 from (a) the original image, and from the results of (b) Pei [ 7 ], (c) the POCS [ 13 ] (d) the directional LMMSE [ 14 ], (e) the PCSD [ 20 ], (f) the homogeneity-directed [ 21 ], (g) a posteriori decision [ 22 ], (h) the variance of color differences [ 23 ], (i) the adaptive heterogeneity-projection [ 25 ], and (j) the proposed method.

Figure 9

The partially magnified images of Kodak 15 from (a) the original image, and from the results of (b) Pei [ 7 ], (c) the POCS [ 13 ] (d) the directional LMMSE [ 14 ] (e) the PCSD [ 20 ], (f) the homogeneity-directed [ 21 ], (g) a posteriori decision [ 22 ], (h) the variance of color differences [ 23 ], (i) the adaptive heterogeneity-projection [ 25 ], and (j) the proposed method.

Figure 10

The results of Bayer CFA raw data 1 of (a) Pei [ 7 ], (b) the POCS [ 13 ], (c) the directional LMMSE [ 14 ], (d) the PCSD [ 20 ], (e) the homogeneity-directed [ 21 ], (f) a posteriori decision [ 22 ], (g) the variance of color differences [ 23 ], (h) the adaptive heterogeneity-projection [ 25 ], and (i) the proposed method.

Figure 11

The results of Bayer CFA raw data 2 of (a) Pei [ 7 ], (b) the POCS [ 13 ], (c) the directional LMMSE [ 14 ], (d) the PCSD [ 20 ], (e) the homogeneity-directed [ 21 ], (f) a posteriori decision [ 22 ], (g) the variance of color differences [ 23 ], (h) the adaptive heterogeneity-projection [ 25 ], and (i) the proposed method.

By using the high-resolution 12-bit Bayer CFA raw data in Figure 7(b), we can demonstrate the performance of each algorithm in the presents of noise. In Figures 10 and 11, the resultant images are shown with the region which contains edge junctions. In these regions, most of the algorithms show zipper artifacts caused by the false estimation of the edge direction. Among the conventional methods, edge directed techniques such as the variance of color differences method and the adaptive heterogeneity-projection method in Figures 10(g) and 10(h) demonstrates good performance on the horizontal and vertical directional edges. Similar results are shown in the diagonal edges in Figures 11(g) and 11(h). However, some artifacts are remained in the edge direction changing regions. In the resultants of the proposed method in Figures 10(i) and 11(i), the interpolated pixels are consistent along the edge and this shows the robustness of the spatial correlation of the Bayer color difference based method.

To show the computational requirements, the averaged run times of 24 images from Kodak PhotoCD image set for each algorithm are calculated in Table 3. The experiments were performed on a PC equipped with an Intel Core2 Duo E8400 CPU. In the table, the processing time is increased depending on the estimation criterion: for example, preinterpolation before estimation and a posteriori decision [22] or the adaptive range of neighborhood for gradient calculation [23] needed more time than the simple estimation [7]. The proposed method consumed more time than these methods due to the multiple steps of the edge oriented region classifier. However, it consumed less time than the homogeneity-directed method [21], minimum mean square error-based interpolation method [14], and the adaptive heterogeneity-projection method [25] while the image qualities were highly improved.

Table 3 Computational complexity comparison for the presented color demosaicing methods (measured in seconds on an Intel Core2 Duo E8400 processor).

5. Conclusion

In this paper, we have proposed the edge adaptive color demosaicking algorithm that effectively estimates the edge direction on the Bayer CFA samples. We examined the spatial correlation on the Bayer color difference plane, and proposed the criteria for the region classification and the edge direction estimation. To estimate the edge direction in the complicated edge regions, the proposed method classified regions of an image into three types: edge, edge pattern, and flat regions. According to the edge types, the edge direction were effectively estimated and the directional interpolation resulted in clear edge. The proposed edge adaptive demosaicking method improved the overall image quality in terms of consistent edge directions around the edges. The proposed method was compared with the conventional edge directed and nonedge directed methods on the several images including the Bayer raw data. The simulation results indicated that the proposed method outperforms conventional edge directed algorithms with respect to both objective and subjective criteria.