Integer fast lapped transforms based on direct-lifting of DCTs for lossy-to-lossless image coding

Abstract

The discrete cosine transforms (DCTs) have found wide applications in image/video compression (image coding). DCT-based lapped transforms (LTs), called fast LTs (FLTs), overcome blocking artifacts generated at low bit rate image coding by DCT while keeping fast implementation. This paper presents a realization of more effective integer FLT (IntFLT) for lossy-to-lossless image coding, which is unified lossless and lossy image coding, than the conventional IntFLTs. It is composed of few operations and direct application of DCTs to lifting blocks, called direct-lifting of DCTs. Since the direct-lifting can reuse any existing software/hardware for DCTs, the proposed IntFLTs have a great potential for fast implementation which is dependent on the architecture design and DCT algorithms. Furthermore, the proposed IntFLTs do not need any side information unlike integer DCT (IntDCT) based on direct-lifting as our previous work. Moreover, they can be easily extended to larger size which is recently required as in DCT for the standard H.26x series. As a result, the proposed method shows better lossy-to-lossless image coding than the conventional IntFLTs.

1 Introduction

The most popular image/video compression (image coding) standards, JPEG [1, 2] and H.26x series [3, 4], employ discrete cosine transform (DCT) [5] at their transformation stages. DCT can be basically classified into types I to IV (DCT-I to IV) and has numerous fast implementations [6–10] and applications for signal processing. In them, DCT-II, so-called DCT, has excellent energy compaction capability and DCT-III is its inverse transform, so-called inverse DCT (IDCT). However, DCT generates annoying blocking artifacts at low bit rates because the DCT bases are short and create discontinuities at block boundaries due to non-overlapping. To overcome this drawback, lapped transforms (LTs), which are classified into lapped orthogonal transform (LOT) and lapped biorthogonal transform (LBT), have received much attention. DCT-based fast LTs (FLTs), which are classified into fast LOT (FLOT) and fast LBT (FLBT), are well-known as fast and effective transform for image coding [11]. FLTs are constructed by cascading DCT-II, DCT-III, DCT-IV, rotation matrices with π/4 angles, ±1 operations, scaling factors, a delay matrix, and permutation matrices. To improve the coding performance and reduce the complexity more, LiftLT with VLSI-friendly implementation has been proposed by Tran [12] ^a. However, the LTs cannot be applied to the lossless mode.

On the other hand, JPEG achieves the lossless mode by using differential pulse code modulation (DPCM) in place of DCT. JPEG 2000 [13] employs 9/7-tap and 5/3-tap discrete wavelet transforms (9/7-DWT and 5/3-DWT) for lossy and lossless modes, respectively [14]. They mean that JPEG and JPEG 2000 do not have compatibility between the lossy and lossless mode. Of course, lossless transform such as 5/3-DWT is applicable to lossy-to-lossless image coding. However, its lossy performance is not good compared with 9/7-DWT because each transform is suitable only in each mode. The next standard JPEG XR [15] has solved this problem by achieving lossy-to-lossless image coding which is unified lossy and lossless image coding. JPEG XR employs only hierarchical lapped transform (HLT) for both of lossy and lossless modes [16]. The HLT is composed of lifting structures [17–19] with rounding operations and achieves integer-to-integer transform, whereas it does not have enough coding performance, especially for images with many high frequency components. Various lifting-based filter banks (L-FBs) [20–28], which contain integer DCTs (IntDCTs) [29–35], have been researched to improve coding performance. However, these except for IntDCTs are not practical due to the complexity.

This paper presents a realization of integer FLT (IntFLT), which is constructed by lifting structures with rounding operations, for lossy-to-lossless image coding. Although FLT can be easily applied to lossy-to-lossless image coding by simple lifting factorizations of rotation matrices and scaling factors, the obtained integer transform is unsuitable due to large rounding error because of many rounding operations. The conventional IntFLTs also have many operations, whereas the proposed IntFLTs have simple implementations with few operations and direct application of DCTs to lifting blocks, called direct-lifting of DCTs. The direct-lifting can reuse any existing software/hardware for DCTs^b. As a result, although the proposed IntFLTs are apparently sacrificing the complexity to achieve the lossless mode compared with LiftLT, they have a great potential for fast implementation which is dependent on the architecture design and DCT algorithms. Furthermore, the proposed IntFLTs do not need any side information unlike IntDCT based on direct-lifting as our previous work [35]. Moreover, they can be easily extended to larger size which is recently required as in DCT for H.26x series. Such IntFLT already proposed in [36] cannot achieve enough coding performance due to the orthogonality. This paper introduces IntFLT without such a restriction. Finally, the proposed method shows better lossy-to-lossless image coding than the conventional IntFLTs.

1.1 Notations

Several special matrices with reserved symbols are as follows: I, J, 0, and D are an identity matrix, a reversal identity matrix, a null matrix, and a diagonal matrix with alternating ±1 entries (i.e., diag{1,-1,1,-1,⋯ }), respectively. Also, ·^T and ·^-1 are transpose and inverse of a matrix, respectively.

2 Review

2.1 Fast lapped transform (FLT)

An M-channel (M = 2^k, $k\in \mathbb{N}$) FLT can be constructed in polyphase structure from components with well-known fast-computable algorithms. One of the most elegant solution is the type-II FLOT. The polyphase matrix E(z) is expressed as [11]

$$\begin{array}{l}\mathbf{E}\left(z\right)=\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& {\mathbf{S}}_{IV}{\mathbf{C}}_{III}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& {\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(1)

where

$$\begin{array}{l}\mathbf{W}=\frac{1}{\sqrt{2}}\left[\begin{array}{cc}\mathbf{I}& \mathbf{I}\\ \mathbf{I}& -\mathbf{I}\end{array}\right],\tilde{\mathbf{I}}=\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{J}\end{array}\right],\mathbf{\Lambda }\left(z\right)=\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& z^{-1}\mathbf{I}\end{array}\right],\end{array}$$

z^-1 is a delay, and C _II , C _III , C _IV , and S _IV are DCT-II, DCT-III, DCT-IV, and type-IV discrete sine transform (DST-IV) matrices whose (m, n)-elements are presented by

$$\begin{array}{ll}{\left[{\mathbf{C}}_{II}\right]}_{m,n}=& \sqrt{\frac{2}{N}}{c}_{m}cos\left(\frac{m\left(n+1/2\right)\pi }{N}\right)\\ {\left[{\mathbf{C}}_{III}\right]}_{m,n}=& \sqrt{\frac{2}{N}}{c}_{n}cos\left(\frac{\left(m+1/2\right)\mathrm{n\pi }}{N}\right)\\ {\left[{\mathbf{C}}_{IV}\right]}_{m,n}=& \sqrt{\frac{2}{N}}cos\left(\frac{\left(m+1/2\right)\left(n+1/2\right)\pi }{N}\right)\\ {\left[{\mathbf{S}}_{IV}\right]}_{m,n}=& \sqrt{\frac{2}{N}}sin\left(\frac{\left(m+1/2\right)\left(n+1/2\right)\pi }{N}\right)\end{array}$$

where $c_i=1/\sqrt{2}$ (i = 0) or 1 (i ≠ 0). Also, ${\mathbf{C}}_{II}^{-1}={\mathbf{C}}_{II}^T={\mathbf{C}}_{III}$, ${\mathbf{C}}_{IV}^{-1}={\mathbf{C}}_{IV}^T={\mathbf{C}}_{IV}$, and ${\mathbf{S}}_{IV}^{-1}={\mathbf{S}}_{IV}^T={\mathbf{S}}_{IV}$. Since the following relationship between DST-IV and DCT-IV matrices can be established: S _IV  = DC _IV J, Equation 1 can be easily represented by

$$\begin{array}{l}\mathbf{E}\left(z\right)=\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& {\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}.\end{array}$$

(2)

On the other hand, the HLT for JPEG XR is based on FLOT with scaling factors [16]. By inspiring it, the FLT in this paper is defined by

$$\begin{array}{l}\mathbf{E}\left(z\right)=\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\left[\begin{array}{cc}{s}_0{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(3)

where $s_1=s_0^{-1}$ which is the restriction for lifting factorization. This is called FLBT in this paper. Since FLOT in Equation 2 is understandably equal to FLBT in Equation 3 with s ₀ = s ₁ = 1, we use this equation (3) as a representative expression of FLT. The FLT with this polyphase matrix is implemented as shown in the top half in Figure 1.

Figure 1

Parallel process of two different type M -channel FLTs (drawn for M = 8 ). (Top) FLT in Equation 3; (bottom) FLT in Equation 6.

2.2 Direct-lifting structure

In [35], we have presented direct-lifting which is a class of block-lifting [25] known as a more effective lifting structure for lossy-to-lossless image coding than standard lifting structure [17–19]. The block-lifting reduces rounding error by merging many rounding operations. The direct-lifting is a key technology to produce novel IntFLTs. To achieve the lifting, we suppose a processing of two individual M × 1 signals x _i and x _j by an M × M arbitrary nonsingular matrix T and its inverse transform matrix T^-1, respectively, as shown at the left side of Figure 2. The input signals x _i and x _j are simultaneously transformed to the output signals y _i and y _j by T and T^-1 as

$$\begin{array}{l}\left[\begin{array}{c}{\mathbf{y}}_i\\ {\mathbf{y}}_j\end{array}\right]=\left[\begin{array}{cc}\mathbf{T}& \mathbf{0}\\ \mathbf{0}& {\mathbf{T}}^{-1}\end{array}\right]\left[\begin{array}{c}{\mathbf{x}}_i\\ {\mathbf{x}}_j\end{array}\right].\end{array}$$

Figure 2

Direct-lifting structure (white circles represent rounding operations).

This block diagonal matrix diag{T, T^-1} can be factorized into complete block-liftings such as

$$\begin{array}{l}\left[\begin{array}{cc}\mathbf{T}& \mathbf{0}\\ \mathbf{0}& {\mathbf{T}}^{-1}\end{array}\right]=\left[\begin{array}{cc}\mathbf{0}& \mathbf{I}\\ -\mathbf{I}& \mathbf{0}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{T}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -{\mathbf{T}}^{-1}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{T}& \mathbf{I}\end{array}\right].\end{array}$$

(4)

Thus, the parallel block system of T and T^-1 can be efficiently implemented by the block-liftings as shown at the right side of Figure 2. This is a breakthrough structure because any block T and its inverse one T^-1 can be directly applied to the block-lifting coefficients without breaking their forms. Although any existing software/hardware for DCT cannot be directly reused for the conventional IntDCTs, we can admit any of them as the lifting blocks when T = C _II .

3 IntFLTs based on direct-lifting of DCTs

This section presents a realization of IntFLT for lossy-to-lossless image coding. The IntFLTs have simple implementations with few operations and direct-lifting of DCTs.

3.1 Direct-lifting of DCTs

FLT in Equation 3 is transferred into the another type so that direct-lifting (4) can be applied. First, Equation 3 is rewritten as

$$\begin{array}{ll}\mathbf{E}\left(z\right)=& \left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& {\mathbf{C}}_{II}\end{array}\right]\\ \times \left[\begin{array}{cc}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{III}{\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(5)

where C _II C _III  = I. By moving diag{C _II , C _II } to the postprocessing part, Equation 5 is rewritten as

$$\begin{array}{ll}\mathbf{E}\left(z\right)& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& {\mathbf{C}}_{II}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\\ \times \left[\begin{array}{cc}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{III}{\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}\\ =\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}\end{array}\right]\mathbf{W}\mathbf{\Lambda }\left(z\right)\mathbf{W}\left[\begin{array}{cc}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{III}{\mathbf{C}}_{IV}\end{array}\right]\mathbf{W}\tilde{\mathbf{I}}\mathbf{J}\triangleq \tilde{\mathbf{E}}\left(z\right)\end{array}$$

(6)

where C _III C _II  = I and $\tilde{\mathbf{E}}\left(z\right)$ are used to distinguish from the original E(z) in Equation 3. Of course, $\tilde{\mathbf{E}}\left(z\right)$ is the same transfer function as E(z). The FLT with this polyphase matrix $\tilde{\mathbf{E}}\left(z\right)$ is implemented as shown at the bottom half in Figure 1.

Next, as already mentioned, we consider the parallel process of two different type FLTs in Equations 3 and 6 as follows:

$$\begin{array}{l}\left[\begin{array}{c}{\mathbf{y}}_i\\ {\mathbf{y}}_j\end{array}\right]=\left[\begin{array}{cc}\mathbf{E}\left(z\right)& \mathbf{0}\\ \mathbf{0}& \tilde{\mathbf{E}}\left(z\right)\end{array}\right]\left[\begin{array}{c}{\mathbf{x}}_i\\ {\mathbf{x}}_j\end{array}\right]\end{array}$$

where x _i and x _j are individual input signals along process direction, and y _i and y _j are their output signals as shown in Figure 1. It means that when a row (column) signals are processed by Equation 3, other row (column) signals are processed by Equation 6. However, each DCT matrix in both FLT is processed by direct-lifting of each combination of DCT-II/DCT-III, DCT-III/DCT-II, and DCT-IV/DCT-IV as shown in dashed line box in Figure 1. For example, the combination of DCT-II/DCT-III is factorized as

$$\begin{array}{ll}\left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& {\mathbf{C}}_{III}\end{array}\right]& =\left[\begin{array}{cc}\mathbf{0}& \mathbf{I}\\ -\mathbf{I}& \mathbf{0}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ {\mathbf{C}}_{II}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -{\mathbf{C}}_{III}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ {\mathbf{C}}_{II}& \mathbf{I}\end{array}\right]\end{array}$$

by substituting it into Equation 4.

3.2 Lifting structure of rotation matrix with π/4 angle

Since W in Equations 3 and 6 and Figure 1 includes the scaling factor $1/\sqrt{2}$, we factorize this into lifting structure. In [14], W is simply factorized as

$$\begin{array}{l}\mathbf{W}=\left[\begin{array}{cc}\mathbf{I}& w_0\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ w_1\mathbf{I}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& w_0\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}\end{array}\right]\end{array}$$

where $w_0=1-\sqrt{2}$ and $w_1=1/\sqrt{2}$. But this factorization includes many floating-point multipliers. To eliminate as much multipliers as possible, the following scaled matrices are used in place of pure W.

$$\begin{array}{ll}\left[\begin{array}{cc}\frac{1}{\sqrt{2}}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \sqrt{2}\mathbf{I}\end{array}\right]\mathbf{W}& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \frac{1}{2}\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ -\mathbf{I}& \mathbf{I}\end{array}\right]\triangleq {\mathbf{W}}_1\\ \mathbf{W}\left[\begin{array}{cc}\sqrt{2}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \frac{1}{\sqrt{2}}\mathbf{I}\end{array}\right]& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{I}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -\frac{1}{2}\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}\end{array}\right]\triangleq {\mathbf{W}}_2\\ \left[\begin{array}{cc}\sqrt{2}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \frac{1}{\sqrt{2}}\mathbf{I}\end{array}\right]\mathbf{W}& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \frac{1}{2}\mathbf{I}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}\end{array}\right]\triangleq {\mathbf{W}}_3\\ \mathbf{W}\left[\begin{array}{cc}\frac{1}{\sqrt{2}}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \sqrt{2}\mathbf{I}\end{array}\right]& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ -\frac{1}{2}\mathbf{I}& \mathbf{I}\end{array}\right]\triangleq {\mathbf{W}}_4\end{array}$$

Note that a lifting structure with coefficient 1/2 and rounding operation can be replaced by one adder and one bit-shifter [37], i.e., multiplierless operations. With these matrices, Equations 3 and (6) are represented as follows:

$$\begin{array}{ll}\left[\begin{array}{cc}\frac{1}{\sqrt{2}}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \sqrt{2}\mathbf{I}\end{array}\right]\mathbf{E}\left(z\right)=& \left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]{\mathbf{W}}_1\mathrm{\Lambda }\left(z\right){\mathbf{W}}_2\\ \times \left[\begin{array}{cc}{s}_0{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{IV}\end{array}\right]{\mathbf{W}}_1\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(7)

$$\begin{array}{ll}\left[\begin{array}{cc}\sqrt{2}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \frac{1}{\sqrt{2}}\mathbf{I}\end{array}\right]\tilde{\mathbf{E}}\left(z\right)=& \left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}\end{array}\right]{\mathbf{W}}_3\mathbf{\Lambda }\left(z\right){\mathbf{W}}_2\\ \times \left[\begin{array}{cc}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{III}{\mathbf{C}}_{IV}\end{array}\right]{\mathbf{W}}_1\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(8)

Figure 3 shows the process that an image is two-dimensionally transformed by the proposed IntFLT. The i th (1 ≤ ((i + 1) mod M) ≤ M / 2) row signals and the j th ((M / 2 + 1) ≤ ((j + 1) mod M) ≤ M) row signals, i.e., the yellow and green areas in Figure 3, are processed by FLTs in Equations 7 and 8, respectively. Here, note that the one-dimensionally transformed output signals are scaled by $1/\sqrt{2}$ and $\sqrt{2}$ as compared with the output signals transformed by normal FLTs as shown in the dashed line box in Figure 3. By considering these scales $1/\sqrt{2}$ and $\sqrt{2}$ for the next column process, Equations 3 and 6 are represented again as follows:

$$\begin{array}{ll}\mathbf{E}\left(z\right)\left[\begin{array}{cc}\sqrt{2}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \frac{1}{\sqrt{2}}\mathbf{I}\end{array}\right]=& \left[\begin{array}{cc}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}{\mathbf{C}}_{III}\end{array}\right]{\mathbf{W}}_2\mathrm{\Lambda }\left(z\right){\mathbf{W}}_1\\ \times \left[\begin{array}{cc}{s}_0{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{IV}\end{array}\right]{\mathbf{W}}_2\tilde{\mathbf{I}}\mathbf{J}\end{array}$$

(9)

Figure 3

Process of the two-dimensional transform of an image by the proposed IntFLTs.

$$\begin{array}{ll}\tilde{\mathbf{E}}\left(z\right)\left[\begin{array}{cc}\frac{1}{\sqrt{2}}\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \sqrt{2}\mathbf{I}\end{array}\right]=& \left[\begin{array}{cc}{\mathbf{C}}_{II}& \mathbf{0}\\ \mathbf{0}& \mathbf{D}{\mathbf{C}}_{IV}\mathbf{J}\end{array}\right]{\mathbf{W}}_4\mathrm{\Lambda }\left(z\right){\mathbf{W}}_3\\ \times \left[\begin{array}{cc}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1{\mathbf{C}}_{III}{\mathbf{C}}_{IV}\end{array}\right]{\mathbf{W}}_4\tilde{\mathbf{I}}\mathbf{J}.\end{array}$$

(10)

Similarly, the i th column signals and the j th column signals, i.e., the red and blue areas in Figure 3, are processed by FLTs in Equations 9 and 10, respectively. Consequently, the scales are changed temporarily for fast implementation and restored after two-dimensional transform.

3.3 Lifting structure of scaling part

In this subsection, we present lifting structures of each scaling part diag{s ₀ I, s ₁ I} including in Equations 7 to 10. According to Equation 4, we define a simple realization of integer transform in the scaling part as follows:

$$\begin{array}{l}\left[\begin{array}{ll}{s}_0\mathbf{I}& \mathbf{0}\\ \mathbf{0}& s_1\mathbf{I}\end{array}\right]=\left[\begin{array}{ll}\mathbf{0}& \mathbf{I}\\ -\mathbf{I}& \mathbf{0}\end{array}\right]\left[\begin{array}{ll}\mathbf{I}& \mathbf{0}\\ s_0\mathbf{I}& \mathbf{I}\end{array}\right]\left[\begin{array}{ll}\mathbf{I}& -s_1\mathbf{I}\\ \mathbf{0}& \mathbf{I}\end{array}\right]\left[\begin{array}{ll}\mathbf{I}& \mathbf{0}\\ s_0\mathbf{I}& \mathbf{I}\end{array}\right]\end{array}$$

where $s_1=s_0^{-1}$. The lifting coefficients s ₀ and s ₁ in the scaling part are empirically determined.

4 Results

4.1 Coding gain

This paper designed 8 × 16 and 16 × 32 IntFLTs. First, the comparison of coding gain of the ideal FLTs and the proposed IntFLTs is shown.

The coding gain is one of the most important factors to be considered in compression applications. A transform with higher coding gain compacts more energy into a fewer number of coefficients. As a result, higher objective performance such as PSNR would be achieved after quantization. The biorthogonal coding gain is defined as [38]

$$\begin{array}{l}\text{Coding gain}\left[\text{dB}\right]=10{log}_{10}\frac{{\sigma }_x^2}{\prod _{k=0}^{M-1}{\sigma }_{x_k}^2\parallel f_k{\parallel }^2}\end{array}$$

where ${\sigma }_x^2$ is the variance of the input signal, ${\sigma }_{x_k}^2$ is the variance of the k th subbands and ∥ f _k ∥² is the norm of the k th synthesis filter. Although the coding gain does not completely dominate all image coding results due to rounding error, it is clear that all of coding gain are not lost as shown in Table 1.

Table 1 Comparisons of coding gain of the ideal FLTs and the proposed IntFLTs (dB)

For comparison, the coding gain of LiftLT [12] is 9.5378 (dB) which is higher than the proposed 8 × 16 IntFLTs because this is optimized for lossy coding.

4.2 Lossy-to-lossless image coding

Lossy-to-lossless image coding results by the designed IntFLTs are shown in this subsection. As targets for comparison, LiftLT [12], 5/3-DWT and 9/7-DWT for JPEG 2000 [14], HLT for JPEG XR [16], and the conventional 8 × 16 and 16 × 32 IntFLTs were applied. The conventional 8 × 16 and 16 × 32 IntFLTs are based on simple three-step lifting factorizations of rotation matrices and scaling factors [14]. The periodic extension was used for image boundaries except for DWTs and HLT. To evaluate transform performance fairly, a very common wavelet-based zerotree coder SPIHT [39] was adopted for all^c. Moreover, we used 8-bit gray scale test images with 512 × 512 size such as Barbara.

First, the proposed IntFLTs and the conventional methods are applied to lossless image coding. The comparison of lossless bit rate (LBR)

$$\begin{array}{ll}\text{LBR}& \left(\text{bpp}\right)=\frac{\text{Total number of bits}\left[\text{bit}\right]}{\text{Total number of pixels}\left[\text{pixel}\right]}\end{array}$$

is shown in Table 2.

Table 2 Comparison of lossless image coding (LBR (bpp))

If lossy compressed data is required, it can be achieved by interrupting the obtained lossless bitstream. The comparison of peak signal-to-noise ratio (PSNR)

$$\text{PSNR}\left(\text{dB}\right)=10{log}_{10}\left(\frac{255^2}{\text{MSE}}\right)$$

where MSE is the mean squared error, is shown in Table 3.

Table 3 Comparison of lossy image coding (PSNR (dB))

Even though the proposed and conventional IntFLTs have same transfer function, the proposed IntFLTs perform better coding than the conventional IntFLTs, especially lossy image coding results show excellent performance. We consider that this is mainly due to the reduction of rounding operations as shown in Table 4 and no large lifting coefficients^d. Moreover, note that the proposed IntFLTs have a more effective implementation than the conventional IntFLTs due to the construction with few operations and direct-lifting of DCTs. The direct-lifting can reuse any existing software/hardware for DCTs. On the other hand, LiftLT and 9/7-DWT perform often good lossy image coding because they were designed for the lossy mode. However, it cannot preserve the high frequency components in the images as shown in Figure 4, whereas the proposed IntFLTs, especially the proposed 16 × 32 IntFLT, can preserve them.

Table 4 Comparison of number of rounding operations in each one-dimensional transform of M × 1 signals

Figure 4

Comparison of particular area of reconstructed image Barbara when bit rate is 0.25 (bpp). (Left to right) 9/7-DWT, LiftLT, and 16 × 32 FLBT.

5 Conclusions

This paper presented integer fast lapped transforms (IntFLTs) for effective lossy-to-lossless image coding, which were constructed by few operations and direct-lifting of discrete cosine transforms (DCTs). Due to merging, many rounding operations and keeping small lifting coefficients by use of direct-lifting, the proposed IntFLTs performed better coding than the conventional IntFLTs in lossy-to-lossless image coding. Also, the proposed IntFLTs can preserve the high frequency components in the images. Since the direct-lifting can reuse any existing software/hardware for DCTs, the proposed IntFLTs have a great potential for fast implementation which is dependent on the architecture design and DCT algorithms. Furthermore, the proposed IntFLTs do not need any side information unlike IntDCT based on direct-lifting as our previous work.

Endnotes

^a “The conventional IntFLTs” do not include LiftLT in this paper.

^b Any other lifting-based DCTs cannot reuse all existing software/hardware for DCTs.

^c The block transform coefficients through 2^k-channel ($k\in \mathbb{N}$) FLTs are rearranged to a k-level wavelet-like multi-resolution representation, and they are applied to the zerotree coder [40], e.g., 3-level wavelet-like multi-resolution representation when M = 8.

^d The IntFLT referred by [20] has less rounding operations. However, it performs undesirable coding due to large lifting coefficients. For example, although the 8 × 16 FLOT has only five rounding operations in each 4 × 4 DCT, its application of lossless image coding shows 5.06 (bpp) for Barbara.