Reduction of rounding noise and lifting steps in non-separable four-dimensional quadruple lifting integer wavelet transform

Discrete cosine transform (DCT)-based digital image signal compression was superseded by the discrete wavelet transform (DWT)-based JPEG 2000 as the standard used to compress digital images [1]. The JPEG 2000 restricts the user’s choice to two wavelet transforms—Daubechies 9/7 for lossy compression [2] and the 5/3 LeGall wavelet [3], which has rational coefficients for reversible or lossless compression. The JPEG 2000 also supports arbitrary transform kernels and specifies that they should be implemented by using a lifting scheme [4].

Recent advances in multidimensional image data have enhanced the importance of research on suitable compression methods. Digital multimedia technologies have progressed from one-dimensional (1D) audio signals, and 2D and 3D image signals to 4D signals. The medical imaging industry has also progressed to a filmless environment where the amount of digital data that need to be managed presents a significant challenge. Four-dimensional images are increasingly being collected and used in clinical and research applications, such as 4D magnetic resonance imaging (MRI), computed tomography (CT), ultrasound, and functional MRI. Four-dimensional medical images have had a significant influence on the diagnosis of diseases and surgical planning [5]. An image slice resolution of 512 × 512 has been the minimum standard, but nowadays, state-of-the-art scanning systems can output image slices at spatial resolutions of 1024 × 1024 or more at increasing pixel bit depths [6]. Limitations on storage space and transmission bandwidth, on the other hand, and the growing size of medical image datasets, on the other, have spurred research on the design of ad hoc tools. The increasing demand for efficiently storing and transmitting digital medical datasets has triggered investigations into multidimensional and dynamic image compression. Thus, a number of studies have examined the compression of 4D medical images, such as [7–9].

By adopting the Joint Photographic Experts Group (JPEG) international standard, a class of separable 2D WT has been broadly developed for various applications designed to efficiently compress digital still images. As its transfer function is composed of the product of a 1D transfer function in two spatial dimensions, it can inherit the legacy of previously designed 1D structures suitable for hardware implementation [10, 11]. It can also feature regularity and low sensitivity to various kinds of noise [12]. Non-separable structures have been primarily introduced to enhance the accuracy of prediction by adapting to the local context of neighboring pixels [13, 14]. Furthermore, several studies on reducing hardware complexity by introducing parallel processing to image coding were reviewed in [15] and a parallelization of the 2D fast wavelet transform was proposed in [16]. Directionality has recently been utilized in a generalized poly-phase representation [17–19] with the aim of designing adaptive high-pass filters of wavelet transforms.

A new class of non-separable 2D structures has been reported in [20–22], where the transfer function can be expressed as a product of four 1D functions. The transform based on this structure is compatible with the separable transform. The non-separable structure is not a cascade of instances of 1D signal processing in a 1D structure, but requires multidimensional memory access. This structure can reduce the total number of steps of the lifting scheme and the rounding operations therein.

Various types of wavelet transforms have been reported to analyze the geometry of 4D images [23], 4D hyperspectral images [24], 4D medical volumetric data [8, 9], 4D light field data [25], and 4D color images [26]. However, most of them use the separable 4D WT that contains a large amount of rounding noise. Therefore, a non-separable 3D integer WT was proposed in [27] to overcome its limitations. Unfortunately, this was limited to a double-lifting integer WT with a 5/3 filter especially applied for lossless coding. A non-separable quadruple 3D WT with a 9/7 filter was subsequently proposed in [28] for lossy coding. Nevertheless, unlike in the double-lifting WT, the variance of rounding noise increased in the quadruple lifting WT even though the number of lifting steps decreased. Rounding noise in the transform can reduce the efficiency of the lossy coding structure. This paper is the first to use a non-separable 4D quadruple lifting integer WT with the aim of reducing rounding noise inside the transform as well as improving its coding performance. Note that a part of this paper was presented in [29].

This paper focuses on lossy coding of 4D signals using the 9/7-type transform based on the quadruple lifting steps, and a reduction in rounding noise in the integer implementation of the transform is affected. As a lifting step needs to wait for the results of calculations from the previous lifting step, many sequential lifting steps incur a long delay between input and output. The real numbers assumed as signal values inside the transform are rounded into finite-length rational numbers. Shorter lengths imply lower computational load but more rounding noise. The space needed for memory storage can be reduced in a tradeoff with rounding noise [30].

This paper proposes a non-separable 2D quadruple lifting structure for 4D input signals to deal with the problem of degradation in image quality due to integer implementation. It has the advantage that its output signals, apart from rounding noise, are identical to those of a conventional transform the transfer function of which is a product of 16 1D transfer functions. Unlike the prevalent 3D quadruple lifting structure, the order of lifting steps in the original separable 4D transform is preserved. Thus, the total rounding noise is reduced even though the total number of rounding operators remains the same as in prevalent methods. Experiments confirmed that the total rounding noise observed in each frequency band of the decoded images was significantly smaller. The upper bounds of the quality-decoded images in lossy coding mode also improved.

The remainder of this paper is organized as follows: Section 2 introduces the two types of WT and the lifting structure in 1D WT. This structure is extended to the 4D case in Section 3, where the separable 4D structure is presented as “existing I” method. The non-separable structure is introduced in Section 3.2. It uses a 3D structure for 4D WT and is referred to as “existing II” method. In Section 4, the proposed methods are introduced and compared with the existing methods. They are combinations of non-separable 2D and 3D structures, called the “proposed I method,” and a non-separable 2D structure called “proposed II” method. All methods are experimentally compared in terms of various aspects of six input signals in Section 5. The conclusions of this paper are detailed in Section 6.

Figure 1 shows the forward and backward transforms of the integer WT developed for the 5/3-type transform of the lossless coding of a discrete 1D signal in JPEG 2000 [31]. Figure 1a shows the forward transform of the integer WT and Fig. 1b shows its backward transform. It is composed of two lifting steps. The input signal X is down-sampled and fed into the forward transform, which is transformed into a low-frequency band signal, Y_L, and a high-frequency band signal Y_H. A₁ and A₂ are the coefficients of the filter bank, and Y_L and Y_H are coded with an entropy encoder to generate a bit stream for storage and communication. The band signals are then decoded and inversely transformed to obtain the reconstructed signal, X.

Figure 2 shows a 9/7-type transform developed for lossy coding. Two more lifting steps and scaling with a constant k are added in this type. This paper focuses on the 9/7-type transform for lossy coding of 4D signals. A problem related to the integer implementation of the transform is addressed here.

Note that R[ ] denotes the rounding operator that truncates a pixel value in real numbers to an integer. Calculation in a lifting step starts after the calculation results of the previous have been obtained. The greater the number of lifting steps, the higher the latency (or delay). Therefore, the authors reduce the total numbers of lifting steps and rounding operators in the 4D integer WT in the 9/7-type transform. Note that this paper focuses on reducing rounding noise in the transform to increase the coding performance for 4D data in lossy mode.

Lossless reconstruction can be guaranteed with the scaling factors, k ⁻¹ and k are 1.

Note that the input signal is scaled with 2 ^F beforehand as shown in Fig. 2. In the integer implementation, F is set as a positive number. The smaller the F is, the shorter the bit depth of signals inside the transform will be.

Note that the MRI, functional magnetic resonance image (fMRI)(II), and US data were retrieved from [33, 34] and [35], respectively. MRI data represent highly correlated data at consecutive time points with limited motion in the temporal dimension t and structural changes in the spatial dimension z.

Each image is normalized to the range [0, 255] for display purposes as shown in Fig. 12a–f. In this paper, the variance in noise in frequency domain and coding performance in lossless and lossy coding modes were investigated.

Note that ρ was set to 0.9 in the experiments in this paper, as the typical values of ρ for natural images are between 0.9 and 0.98 [36].

5.1 Evaluation of rounding noise

As fewer rounding operators do not imply fewer rounding errors in total [21, 22], the total rounding noise in the output frequency band signal was investigated. The rounding operation is a vital part of lossy compression.

A non-linear operation transforms a floating-point signal into an integer signal. An equivalent expression to the rounding operation is shown in Fig. 13. A non-linear equation with additive noise is defined as

$$ {S}_{R_o}={S}_{R_i}(z)+{N}_R(z), $$

(44)

where \( {S}_{R_i} \), \( {S}_{R_o} \), and N _R denote the input signal, the output signal, and the additive noise of the rounding operation, respectively. As the correlation between each of the of the errors and the signals was zero (based on statistical independence), the variance of output signal was calculated from

$$ {\sigma}_{S_{R_o}}^2={\sigma}_{S_{R_i}}^2+{\sigma}_{N_R}^2, $$

(45)

where \( {\sigma}_{S_{R_i}}^2 \), \( {\sigma}_{S_{R_o}}^2 \), and \( {\sigma}_{N_R}^2 \) refer to the variance of the input signal, the output signal, and the additive noise of the rounding operators, respectively. As the power of the probability density function (PDF) of additive noise is approximately flat, as shown in Fig. 14, the variance of the additive noise for the rounding operations was calculated as

$$ {\sigma}_{N_R}^2={\int}_{-0.5}^{0.5}{x}^2 dx=\frac{1}{12}, $$

(46)

In this study, rounding noise inside the circuit was measured to observe the accumulative error in it due to the rounding operators. Figure 15 illustrates that rounding noise was examined by using the error between the integer signal and the real number signal. Rounding noise is defined by Eq. (47).

$$ error=y-\widehat{y,} $$

(47)

Rounding noise was evaluated in six types of input signals to compare the conventional and the proposed lifting structures, as shown in Fig. 16a–f.

Figure 16a–f indicate the variance of the rounding error in each frequency band for fMRI(I), CT, fMRI(II), MRI, AR, and US data respectively. The average magnitude of rounding errors for all data was reduced to 19.11, 16.08, 16.81, 5.63, and 16.09%, respectively, except for US data, which was increased by 15.47%, between existing I structure and proposed II structure, as shown in Fig. 17. However, as the rounding noise in the LLLL frequency band for proposed II structure for the US data was lower than that in existing I, the coding performance on the US data for proposed II method improved as shown in Fig. 20f. The HHHH frequency band of the existing II structure was the highest and resulted in a degradation in coding performance. The energy of the frequency band should be compacted to a low-frequency band signal to reduce entropy in the compressed image and enhanced coding performance. The higher the variance of rounding noise in the low-frequency band signal, the lower the entropy of the compressed image. However, all structures had the highest compacted energy in the high-frequency band signals. Therefore, we conclude that the lowest variance in rounding noise among all frequency band signals yields the best coding performance.

The reason for why the total number of rounding operators did not influence total rounding noise is investigated in Fig. 18, which shows the variance of rounding noise in the frequency domain. In an experiment, only one rounding operator in the forward transform was activated. The horizontal axis denotes the activated rounding operator, numbered from top to bottom, from left to right in Figs. 4 to 7. As summarized in Table 1, the existing I, existing II, proposed I, and proposed II structures had 192, 96, 72, and 96 rounding operations, respectively. The total rounding noise in proposed II decreased even though the number of rounding operations, the source of the noise, remained the same as that in the existing II structure. This is because of the relatively large variance in error in the structure. The variance in rounding noise was the highest in the 63rd rounding operator at 0.1844, the ninth rounding operator at 1.2217, the 14th rounding operator at 0.5964, and the 56th rounding operator at 0.2127 for existing I, existing II, proposed I, and proposed II, respectively. However, the average rounding noise in proposed II relative to existing I decreased by almost half from 0.0646 to 0.0382. It can be concluded that the more frequent and the greater the extent to which noise is amplified in the lifting structure, the higher the rounding noise inside it. As the rounding noise inside the transform influences coding performance, it is investigated in Section 5.2.

5.2 Evaluation of coding performance

Figure 19 illustrates the rate distortion curve, which compares the performance of the methods in the lossy coding mode for CT images. The horizontal and vertical axes represent the entropy rate measured in bits per pixel (bpp) and the peak signal-to-noise ratio (PSNR) of the reconstructed signal, respectively. It shows that the proposed II structure outperformed the others at the same bitrate. The quality of the reconstructed signal in proposed II increased by 0.18, 16.11, and 26.44 dB over existing I, existing II, and proposed I, respectively. As the quality of the reconstructed image deteriorated considerably if it was transformed using existing II and proposed I. Figure 20a–e shows the coding performance results when images were transformed using existing I and proposed II only.

Figure 20a–e shows the coding performance of existing I and proposed II on fMRI(I), fMRI(II), MRI, AR, and US data, respectively. Under the same bitrate, the quality of the reconstructed signal for proposed II increased by 0.40, 2.10, 0.27, 2.57, and 0.72 dB for fMRI(I), fMRI(II), MRI, AR, and US data, respectively. As both quantization noise and rounding noise were present in the lossy compression system, even though the former for all structures was the same, rounding noise in the proposed II structure was the lowest. Thus, the coding performance of this structure was the best. In conclusion, proposed II structure outperformed all other structures in the experiments.

This paper proposed a non-separable 2D structure for 4D input signals in lieu of non-separable 3D structures. The total number of rounding operators was reduced by half compared with the prevalent separable 4D structure. However, rounding noise owing to the integer implementation of signal values inside the transform increased in the non-separable 3D structure, due to a change in its original lifting scheme. Therefore, by maintaining the original scheme, the proposed non-separable 2D structure reduced total rounding noise in it and enhanced the quality of the reconstructed signal in lossy coding. Furthermore, the number of lifting steps, a reduction in which reduces the latency of the overall transform, was also reduced by 18.75% between the proposed non-separable 2D structure and the conventional separable 4D structure on the quadruple 4D integer WT. Our integer WT has the advantage of compatibility with the conventional integer WT. It also enhances compression performance on 4D images, such as medical images.