Steganography algorithm based on modified EMD-coded PU partition modes for HEVC videos

Abstract

As High Efficiency Video Coding (HEVC) is a worldwide popular video coding standard, the steganography of HEVC videos has gained more and more attention. Prediction unit (PU) is one of the most important innovative modules of HEVC; thus, PU partition mode-based steganography is becoming a novel branch of HEVC steganography. However, the embedding capacity of this kind of steganography is limited by the types of PU partition modes. To solve the problem, modified exploiting modification direction (EMD)-coded PU partition mode-based steganography is proposed in this paper, which can hide a secret digit in a (2^n + x − 1)-ary notational system in a pair of PU partition modes and thus enlarging the capacity. Furthermore, two mapping patterns for PU partition modes are analyzed, and the one that performs the better is selected as the final mapping pattern. Firstly, 8 × 8- and 16 × 16-sized PU partition modes are recorded according to the optimal mapping pattern in the video encoding process. Then, PU partition modes are modified by using the proposed method to satisfy the requirement of secret information. Finally, the stego video can be obtained by re-encoding the video with the modified PU partition modes. Experimental results show that the embedding capacity can be significantly enlarged, and compared with the state-of-the-art work, the proposed method has much larger capacity while keeping high visual quality.

1 Introduction

Steganography is a hot research issue in the field of information security. It can imperceptibly conceal secret information in the carrier such as texts, images, and videos and has extensive applications in copyright protection, identity authentication, covert communication, and so on. Among the various branches of steganography technique, video steganography has attracted more and more attention because video is one of the most widely used digital media and has larger capacity for data embedding compared with other carriers such as texts and audios.

Video steganography is closely linked to video coding standard. High Efficiency Video Coding (HEVC) is the latest video coding standard possessing the best compression efficiency compared with its predecessors such as H.264. HEVC is specifically designed for the increasingly popular high-definition and ultra-high-definition videos [1]. At present, a number of enterprises including Apple and Microsoft have designed products supporting HEVC standard. Therefore, it is worthwhile to develop steganography schemes for HEVC videos.

Modifying DCT/DST coefficients is one common approach to hide data in compressed HEVC videos. Chang et al. [2] classified the blocks in intra-coded frames into 5 categories. Different perturbation patterns were introduced to the luma blocks of each category to avoid intra-frame distortion drift. Similar ideas were taken by Liu et al. to embed data in 4× 4 DCT blocks [3] and 4× 4 DST blocks [4] of selected frames. Ref. [2,3,4] were selection-based intra drift free methods. Arige et al. [5] designed the first cancelation-based intra drift free method for HEVC videos which was more general than Ref. [2,3,4]. Besides 4 × 4 blocks, 8 × 8 blocks, and 16 × 16 blocks which satisfy certain coarseness conditions were also employed as carriers in Ref. [6], which made its embedding capacity enlarged.

Intra-prediction mode (IPM)-based steganography is also a hot topic for HEVC videos. Wang et al. [7] proposed an IPM-based stegography method by introducing (4, 3) standard array decoding table. The method achieved satisfying embedding capacity while modifying IPMs as few as possible. After that, the authors designed a new mapping table between the angle differences of IPMs and secret information to embed data [8]. The study in [9] explored diamond coding to increase the embedding capacity and could conceal 2–3 bits data by modifying 2 IPMs at most. Wang et al. [10] summarized the existing IPM-based steganography methods and proposed a cover selecting rule by analyzing the probability distribution of 4 × 4 IPMs.

Besides DCT/DST coefficients and IPM, motion vector and quantization parameter (QP) are another two popular carriers to hide secret information. Van et al. [11] proposed a low complexity steganography method by modifying the motion vector differences and DCT coefficients selected from the frames classified by the inter-prediction dependencies between them. An efficient steganography method based on motion vector space encoding was presented in [12]. Since not every CTU could be used as carrier, the capacity of the method was limited. In the aspect of QP based steganography methods, Zuo et al. [13] combined the just noticeable coding distortion model and exploiting modification direction method to modify the optimal QP values for each coding unit (CU) and embed secret information.

The aforementioned HEVC steganography methods are based on the traditional carriers that are the same with previous video coding standards such as MPEG-4 and H.264. However, HEVC has developed some innovative technologies including flexible quatree partition architecture (CU, PU, and transform unit (TU)) and sample adaptive offset. The innovative technologies also offer abundant carriers to embed secret information, thus they opened up a new research field for HEVC steganography.

Shanableh [14] utilized the flags indicating the partition of 16 × 16 sub-CUs as carriers to embed data, where up to an average payload of 32.6K bit/s can be achieved. Tew et al. [15] divided the prediction block (PB) into two categories. One indicated “0” and the other indicated “1.” Data were hidden by modifying the partition mode of PBs. Inspired by Ref. [15], Xie et al. [16] classified the PU partition mode into 3 categories except for “N × N”and “2N × 2N” mode, which enlarged the embedding capacity. By modifying Ref. [16], a multi-level steganography method was designed in Ref. [17], where the embedding capacity can be adjusted according to the actual requirement. Exploiting modification direction (EMD) was applied to PU partition mode in Ref. [18] to further enlarge the embedding capacity. These PU partitioning mode based steganography methods have achieved excellent visual quality, but their capacities are relatively low for the limited number of CUs. Even the method of [18], which has improved the capacity a lot compared with [15,16,17], still required three consecutive CUs to form a group to hide a digit in 7-ary notational system. Stimulated by this, we adopt a modified EMD coding method [19] which only required two elements in each group to pre-code PU partition modes. By modifying the characteristic value function and designing a new codebook, a high-performance HEVC video steganography is proposed in this paper.

CUs with different sizes own different PU partition modes, which have an impact on embedding capacity; thus, we investigate the variety of PU partition modes for different sized CUs and select CUs with sizes of 16 × 16 and 8 × 8 to hide information. Furthermore, it is essential to map PU partition modes into integers for PU partition modes based steganography, and two mapping patterns are discussed in this paper. After that, PU partition modes of 16 × 16-sized CUs and 8 × 8-sized CUs are forced to be altered according to the modified EMD rules to embed secret data. The experimental results show that the proposed method has much larger capacity than the state-of-the-art works while keeping high visual quality.

The rest of the paper is organized as follows. CU and PU partition modes in HEVC are demonstrated in Section 2. Section 3 discusses the modified EMD method for PU partition modes and the designed mapping patterns. Detailed description of the proposed steganography method is also presented in Section 3. Experimental results are illustrated in Section 4, and Section 5 summarizes the paper.

2 Background of CU and PU partition modes in HEVC

HEVC was jointly released by ITU-T [20] and ISO/IEC [21] and has brought in various new techniques to increase compression efficiency. One of the key techniques is the introduction of quadtree partition structure. In the quadtree partition structure, CU, PU, and TU are introduced. CU is the basic coding unit, and PU and TU are the basic units for intra/inter prediction and transform/quantization, respectively.

In encoding process, an image is firstly divided into non-overlapping coding tree unit (CTU) whose size is set by the user with 64 × 64 as default. Then each CTU is recursively divided into square shapes in a quadtree structure. Every leaf node of a CTU is called a CU which is the basic coding unit in HEVC. There are 4 allowed types of CUs: 64 × 64, 32 × 32, 16 × 16, and 8 × 8. As shown in Fig. 1b, the quadtree structure of the left top CTU surrounded by red square in Fig. 1a is displayed. In Fig. 1a, the white lines indicate the boundary of CUs, and the digit denotes the coding order of each CU. We can see that the quadtree partition of CTU is closely related to the image texture. In the region with simple texture such as the tree trunk in Fig. 1a, larger-sized CUs are adopted. While in the region with complex texture such as the border between the tree trunk and the road, more detailed CUs are adopted.

Fig. 1

Example of quadtree partition of CTU into CUs and PUs. a CTU partition. b CU partition quatree. c PU partition

When performing CTU quadtree tree partition, PU partition mode is determined for each CU in the traversal way according to the rate distortion function. PU partition modes are different under different prediction mode, as presented in Fig. 2. In intra prediction mode, one 2N × 2N (N = 4, 8, 16, 32) CU can be encoded as one single PU or be further split into 4 PUs with equal size (N × N). In inter prediction mode, the CU can be divided into symmetric or asymmetric PUs. Both symmetric and asymmetric PUs are composed of 4 partition modes. Figure 1c shows the PU partition mode of the left top CTU marked red in Fig. 1a. The black line implies that the PU has the same size as the corresponding CU, and the purple line means that the CU needs to be subdivided into two PUs in order to get minimum rate distortion.

Fig. 2

PU partition modes in different prediction modes

From Fig. 2, we can see that more various PU partition modes are available in inter prediction mode than in intra prediction mode. Since we aim to hide information by modifying the PU partition mode, we select the PU partition mode in inter prediction mode, which means that P pictures are adopted to hide information in this paper. Li et al. [18] stated that 8 × 8- and 16× 16-sized CU account for the dominant number of CUs, so we only take 8 × 8- and 16 × 16-sized CU into consideration. We can see that 8 PU partition modes for 8× 8-sized CU are presented in Fig. 2, but in HM 16.15 [22] which is the reference software of HEVC, N× N PU partition mode is not performed for CUs with size larger than 8 ×8. Therefore, there are 7 optional PU partition modes for 16 ×16 CU. Tables 1 and 2 exhibit the supported PU partition modes for 8 ×8 and 16 ×16 CU, respectively. Due to the variety of PU partition modes, we determine to use PU partition modes of 8 ×8-sized CUs and 16× 16-sized CUs to hide information in this paper.

Table 1 PU partition mode for 8 × 8 CU

Table 2 PU partition mode for 16 × 16 CU

3 The proposed steganography method

In this section, the EMD-coded PU partition mode-based steganography [18] is firstly described briefly, and then the proposed steganography based on the modified EMD is illustrated. After that, two mapping patterns for PU partition modes are discussed, and the detailed data hiding process which adopts the optimal designed mapping pattern is described.

3.1 EMD-coded PU partition mode-based steganography

In the latest EMD-coded PU partition mode-based steganography [18], each secret digit s in a (2n + 1)-ary notational system can be concealed by modifying only one PU partition mode in the group of n (n ≥ 2) PU partition modes. In the EMD method, each PU partition mode is mapped into an integer. For 16 × 16-sized CU, the integer ranges from 0 to 7, while for 8 × 8, the integer ranges from 0 to 4. Then the characteristic value F of each PU partition mode group can be calculated as Eq. (1), where (p₁, p₂, ⋯, p_n) denotes the PU partition mode group.

$$ F=f\left({p}_1,{p}_2,\cdots, {p}_n\right)=\left[\sum \limits_{i=1}^n\left({p}_i\cdot i\right)\right]\operatorname{mod}\kern0.2em \left(2n+1\right) $$

(1)

If the secret digit s equals to the characteristic value F, then no alteration of PU partition mode is needed. Otherwise, d = (s ‐ F) mod  (2n + 1) is calculated. If s > F, the value of p_d is increased by 1, and if s ≤ F, p_{2n + 1 ‐ d} is decreased by 1. According to this rule, secret digit can be concealed in the PU partition group.

From the discription, we can get the embedding rate of EMD scheme which is log₂(2n + 1)/n. When n equals to 3, the embedding rate is log₂7/3and the embedding capacity is limitied. However, the embedding capacity can be enlarged by utilizing the modified EMD method which introduces the concept of codebook. In the modified EMD coding, a (2^n + x − 1) base digit can be concealed in one PU partition mode group. Stimulated by this, the modified EMD method is introduced to pre-code PU partition modes for further improving the embedding capacity in this paper and will be presented in the next subsection.

3.2 Proposed steganography based on PU partition modes

In the proposed steganography, CTU is assigned as the basic processing unit. For a CTU which consists of T 16 × 16-sized CUs, it can be divided into a series of groups containing n (n ≤ T) consecutive CUs each. Since one CU corresponds to one PU partition mode, each CU group will contain n PU partition modes. After mapping (two mapping patterns have been designed and will be discussed in next section) each PU partition mode into an integer p_i, i = 1, 2, ⋯, n, a vector of (p₁, p₂, ⋯, p_n) which denotes one PU partition mode group can be achieved. Then the characteristic value F of the group can be calculated as a weighted sum modulo (2^n + x − 1), as shown in Eqs. (2) and (3), where (b₁, b₂, ⋯, b_n) denotes a basis vector which depends on the integer x.

$$ F=f\left({p}_1,{p}_2,\cdots, {p}_n\right)=\left[\sum \limits_{i=1}^n\left({p}_i\cdot {b}_i\right)\right]\operatorname{mod}\kern0.2em \left({2}^{n+x}-1\right) $$

(2)

$$ \left[{b}_1,{b}_2,\cdots, {b}_n\right]=\left\{\begin{array}{cc}1,6,\cdots, \left(n-1\right)\times 6,& \mathrm{if}\left(x\ge 3\kern0.2em \mathrm{and}\kern0.2em n\ge 2\right)\\ {}1,4,\cdots, \left(n-1\right)\times 4,& \mathrm{if}\left(1\le x\le 3\kern0.2em \mathrm{and}\kern0.2em n\ge 2\right)\end{array}\right. $$

(3)

The hiding rule of the proposed steganography with modified EMD is to make the characteristic value F equal to the secret digit s in a (2^n + x − 1)-ary notational system. If s = F, no alteration of the current group of PU partition modes is needed. On the contrary, a codebook generated by the modified EMD is designed to change the PU partition mode. The index of the codebook satisfies Eq. (2), and the rule of the codebook is to make minimum alteration of the integers which represent different PU partition modes.

Table 3 illustrates the codebook when n = 2 and x = 3. If s ≠ F, the index d is calculated by Eq. (4). Then look up the codebook and find the row with the index d, the vector (d₁, d₂, ⋯, d_n) in the specific row is used to modify the PU partition modes group. Finally the vector of the modified PU partition modes group $\left({p}_1^{\prime },{p}_2^{\prime },\cdots, {p}_n^{\prime}\right)$ can be obtained by Eq. (5), and the characteristic value $f\left({p}_1^{\prime },{p}_2^{\prime },\cdots, {p}_n^{\prime}\right)$ will be equal to the secret digit s.

$$ d=\left\{\begin{array}{cc}s-F& \mathrm{if}\kern0.2em \left(s\ge F\right)\\ {}{2}^{n+x}-1-\left|s-F\right|& \mathrm{if}\left(s<F\kern0.2em \right)\end{array}\right. $$

(4)

$$ {p}_i^{\prime }={p}_i+{d}_i $$

(5)

Table 3 Codebook when n = 2 and x = 3 with b₁ = 1 and b₂ = 6

When applying modified EMD-coded PU partition modes, two important issues should be taken into consideration. One is the selection of n and x. The other one is the overflow or underflow problem.

As for the first consideration, the setting of n and x is closely related to the embedding capacity. Supposing that the number of 16 × 16-sized CU in one video picture is denoted by Q, since log₂(2^n + x − 1) bits can be hidden in one group containing n PU partition modes, the embedding capacity can be obtained according to Eq. (6), where C means the capacity of one video picture. The partial derivative of C to n in Eq. (7) shows that the capacity will decrease with the increase of n. Therefore, in order to get the maximum capacity, we set n = 2 in this paper. Please note that the value of n should be larger than one. The reason is that if n = 1, only one PU partition mode is contained in one group, and the minimum alteration of the PU partition mode cannot be guaranteed when the difference between the characteristic value F and secret digit s is large, which will further affect the visual quality of the video.

$$ C=Q{\log}_2\left({2}^{n+x}-1\right)/n $$

(6)

$$ {\displaystyle \begin{array}{l}\frac{\partial C}{\partial n}=\frac{\partial }{\partial n}\left(Q{\log}_2\left({2}^{n+x}-1\right)/n\right)\\ {}\kern1.8em =Q\frac{1+\left(\frac{1}{2^{n+x}-1}\right)-\mathrm{lo}{\mathrm{g}}_2\left({2}^{n+x}-1\right)}{n^2}<0,\kern0.7em n\ge 2,x\ge 1\end{array}} $$

(7)

Referring to the relationship between C and x, it is obvious that more bits can be hidden with larger x. However, the value of x is limited by the available number of PU partition modes. As n = 2, (p₁, p₂) denotes a group of PU partition modes, 49 combinations of (p₁, p₂) can be got because bothp₁ and p₂ own 7 available values. One requirement of the modified EMD coding is that the characteristic values F calculated by all the combinations of (p₁, p₂) should cover every value in the (2^n + x − 1)-ary notational system, i.e. from 0 to 2^n + x − 2. Through theoretical analysis, it is found whenx > 3, the range from 0 to (2^n + x − 2) is too large to cover by these 49 combinations of (p₁, p₂); hence, it is better to set x ≤ 3. Taking the above two factors together, we set n = 2 and x = 3 for 16 ×16-sized PU partition modes in this paper. For 8 ×8-sized CUs, the hiding rule is the same as 16 × 16-sized CUs. By analyzing the setting of n and x in a similar way to 16 ×16-sized CUs, we select n = 2 and x = 2 for 8 ×8-sized CUs to get the maximum embedding capacity in this paper. The codebook for n = 2 and x = 2 is exhibited in Table 4.

Table 4 Codebook for n = 2 and x = 2 with b₁ = 1and b₂ = 4

For 16 ×16-sized CUs, from Eqs. (4) and (5) and Table 3, we can see that the modified PU partition mode${p}_i^{\prime}\kern0.1em$ is obtained byp_i adding or subtracting an integer. The modifying operation may lead to ${p}_i^{\prime}\kern0.1em$larger than 7 or smaller than 0, which is called overflow or underflow. Take (p₁, p₂) = (1, 2) for instance, the characteristic value F = f (p₁, p₂)is 13 with n = 2 and x=3. If the secret digit s = 4, according to the codebook and Eqs. (4) and (5), $\left({p}_1^{\prime },{p}_2^{\prime}\right)=\left({p}_1,{p}_2\right)+\left(-3,-1\right)=\left(-2,1\right)$ can be got, which indicates the case of underflow that ${p}_1^{\prime }$ is out of the range from 0 to 6 and the PU partition mode corresponding to -2 cannot be found. To solve the problem, the 49 combinations of (p₁, p₂)are traversed, and the candidate groups, whose f(p₁, p₂)value equals to the secret digit, are selected. Let $\mathbf{P}=\left\{{\mathbf{p}}_l=\left({p}_{l_1},{p}_{l_2}\right),l\in \Big\{1,2,\cdots, k\Big\}\right\}$denotes the set of the candidate groups, where k means the number of candidate groups, the Manhattan Distance md of each group from (p₁, p₂)is calculated as Eq. (8) and the group which owns the minimum Manhattan Distance from (p₁, p₂)is considered as the final group to replace(p₁, p₂). The overflow and underflow question for 8 ×8-sized CUs are considered in the same way as 16×16 sized CUs.

$$ md=\left|{\mathbf{p}}_l-\mathbf{p}\right|=\left|{p}_{l_1}\hbox{-} {p}_1\right|+\left|{p}_{l_2}\hbox{-} {p}_2\right| $$

(8)

3.3 Optimal mapping pattern for PU partition modes

As demonstrated in Section 3.1, each PU partition mode should be mapped into an integer for being coded by the modified EMD method. There are 7 and 4 different PU partition modes for one 16 × 16-sized CU and 8 ×8-sized CU, respectively; hence, 7 integers range from 0 to 6 and 4 integers range from 0 to 3 are adopted to represent PU partition modes for one 16×16-sized CU and 8×8-sized CU, respectively. However, different mapping patterns will affect the visual quality and bitrate of the video at different levels. Therefore, two mapping patterns for PU partition modes are considered in this paper, and the mapping pattern which causes the less change in the visual quality and bitrate of the video is selected.

The first mapping pattern is determined by the relationship between the distribution of PU partition modes and the probability of modifying other integers to each specific integer in the modified EMD method. We call this mapping pattern “mapping 1” for abbreviation. The aim of mapping 1 is to make least change on the PU partition mode distribution. Besides the consideration of mapping 1, the similarity between adjacent PU partition modes is also considered in the second mapping pattern which is referred to as “mapping 2.”

For mapping 1, we firstly calculate the probability of modifying other integers to each specific integer in the modified EMD method. The situation with n = 2 and x = 3 as shown in Table 3 for the proposed steganography is tested, which indicates that a pair of PU partition modes denoted by (p₁, p₂) are taken as covers. All available integers for (p₁, p₂) pairs are traversed in the test. For each pair of (p₁, p₂), every 2^n + x − 1 base number which suggests the secret digit, is embedded following the proposed steganography. During the traversal, (p₁, p₂)will be changed to fit the 2^n + x − 1 base number. For example, if (p₁, p₂) equal to (0,1) and the secret digit is 1, then according to Eqs. (2–5), p₁ and p₂ are changed to 1 and 0, respectively. It means that the count of p₁ changed from 0 to 1 is increased by 1, and the count of p₂ changed from 1 to 0 is increased by 1. After the traversal, we can get one matrix which records the count of modifying other integers to each specific integer ranges from 0 to 6 for p₁and p₂, respectively. Then we do element-wise addition with the two matrixes, and the probability of modifying (p₁, p₂) to each integer in the set {0,1,2,3,4,5,6} can be achieved by dividing the sum of the sum matrix into each element in the sum matrix. The results are shown in Table 5. We can see that the integer “3” owns the largest probability among the 7 integers, it means that most PU partition modes would be altered to the PU partition mode which is mapped to integer 3 during the embedding process.

Table 5 The probability of modifying other integers to each specific integer for 16 × 16-sized CU in the modified EMD coding method

Besides, the distribution of each PU partition mode for videos with different resolutions has been investigated and shown in Fig. 3. Each video contains six P pictures, and the quantization parameter adopted is 32. It is observed that 2N × 2N PU partition mode takes the first place at any resolution for 16 ×16-sized CU, N × 2N PU partition mode the second, and 2N × N PU partition mode the third. In order to present the PU partition mode distribution more objectively, for each video shown in Fig. 3, we calculate the ratio of the number of each PU partion mode to the total number of all the PU partition modes. Then we average the ratios of the four videos, and the PU partition mode distribution can be obtained. Table 6 shows the PU partition mode distribution for 16 ×16-sized CU.

Fig. 3

Distribution of PU partition modes for videos with different resolutions. a BasketballDrive_1920 × 1080. b PeopleOnStreet_2560 × 1600. c Chinaspeed_1024 × 768. d Race Horses_832 × 480

Table 6 PU partition mode distribution for 16× 16-sized CU

When designing mapping rule, it is a feasible way to map the PU partition mode to the specific integer according to the occurrence frequency of the PU partition mode and the probability of modifying other integers to the specific integer. For example, 2N × 2N PU partition mode is the dominant one in the distribution of PU partition modes, so we map it to the integer 3 that owns the largest probability to be altered to. In this way, the distribution of each PU partition mode would change the least before and after embedding with modified EMD method. With this consideration, mapping 1 of PU partition modes for 16 ×16-sized CU can be obtained, as listed in Table 7.

Table 7 Mapping 1 of PU partition modes for 16× 16-sized CU

The relationship between the distribution of PU partition modes and the probability of modifying other integers to each specific integer for 8 ×8-sized CU is also considered in the proposed stenography. In order to save the space of the paper, considering the experiments and analysis are the same as 16 ×16-sized CU, we just present the designed mapping 1 pattern for 8×8-sized CU, as shown in Table 8.

Table 8 Mapping 1 of PU partition modes for 8 ×8-sized CU

The idea of modified EMD method is to change the PU partition mode to its adjacent mode if modification is required. The change of PU partition mode will increase the video’s bitrate and decrease its visual quality. In order to reduce the effect of the modification, we also consider the similarity between adjacent PU partition modes on the basis of mapping 1. Since 2N × 2N, N × 2N, and 2N × N PU partition mode take the largest, second largest, and third largest probability of all the optional PU partition modes, respectively, the mapping of them in mapping 2 remains the same as that in mapping 1. The adjacent similarity is considered for the left four PU partition modes and mapping 2 can be obtained (Tables 9 and 10). Table 9 and Table 10 exhibit the specific mapping pattern of mapping 2 for 16 ×16-sized CU and 8 × 8-sized CU, respectively. Taking integer “1” in Table 9 as an example, both its neighbors “0” and “2” indicate the similar PU partition mode with it.

Table 9 Mapping 2 of PU partition modes for 16×16-sized CU

Table 10 Mapping 2 of PU partition modes for 8×8-sized CU

In order to take a comparison of the two mapping rules, four YUV videos with different resolutions are adopted to implement data hiding on 16×16-sized CUs using the proposed steganography. Quantization parameter (QP for abbreviation) is one of the most important parameters for HEVC steganography, thus three QPs are tested in the experiments. Since data embedding process influences the video’s visual quality and bitrate, ∆PSNR and BRI are used as a measurement to indicate the change of visual quality and bitrate, respectively. Equations (9) and (10) illustrate the definition of ∆PSNR and BRI, where PSNR and PSNR^′ denote the PSNR of the cover video without and with data embedding, respectively, and BR and BR^′ represent the bitrate of the cover video without and with data embedding, respectively.

$$ \Delta \mathrm{PSNRR}=\mathrm{PSN}{\mathrm{R}}^{\prime }-\mathrm{PSNR} $$

(9)

$$ \mathrm{BRI}=\left(\mathrm{B}{\mathrm{R}}^{\prime }-\mathrm{BR}\right)/\mathrm{BR} $$

(10)

Figures 4 and 5 exhibit the average ∆PSNR and BRI of the four videos for the two mapping patterns. From Fig. 4, we can see that both the two mapping patterns make a decrease on visual quality at any QPs, but mapping 2 decreases the less. The same phenomenon occurs in Fig. 5, where mapping 2 has the lower BRI comparing with mapping 1. Therefore, mapping 2 is adopted in the proposed method, and the experimental results are based on mapping 2 hereafter.

Fig. 4

The average ∆PSNR of the four videos for the two mapping patterns

Fig. 5

The average BRI of the four videos for the two mapping patterns

3.4 Embedding and extraction process

Here, we will demonstrate the data embedding and extraction process of the proposed steganography in detail.

Step 1: For one CTU, encode it with HEVC encoders and record the numbers of 16×16-sized CUs, 8×8-sized CUs, and the optimal PU partition mode for each CU.

Step 2: For both the 16×16-sized CUs and the 8×8-sized CUs, divide the CUs into multiple groups containing n CUs each. For each group, calculate the characteristic value of the group according to Eqs. (2) and (3). For the given secret digit s in a (2^n + x − 1)-ary notational system, calculate the index d of the codebook by utilizing Eq. (4), and then the modified PU partition modes of the current group can be obtained.

Step 3: Repeat Step 2 to get the modified PU partition modes for each group of the CUs. After that, the CTU is re-encoded by using the modified PU partition modes.

Step 4: Perform Steps 1–3 for all the CTUs in every P-picture of the video.

In order to demonstrate the steps of the proposed method more reproducible, the pseudocode of the embedding process of one PU partition modes group from 16 × 16-sized CUs is given in Table 11. The pseudocode of the embedding process of one PU partition modes group from 16 × 16-sized CUs

Table 11 The pseudocode of the embedding process of one PU partition modes group from 16 ×16-sized CUs

where n = 2 and x = 3.

In the stage of extracting secret information from the stego video, we just need to divide the 16 ×16-sized CUs and 8 ×8-sized CUs into multiple groups and calculate the characteristic value of each group, then concatenate the characteristic value of each group to get the secret information as the secret digit equals to the characteristic value of each PU partition modes group in the stego video.

4 Results and discussion

4.1 Configurations

In this section, the experimental results of the proposed algorithm are presented. Twelve common YUV video sequences with different resolutions, obtained from HEVC test sequences of Joint Collaborative Team on Video Coding (JCT-VC), are used to conduct the experiments, including 1 sequence with resolution 2560 × 1600: Traffic; 4 sequences with resolution 1920 × 1080: Kimono, ParkScene, Cactus, Ducks_take_off; 2 sequences with resolution 1600 × 1200: Intersection, Mainroad; 3 sequences with resolution 1280 × 720: City, Crew, Johnny; 2 sequences with resolution 832 × 480: Keiba, PartyScene. With respect to the video codec platform, HM 16.15 [22] with the main profile is employed. The frame rate, GOP size, and GOP structure are set as 25, 4, and IPPP, respectively. All the videos are tested with three different QPs: 26, 32, and 38.

4.2 Evaluation of subjective visual quality

High subjective visual quality is one of the good behaviors of an excellent video steganography, which means that human eyes cannot perceive the visual distortion of the stego-video. Figure 6 displays the first P picture of sequence Kimino before and after data embedding at different QPs. It is shown that the difference between the original picture and the information hidden picture is so small that it is difficult for human eyes to visually perceived, which indicates the high subjective visual quality of the proposed method.

Fig. 6

Subjective visual quality evaluation at different QPs for Kimino 1920 × 1080. The original pictures are laid on the first column, and pictures with hidden information are placed in the second column. The QP adopted in the first line, second line, and third line are 26, 32, and 38, respectively

4.3 Objective performance

In order to evaluate the performance of the proposed algorithm objectively, three evaluation metrics are adopted in this paper: ∆PSNR, capacity, and BRI. ∆PSNR and BRI, as shown in Eqs. (9) and (10), are used to assess the influence of the proposed embedding process on the visual quality and bitrate of the cover video, respectively. The capacity of the proposed algorithm is calculated as the average bits that can be hidden in one P picture.

Table 12 exhibits the results of the proposed algorithm on the three evaluation metrics. From Table 12, we can get that the average capacity of all the sequences for QP 26, 32, and 38 are 4190, 1379.83, and 437.75 bits per picture, respectively, which shows the high capacity of the proposed method. Furthermore, it shows a tendency that more bits can be hidden for sequences with lower QP or higher resolution in most cases. For example, the capacity of sequence “Traffic” with QP 26 is 10,971 bits, 7474 bits more than that with QP 32, and 7700 bits more than sequence “Kimino” with QP 26 which has lower resolution. This is because more 8 ×8- and 16 ×16-sized CUs will be allocated by the encoder in the case of lower QP or higher resolution, leading to more available PU partition modes that can be embedded in.

Table 12 The ∆PSNR, capacity, and BRI of the proposed algorithm with different QPs

In Table 12, PSNR (clean) means the PSNR of the video sequence without data hiding. ∆PSNR indicates the visual quality distortion caused by the proposed embedding process, the smaller the absolute value of ∆PSNR is, the less distortion it will be. From Table 12, we can see that the value of ∆PSNR is within 0.1 dB in most cases, even larger than zero in some cases. For example, the value of ∆PSNR for sequence “Intersection” with QP 38 is 0.009 dB. The average ∆PSNR value of all the sequences for QP 26, 32, and 38 are − 0.0546, − 0.0373, and − 0.0127, respectively. These results certify that the stego videos after data embedding with the proposed method still keep high visual quality. Furthermore, it is shown that the absolute value of ∆PSNR decreases with the increase of QP as a whole, which means less visual quality distortion for larger QPs. The reason for this is that the numbers of 8 ×8- and 16 ×16-sized CUs are smaller for larger QP than lower QP, and less 8× 8 and 16 ×16 PU partition modes are modified for larger QP.

Since the proposed method altered the PU partition modes from optimal to non-optimal, it will cause the increase of the bitrate. BRI is a common measure for bitrate increase and low BRI is pursued. From Table 12, we can see that except the sequence “Keiba,” the BRI of the proposed method for the other sequences is less than 0.06, and the average BRI of all the sequences for QP 26, 32, and 38 is 0.040, 0.025, and 0.017, respectively, which means that the bitrate increase of the proposed method is acceptable.

4.4 Comparative analysis

To evaluate the performance of the proposed method, two outstanding PU partition mode-based steganography algorithms are selected for comparison. The first one, named Yang’s method [17], is published in 2018, in which a multilevel information hiding algorithm is proposed and different levels can be chosen according to the requirement of embedding capacity and video visual quality. The second one, called Li’s method [18], is the latest PU partition mode-based steganography and has substantial embedding capacity. In this section, the three algorithms are implemented on four different video sequences for comparison, including Kimono and ParkScene with resolution 1920 × 1080 and Keiba and PartyScene with resolution 832 ×480. Because different algorithms cause different compression bitrates though the same video sequence is tested, in order to demonstrate the relationship between visual quality, embedding capacity, and the compression bitrate more intuitively, the curves of bitrate and capacity (RC) and bitrate and distortion (RD) are plotted to compare the results.

Figures 7 and 8 describe the RC curves of the three methods. As the capacity of each of the four levels in Yang’s method is quite different, and level 4 has the maximum capacity, we chose level 4 for Yang’s method for comparison. The cross lines, diamond lines, and asterisk lines represent the RC curves of Li’s method, Yang’s method, and the proposed method, respectively. From Figs. 7 and 8, we can see that the capacity of the proposed method is the largest among the three methods at any bitrate. Averagely, the capacity of the proposed method is 68% higher than Li’s method, and it is about 4 times that of Yang’s method. Since 2N × 2N PU partition modes which account for the largest proportion are not used to hide information, the lowest embedding capacity is obtained in Yang’s method. Furthermore, in Li’s method, log₂3 bits andlog₂7/3 bits can be hidden for modifying one 8 × 8 PU partition mode and one 16 × 16 PU partition mode, respectively, while the numerical values are log₂15/2 bits and log₂31/2 bits for modifying one 8 × 8 PU partition modes and one 16 × 16 PU partition modes, respectively, in the proposed method. This explains the largest capacity of the proposed method theoretically.

Fig. 7

Comparison of RC with Li’s method and Yang’s method for sequences Kimino and ParkScene

Fig. 8

Comparison of RC with Li’s method and Yang’s method for sequences Keiba and PartyScene

RD curves of the three methods are plotted in Figs. 9 and 10. From Fig. 9, we can see that the PSNR of the proposed method keeps nearly the same with Yang’s method and Li’s method. In Fig. 10, the PSNR of the proposed method is second-best among the three methods, with 0.06 dB lower than Li’s method and 0.23 dB higher than Yang’s method on average. The reason for the little lower PSNR of the proposed method than Li’s method is that more PU partition modes are modified in the proposed method to get a larger embedding capacity. Since the capacity of Li’s method is enlarged by 68% by the proposed method, it is acceptable to get a 0.06 dB visual quality distortion.

Fig. 9

Comparison of RD with Li’s method and Yang’s method for sequences Kimino and ParkScene

Fig. 10

Comparison of RD with Li’s method and Yang’s method for sequences Keiba and PartyScene

4.5 Anti-steganalysis analysis

Steganalysis is the counter-technique to steganography, which aims to detect the presence of secret information in digital media. Thus having resistance to steganalysis attacks is an important requirement for a good steganography method. Sheng et al. [23] proposed the first HEVC steganalysis method for PU partition mode-based steganography. The amount and proportion change rate of 4 × 4, 8 × 8, and 16 × 16 PU in I-pictures before and after recompression were utilized as the feature set, which can effectively detect the steganography. Thus Sheng’s method is reconstructed and used to test the security of the proposed method. Thirty-seven YUV sequences with various resolutions are used as the video set, among which 30 sequences are used as training samples and the left are used as testing samples. The training and testing process are repeated 100 times and the mean value of detection rate is listed in Table 13. From Table 13, we can see that the detection rate of Sheng’s method is about 0.5 at any QP, which indicates that Sheng’s algorithm is almost inefficient to detect the proposed method, and thus the proposed steganography has acceptable resistance to Sheng’s method.

Table 13 The detection rate of Sheng’s method to test the proposed algorithm

5 Conclusions

In this paper, an effective HEVC steganography algorithm based on modified EMD-coded PU partition modes is proposed. The characteristic value calculation equation is modified and the optimal mapping pattern is investigated in order to enlarge the capacity of the PU partition mode based steganography. Then 8 × 8- and 16 × 16-sized PU partition modes are preferred as carriers to hide information according to the embedding process described in Section 3.3. As a result, the embedding capacity can be enlarged significantly, with nearly 1.68 times and 4 times of Li’s method and Yang’s method, respectively. At the same time, the proposed method keeps a high visual quality with no more than 0.1 dB distortion less than cover videos in most cases. In future work, we will look for more outstanding coding methods for PU partition modes or other HEVC features to further improve the performance of the proposed method.