Content and buffer status aware packet scheduling and resource management framework for video streaming over LTE system

Abstract

With the development of the video encoding and wireless communication technologies, DASH (Dynamic Adaptive Streaming over HTTP) services have an increasing and great share of all the mobile services. However, we find some problems which still need to be addressed for DASH service optimization: (1) the limitation of the video segment representations cannot keep pace with the change of wireless channel states; (2) the characteristics of the video transmission have been not fully utilized in current DASH services; and (3) long interruption time will yield poor QoE (Quality of Experience). To solve these problems, we build a new transmission architecture by modifying the traditional TCP transmission flow. We first add a TCP (Transmission Control Protocol) proxy responsible for discarding the expiry packets automatically before putting them into the TCP sending window and then build a “ACK (acknowledgement) reconstruction module” to reconstruct the received ACK for concealing the packets discarded by the packet scheduler. Based on the new framework, we explore the interdependence among all the packets from the encoder to indicate the importance of each packet and update the interdependence relationship of every scheduling period based on the feedback ACK information. At MAC (Media Access Control) layer, a buffer status estimation module is employed to estimate the client buffer and playback information, which can be used to calculate the packet urgency. Then, a cross-layer design, which consists of an application layer of media server and client, TCP layer, MAC layer, and physical layer, is formulated and the packet scheduling and resource allocation can be jointly optimized. At client, through the analysis of the status of client buffer and MAC queue, an adaptive segment request scheme is developed to determine how and when to send the segment request. Simulation results show that the proposed algorithms can efficiently improve the received video quality as well as the playback continuity compared with other existing algorithms.

1 Introduction

1.1 Motivation

With the development of the video compression technology and the wireless network, video streaming in wireless system has taken a growing part of mobile service and attracted more and more attention. Representing the next generation in mobile technology, LTE system based on OFDMA (orthogonal frequency division multiple access) and MIMO (multiple-input multiple-output) technologies can promise faster speed and better coverage. Therefore, it is crucial to investigate how to provide satisfactory multi-client video streaming services to mobile clients by using this promising technology.

Researches have been carried out to enable high-quality video services like HTTP streaming over TCP (Transmission Control Protocol), where each video sequence is pre-encoded into many segments which cannot start to play until all the packets of it have been received [1, 2]. However, besides the high received video quality, continuous playback experience is also an important issue to be addressed from the perspective of the end clients. In addition, due to the error concealment strategy employed in the decoding, some packet loss can be tolerated for video applications. Waiting for the arrivals of all the packets inevitably leads to some unnecessary interruptions for the playback. There are some prior works [3–5] in which the playback time of each packet was fixed. Those algorithms can provide continuous playback but may result in large fluctuation of the packet loss rate because of the time-varying channel states. To get a better trade-off between the video quality and the playback continuity, we introduce a maximum allowed waiting time for the playback of each frame. In our multi-client video streaming system, the packets whose arrival times expire the maximum allowed playback time are discarded. Besides time-varying wireless channel states and the limited availability of resources, unique characteristics of video packets such as the packet priority and stringent playback deadline are also considered in our proposed packet scheduling and RB assignment scheme.

To improve the system performance of video streaming, it is equally important to keep the client buffer and MAC queue at a desired level. Conventional strategies usually request segment with fixed time interval [1] or only consider the status of client buffer [6]. All of those algorithms may result in a low level of client buffer but too many packets accumulated in MAC queue due to the bad channel states. In a scenario with limited MAC queue buffer size, the packet loss rate would increase due to the overflow of the MAC queue. As a result, we propose a MAC (Medium Access Control) queue and client buffer aware segment request strategy to determine the appropriate time to send the segment request.

Since only the above two schemes cannot provide the best system performance, to make the packet scheduling more flexible, many prior works have been carried out to select the most appropriate video presentation or even code the video source to match the time-varying channel states [7–10] or control the client buffer length [1]. However, none of them consider the status of MAC queue. Therefore, we propose a rate adaption strategy based on the MAC queue and the client buffer.

1.2 Related work

The MLWDF scheduling rule in [11] aimed at maximizing the system throughput by considering the packet delay rather than the queue length. Considering both current channel conditions and the packet delay makes MLWDF more appropriate for real-time applications. The authors in [12–17] paid their attention to DASH service optimization by determining the segment adaptation and resource allocation scheme. The authors in [13] presented a system framework description of DASH service and mentioned many key factors that we should consider in DASH service optimization. The authors in [14] proposed a QoE-driven cross-layer optimization for wireless dynamic adaptive streaming of scalable videos over HTTP. They first designed a mapping scheme from scalable video coding layers to DASH layers. Then, they developed a cross-layer design to achieve the resource allocation by considering both the video content and the client buffer status. They also proposed a rate adaptation scheme based on the client buffer status. In general, these works are worthy to be referred and investigated. However, the limitation of the video segment representations cannot keep pace with the change of wireless channel states. Moreover, the characteristics of the video transmission have been not fully utilized in current DASH services and long interruption time will yield poor QoE. Some recent research focused on packet importance and the delay requirements. Our work [18] focused on the video content and estimated the packet importance based on the extracted information from the compressed video. Similarly, [19] employed the video compression characteristics to indicate the packet importance. The wireless channel condition was also considered by using the queue length, input rate, and output rate. However, the packet urgency has not been considered in the above work. In [3–5, 20, 21], the delay requirement depended on the time interval during which the packets can be waiting to be scheduled in MAC queue. They can have a good effect on the video streaming over UDP because the clients experience continuous playback all the time. For instance, the authors in [22] assumed that there is only one frame waiting in MAC queue and the packets of this frame which cannot be transmitted in a certain time interval will be dropped. For the video streaming over TCP, the client will wait until all the packets are received. In this case, different clients will suffer different playback experiences and different playback buffer status. The previous playback experience will affect the following playback condition. Thus, it is inappropriate to use the waiting time in the MAC queue to determine the delay requirement. To accurately examine the packet urgency, the authors in [23–25] analyzed the distribution of the packet display time and proposed a playback control algorithm for real-time video services. In [24], the frame type and its corresponding playback time are considered to improve the received video quality and playback experience. However, their proposed models cannot be adapted in wireless systems because of the varying channel state. Furthermore, since many error concealment strategies can be employed at the decoder and the loss of different video packets will have different effects on the received video quality, it is not necessary to transmit all the video packets for the video service especially in the system with limited wireless resource.

For the segment request, a client usually requests video segments with predetermined time interval (around 2 s for IIS and around 3 s for Netflix in [1]) without considering any information about client in the conventional mechanisms. This may result in some clients having too many frames stored at the client buffers while the others suffer playback buffer underflow. The authors in [6] developed the segment request strategy based on the playback information. But they ignored the information of the MAC queue and the time-varying channel states. With the time when to send the segment request known, how to select the requested segment to adapt to the time-varying channel states becomes a crucial problem to be solved. H.264 SVC (Scalable Video Coding) technology is widely employed in many researches [26–29] to provide different video qualities by encoding each video sequence into many different layers. The authors in [26, 29] exploited the cross-layer design framework for a dynamic scalable video adaptation in varying network capacity. In order to increase the number of the clients that the system can support, the authors in [27] focused on the multi-client video transmission over OFDMA-based communication system by combining the adaptive subcarrier allocation and bit loading with the transmission of the H.264 SVC encoded video sequences. Since scalable video usually consists of multiple scalable layers with different importance which brings high implementation complexity, most of the researches still focus on the pre-encoded video applications. In [7, 8], some studies have been carried out to determine the optimal coding rate that the server should provide. The authors in [7] first allowed the server to drop some unimportant frames by considering the channel states. Then, according to the dropping strategy, the optimal coding rate can be determined by using a simple rate-distortion model. In [8], to aim for maximum QoE (Quality of Experience) for clients, maximum system video throughput, and QoE fairness among clients, a cross-layer optimized coding rate adaptation scheme was proposed. However, both of them focus on the received video quality but neglect another important issue of video streaming: playback continuity. To improve the playback experience, [1] studied a no-reference QoE (quality of experience) monitoring module for adaptive HTTP streaming. The authors showed the playback interruptions can be reduced by changing the QP (quantitative parameters) of the encoder to provide a presentation with a close coding rate to the estimation transmission rate. The authors in [9, 10, 30] formulated the video streaming process with multiple links. Gouache et al. [9] provided a method to estimate the transmission rate of the next scheduling period which can be used to determine the data distribution among the servers. In [10, 30], it is assumed that each client receive the video data from different servers or the other clients in the system. The bit rate can be obtained based on the source node selection strategy. In those algorithms, the bit rate was determined with the assumption that there is no packet in MAC queue waiting to be scheduled. In fact, when we send the next segment request, there may be some packets left in the MAC queue. And due to the different channel states of the end clients, their numbers of the remaining packets maybe different from each other. Since the remaining packets would affect the transmission of the next segment, the status of the clients’ MAC queue should be considered in the bit rate determination.

1.3 Approach

In summary, we find some problems which still need to be addressed for DASH services over TCP as follows: (1) the limitation of the video segment representations cannot keep pace with the change of wireless channel states; (2) the characteristics of the video transmission have been not fully utilized in current DASH services; (3) long interruption time will yield poor QoE. Inspired by the above prior works, we propose a buffer status and content aware packet management framework in LTE system. Our work is novel in the following aspects.

• A new transmission framework for DASH services: We modify the transmission flow of the whole DASH service. With the purpose to transmit the packets which are really important to the received video quality as soon as possible and discard the ones that have little effect on the improvement of the QoE level of all the clients. We added a TCP proxy to control the main functions of the TCP layer. It is responsible for “Packet rearrange” which can discard the expiry packets automatically according to the feedback information from the packet urgency estimation at the MAC layer and the dropped packets will not be put into the “Packet sending window.” At the MAC layer, we build a “ACK reconstruction module” to reconstruct the received ACK for concealing the discarded packets. With the aim of maximizing the sum utility of the scheduled packets, some packets with little utility will be discarded instead of wasting lots of time for their arrival at the client. At the client, we introduce a maximum allowed waiting time for each segment. The client does not need to wait until all the video packets arrive at the client. Then, based on the new transmission architecture, we can improve the QoE of DASH service by jointly optimizing the application layer of media server and client, TCP layer, MAC layer, and PHY layer. The strategy of packet scheduling, resource allocation, and segment request can be obtained in our new framework.

• A new packet importance estimation and update method based on the feedback ACK: We explore the interdependence among all the packets to indicate the importance of each packet. Specially, we update the interdependence relationship between the packets waiting to be scheduled and the packets already transmitted or lost based on the feedback ACK information. This update mechanism makes the packet importance more close to the real contribution that the packet can create. Moreover, we take into consideration the playback information to predict the packet urgency.

• A rate adaptation scheme based on both MAC queue and the client buffer status: We consider the MAC queue status in our rate adaptation scheme. We use the playback information of client to estimate the continuous playback time that the packets in MAC queue can support. Then, by considering the MAC queue, client buffer, and the estimated transmission rate, we can acquire the urgency of each client and determine when and how to send the new segment request.

The rest of the paper is organized as follows. Section 2.1 describes the system model, and Section 2.2 presents how to calculate the packet utility. Section 2.3 show the problem formulation and solution to our proposed packet scheduling and resource management scheme. Section 2.4 presents a rate adaption scheme to derive when and which presentation should be requested and selected. Section 3 provides the discussion and the experimental results of the proposed algorithms. Finally, we draw the conclusion in Section 4. We provide a list of abbreviations used in this paper as shown in Table 1

Table 1 A list of abbreviations

2 Methods

2.1 System Model

Figure 1 shows the framework of this paper. In this system, K video clients can request different video sequences they want. Each video sequence is pre-encoded into a series of video segments with different coding rates (different presentations) by employing the H.264/AVC encoder in advance. The segments are stored at the media server attached to eNodeB via a lossless wired network, and the clients can send the segment requests to the media server. The packet priority can be determined based on the information extracted from the encoder and the feedback ACK information.

Fig. 1

The proposed system framework

We build a TCP proxy to control the main functions of TCP layer. It is responsible for “Packet rearrange,” “ACK receiving window,” and “Packet sending window.” The module “Packet rearrange” can discard the expiry packets according to the feedback information from packet urgency estimation at the MAC layer automatically, and the dropped packets will not be put into the “Packet sending window.” In other words, these packets discarded by TCP have no chance to be scheduled and transmitted. The packets not expiry will be put into “Packet sending window” according to the receiving ACK information from the “ACK receiving window.”

At the MAC layer, the playback information can be estimated based on the feedback information from the clients. Considering the estimated playback information and the packet priority, we can calculate the importance of each packet. Based on the packet importance and the feedback CQIs (channel quality indicator), different RBs are assigned to different clients to maximize the system performance. We build a “ACK reconstruction module” to reconstruct the received ACK for concealing the discarded packets at the MAC layer. It depends on the received ACK information from the “ACK sending window” at the client and the packet scheduling strategy at the MAC layer. When we consider the packet utility during the resource allocation to maximize the sum utility of the scheduled packets, some packets with little utility will be discarded instead of wasting lots of time for their arrival at the client. The ACK of these discarded packets will be constructed and integrated with the received ACKs by the “ACK reconstruction module” and then sent to the “ACK receiving window” at the TCP layer. And thus, the dropped packets at the MAC layer are concealed during the TCP flow. In addition, at the MAC layer of eNodeB, we focus on the estimation of the playback time that the packets in MAC queue can support by using the estimated buffer status and the predicted transmission rate.

At the client, we build “ACK sending window” and “Packet receiving window” to simulate TCP flow. We give a maximum allowed waiting time for each segment. The packets violate its deadline will be discarded by the MAC layer and TCP layer. The exact buffer information helps us to easily get how long the remaining buffered frames in client buffer can take. Based on the analysis of the client buffer and MAC queue, an adaptive segment request strategy is developed to send each segment request at an appropriate time. Knowing when to send the segment request, a rate adaption scheme is proposed based on the estimated transmission rate and the status of the client buffer and MAC queue to provide the presentation which should be selected.

2.2 Packet utility estimation

Packet utility in our work depends on two factors: the packet priority determined by the encoding information and the packet urgency determined by the playback information. In this paper, the packet priority is calculated in terms of the potential degradation caused in the video quality due to loss of the packet. Since the same method is applied to all the video sequences, we omit the client index in the following analysis. Let λ denote the number of B frames inserted into two P frames. Let $D_{f,p}^{\text {loss}}$ denote the distortion caused by the loss of the packet p of frame f (f is the encoding index of the frame), which can be expressed by

$$ D_{f,p}^{\text{loss}} = D_{f,p}^{\text{rec}} + D_{f,p}^{\text{ref}} $$

(1)

It consists of two parts:

(1) $D_{f,p}^{\text {rec}}$ is the relative distortion caused by the reconstructed packet. Here, the distortion $D_{f,p}^{\text {rec}}$ can be obtained by using the method presented in [31].

(2) $D_{f,p}^{\text {ref}}$ is the relative distortion caused by the packets dependent on the current one. As shown in Fig. 2, if we set the encoding parameter Number Reference Frames=1, I or P frame will be directly referred according to the following principles:

• The following two B frames (f+1,⋯,f+λ) refer the frame f for backward prediction.

  Fig. 2

  

  The encoding reference relationship with λ=2 and Number Reference Frames=1

• The following P frame f+λ+1 refers the frame f for forward prediction.

• The following two B frames (f+λ+2,⋯,f+2λ+1) refer the frame f for forward prediction.

Let γ _f,p be the sum times that each of the pixels of packet p of frame f is referenced. To calculate γ _f,p , the information of each packet such as MB (macroblock) partition modes and MVs should be extracted from the encoder. H.264/AVC encoder supports the intra-coded and inter-coded prediction modes and also provides abundant MB partition modes. Intra-coded prediction has 16×16 and 4×4 partition modes, and the inter-coded prediction mode possesses seven MB partition modes, namely 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4 as shown in Fig. 3. Let $\psi _{\eta,\theta }^{\widetilde {f},f,p}$ denote the set of the pixels of packet p of frame f referenced by the SB θ of the MB η of the frame $\widetilde {f}$. The calculation of $\psi _{\eta,\theta }^{\widetilde {f},f,p}$ is described in Table 2.

Fig. 3

The coordinate system of one frame

Table 2 Calculation of $\psi _{\eta,\theta }^{\widetilde {f},f,p}$

Note that I frame is directly referenced by its following P frame. If the P frame is not the last P frame in this GoP, it is also referenced by its following P frame. Except for the first P frame, other P frames refer I frame undirectly. Consequently, the loss of I or P frame will cause the quality degradation of all the following frames in the same GoP. For P frame, the smaller frame encoding index is the more important. In this paper, the referenced times of each packet is calculated by considering both direct and undirect reference modes. Let β specify the total number of I and P frames in one GoP. Any I or P frame is given another index called reference frame index to indicate its index among all the I and P frames in a GoP. Let π _f denote the reference frame index of frame f, 0≤π _f ≤β−1. Let ζ(∙) denote the number of the elements of the input set. For the packet p of frame f:

• If π _f =β−1, frame f is the last P frame in its GoP and it is only referenced by its following B frames in direct mode. γ _f,p can be computed by

  $$ \gamma_{f,p}=\sum\limits_{\widetilde{f}=f+1}^{f+2\lambda+1}{ \sum\limits_{\eta,\theta}{\zeta\left(\psi_{\eta,\theta}^{\widetilde{f},f,p}\right)}} $$

  (2)

• If π _f <β−1, frame f may be referenced by its following B frames and P frames in direct or undirect mode. γ _f,p can be computed by

  $$ \begin{array}{l} \gamma_{f,p} = \underbrace{\sum\limits_{\widetilde{f} = f + 1}^{f + 2\lambda + 1} { \sum\limits_{\eta,\theta} {\zeta\left(\psi_{\eta,\theta }^{\widetilde{f},f,p}\right)} }}_{\mathrm{direct\;reference\;mode}} \\ + \underbrace{\sum\limits_{\widetilde{f} = f + \lambda + 2}^{f + \lambda (\beta - \pi_{f}) + \beta - \pi_{f} - 1} { \sum\limits_{\eta,\theta} {\zeta\left(\widehat{\psi}_{\eta,\theta }^{\widetilde{f},f,p} \right)} }}_{\mathrm{undirect\;reference\;mode}}\\ \end{array} $$

  (3)

where $\widehat {\psi }_{\eta,\theta }^{\widetilde {f},f,p}$ is the region of pixels of packet p of frame f referenced undirectly by SB θ of MB η frame f. To calculate $\widehat {\psi }_{\eta,\theta }^{\widetilde {f},f,p}$, we need to calculate the reference region of frame $\widehat {f}$ directly referenced by frame $\widetilde {f}$, $\phi _{\eta,\theta }^{\widetilde {f},\widehat {f}}$ first. Then, according to all the reference relationships between the undirect reference frames such as $\Psi ({\widehat {f},\widehat {f}-\lambda -1})$, ⋯, Ψ(f−λ−1,f), $\widehat {\psi }_{\eta,\theta }^{\widetilde {f},f,p}$ can be obtained finally, where Ψ(f ₁,f ₂) is the set of the reference relationships for all the SBs between frame f ₁ and f ₂, which can be easily acquired by using the method presented in Table 2. Based on the above analysis, $D_{f,p}^{\text {ref}}$ can be computed by

$$ D_{f,p}^{\text{ref}} = \left \{ \begin{array}{l} D_{f,p}^{\text{rec}} \frac{\gamma_{f,p}}{\zeta(\varphi_{f,p})}, \text{if}\; A_{\hat{f},\hat{p}} = 1\\ D_{f,p}^{\text{con}} \frac{\gamma_{f,p}}{\zeta(\varphi_{f,p})}, \text{if}\; A_{\hat{f},\hat{p}} = 0\\ \end{array} \right. $$

(4)

where $A_{\hat {f},\hat {p}}$ is a binary variable which indicates whether the packet $\hat {p}$ of frame $\hat {f}$ is received or lost. $A_{\hat {f},\hat {p}}=1$ is equal to the successful delivery and $A_{\hat {f},\hat {p}}=0$ otherwise. The value of $A_{\hat {f},\hat {p}}$ depends on the feedback ACK information. $D_{f,p}^{\text {con}}$ is the distortion caused by the loss of its reference packet $\hat {p}$ of frame $\hat {f}$. Finally, $D_{f,p}^{\text {loss}}$ can be obtained according to (1). In the following, $D_{f,p}^{\text {loss}}$ is regarded as the priority of packet p of frame f and at the RLC (radio link control) layer of LTE system, each video packet is fragmented into many small packets. It is assumed that the packets attributed to the same packet have the same priority. In MAC queue, we can acquire the packet priority from the packet header denoted by pri _k,m .

We use the feedback information such as ACKs of the received packets from the clients to predict the playback information and then examine the packet urgency. Let $\widetilde {F}_{c}(k)$ denote the estimated index of the frame being played of client k. Since the frames of one segment should be decoded simultaneously, the packet urgency of all the video packets of the frames of the same segment should be the same. Thus, the packet urgency μ _k,m can be expressed by

$$ \mu_{k,m} = P_{k,m}-\widetilde{F}_{c} (k) $$

(5)

where P _k,m is the index of the first frame of the segment waiting in the MAC queue. For the packets of the same segment, the values of P _k,m are the same. By employing the method proposed in [3], the scheduling probability of packet m for client k, ρ _k,m , can be defined as $\rho _{k,m} = e^{-\mu _{k,m}}\phantom {\dot {i}\!}$.

Then, considering the packet priority and the packet urgency, the utility of packet m of client k, U _k,m , can be calculated by

$$ \begin{array}{l} U_{k,m} = e^{-\mu_{k,m}}\times \text{pri}_{k,m} \end{array} $$

(6)

2.3 Problem formulation and solution

2.3.1 Problem formulation

We formulate the packet scheduling and resource management into a new mathematical model with the objective to maximize the weighted sum of utilities of all the scheduled packets of all the clients:

$$ \begin{array}{l} \text{max} \;\sum\limits_{k = 1}^{K} {w_{k}{\sum\limits_{m = 1}^{M_{k}} {U_{k,m} \tau_{k,m}}}} \\ s.t. \\ (c1)\sum\limits_{n \in \Omega} a_{k,n}{\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j} r_{j}}} \ge \sum\limits_{m = 1}^{M_{k}} {\tau_{k,m} R_{k,m} }, 1\leq k \leq K\\ (c2)\tau_{k,m} \in \{ 0,1\}, 1 \leq m \leq M_{k}, 1\leq k \leq K \\ (c3)\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j}} = 1,\;b_{k,j} \in \{ 0,1\}, 1\leq k \leq K\\ (c4)\sum\limits_{k = 1}^{K} {a_{k,n}} = 1,\;a_{k,n} \in \{ 0,1\}, 1\leq n\leq N \\ \end{array} $$

(7)

where

• Ω: the set of the total available RBs.

• N: the number of the available RBs in this system.

• M _k : the number of the packets in client k’s MAC queue.

• w _k : client k’s weight, a positive constant calculated by $w_{k}=\sum \limits _{k = 1}^{K }(\widetilde {N}_{b}(k))/\widetilde {N}_{b}(k)$, such that $\sum \limits _{k = 1}^{K }(w_{k})=1$. $\widetilde {N}b(k)$ is the predicted number of frames waiting to be played at client buffer. The aim is to balance the playback continuity experience among the clients. $\widetilde {N}b(k)$ can be predicted by using the feedback ACK information with RLC mode AM. It can also be fedback from the client by employing the playback buffer status analysis module at client.

• τ _k,m : τ _m,k =1 means the packet m of client k is selected to be scheduled and τ _m,k =0 otherwise.

• a _k,n : a _k,n =1 indicates the RB n is assigned to client k and a _k,n =0 otherwise.

• N _k : the set of RBs assigned to client k.

• q(N _k ): the maximum MCS (Modulation and Coding Scheme) that client k can support based on N _k .

• b _k,j : b _k,j =1 means client k uses MCS j and b _k,j =0 otherwise.

• r _j : the transmission capacity of each RB by using MCS j.

• R _k,m : the size of the packet m of client k.

Constraint (c1) shows that the transmission capacity of client k should be enough to transmit its scheduled packets. We assume that all the scheduled packets can be transmitted to the client successfully by advanced channel coding. Constraint (c2) indicates that each packet can be scheduled or not at each scheduling period. Constraint (c3) means that each client can only choose one MCS. The last constraint implies that one RB can only be assigned to one client.

2.3.2 Packet scheduling and resource management scheme

From the mathematical expression in Eq. (7), the relationship between the receive time and other parameters cannot be explicitly evaluated without restrictive assumptions on the arrival process and the time-varying channel states. Thus, we can not obtain a mathematically optimal solution of the above problem in its most generic form. Due to this, we develop a heuristic suboptimal solution, which jointly performs RB assignment and packet scheduling by employing our proposed utility function.

To solve the problem (7), a cost function C _k,n is developed to present the increment of the sum utilities of the clients brought by assigning RB n to client k. C _k,n can be expressed by

$$ \begin{array}{l} {}C_{k,n} = f(\tau_{k,m},a_{k,n},b_{k,j},N_{k} \cup \{n\})-f(\tau_{k,m},a_{k,n},b_{k,j},N_{k})\\ \end{array} $$

(8)

where N _k is the set of the RBs already assigned to client k. The function f(∙) can be expressed as

$$ \begin{array}{l} f(\tau_{k,m},a_{k,n},b_{k,j},N_{k})= \text{max} \;{w_{k}{\sum\limits_{m = 1}^{M_{k}} {U_{k,m} \tau_{k,m}}}} \\ s.t. \\ (c1)\sum\limits_{n \in N_{k}} a_{k,n}{\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j} r_{j}} } \ge \sum\limits_{m = 1}^{M_{k}} {\tau_{k,m} R_{k,m} }\\ (c2)\tau_{k,m} \in \{ 0,1\}, 1 \leq m \leq M_{k} \\ (c3)\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j}} = 1,\;b_{k,j} \in \{0,1\}\\ \end{array} $$

(9)

Achieving problem (7), we should assign RB n to the client with the maximum C _k,n . To get C _k,n , we need to solve f(τ _k,m ,a _k,n ,b _k,j ,N _k ) first. Next, we will provide how to calculate f(τ _k,m ,a _k,n ,b _k,j ,N _k ).

For a certain client with fixed RB set, the utility is an increasing function of the transmission capacity. The maximum capacity these RBs can support can be calculated by solving the following problem:

$$ \begin{array}{l} \text{max} \sum\limits_{n \in N_{k}} a_{k,n}{\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j} r_{j}}} \\ s.t. \\ (c1)\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j}} = 1,\;b_{k,j} \in \{ 0,1\}\\ (c2)\sum\limits_{k = 1}^{K} {a_{k,n}} = 1,\;a_{k,n} \in \{0,1\}, \{n\} \in N_{k} \\ \end{array} $$

(10)

Then, we can get the optimal $\left (a_{k,n}^{*},b_{k,j}^{*}\right),$ and then the transmission capacity $\sum \limits _{n \in N_{k}} a_{k,n}^{*}{\sum \limits _{j = 1}^{q(N_{k})} {b_{k,j}^{*} r_{j}} }$ can be obtained according to [32]. Knowing the transmission capacity, the optimal packet scheduling strategy $\tau _{k,m}^{*}$ can be obtained by solving the following problem:

$$ \begin{array}{l} \text{max}\;w_{k}\sum\limits_{m = 1}^{M_{k}} {{U_{k,m} \tau_{k,m}} }\\ s.t. \\ (1)\sum\limits_{n \in N_{k}} a_{k,n}^{*}{\sum\limits_{j = 1}^{q(N_{k})} {b_{k,j}^{*} r_{j}} } \ge \sum\limits_{m = 1}^{M_{k}} {\tau_{k,m} R_{k,m} }\\ (2)\tau_{k,m} \in \{ 0,1\}\\ \end{array} $$

(11)

Optimal $\tau _{k,m}^{*}$ can be easily acquired by our method proposed in [33]. Then, we can have

$$ \begin{array}{l} f\left(\tau_{k,m}^{*},a_{k,n}^{*},b_{k,j}^{*},N_{k}\right)=\sum\limits_{m = 1}^{M_{k} }{\tau_{k,m}^{*}} \\ \end{array} $$

(12)

The detailed process of the packet scheduling and resource management scheme is described in Table 3.

Table 3 Packet scheduling and resource management scheme

2.4 Adaptive segment request scheme

In this section, an adaptive segment request strategy is proposed to determine when to send the segment request and which presentation should be selected.

2.4.1 Segment request time determination

The segment request time determination is carried out at the client side, so we can get the accurate number of the unplayed but successfully decoded frames stored in client k’s playback buffer N _b (k) and how long continuous playback these frames can support T _b (k)=N _b (k)/F _pr , where F _pr is the frame rate. Since some packets in the MAC queue can be transmitted to the client side during T _b (k) and therefore can bring extra continuous playback time, immediately sending the new segment request can not bring benefit to the client but may occupy the wireless resource which is really needed by other clients. Letting the packets in MAC queue are the ones that are really needed by the clients is the most important.

Let T _mac(k) denote how long the continuous playback time interval that the packets in MAC queue can support and R P _k denote the set of the packets in MAC queue of client k after the estimation. The estimation of T _mac(k) is shown in Table 4.

Table 4 Procedure of estimation of T _mac(k)

If R P _k =∅, it means that all the packets in the MAC queue can arrive at the client on time and the client can enjoy T _mac(k)+T _b (k) continuous playback back at least. If R P _k ≠∅, it means that the client k will suffer interruption after the time interval T _mac(k)+T _b (k). Let T _l (k) denote the playback time that the remaining packets in MAC queue will take, which can be calculated by T _l (k)=N _l (k)/F _pr , where N _l (k) is the number of the remaining frames in the MAC queue. The transmission time they will take can be estimated by $T_{r}(k)=\text {SM}(k)/\widehat {r}_{k}$, where SM(k) indicates the sum size of the left packets in the MAC queue. A guard time interval T _g (k) is introduced to protect the client against suffering playback interruption. The appropriate time interval needed to send the next segment request, T _i (k), can be computed by

$$ T_{i} (k) = \text{max} (T_{\text{mac}} (k) + T_{b} (k) + T_{l} (k) +T_{r}(k)- T_{g} (k),0) $$

(13)

2.4.2 Rate adaption at client

With the time when to send the segment request known, we need to determine which presentation should be selected. To keep the continuous playback, the next requested segment should be transmitted to the client during the time we estimated above. Based on the estimated transmission rate, we can determine the optimal coding rate we should request. With the optimal coding rate known, we can select the segment with most closest coding rate. Using the metadata contained in MPD (Media Presentation Description), the unique URL of the requested segment can be constructed. Then, the request will be sent to the media server by HTTP GET command. The media server will pick the segment with the unique URL and send it to the DASH client.

Let S _mac(k) denote the sum size of the packets in client k’s MAC queue, $S_{\text {mac}}(k)=\sum \limits _{m=1}^{M_{k}}{R_{k,m}}$. The transmission time taken by those packets, $\widehat {T}_{\text {mac}}(k),$ can be estimated by

$$ \widehat{T}_{\text{mac}}(k) = \frac{S_{\text{mac}}(k)}{\widehat{r}_{k}} $$

(14)

There are two possible transmission cases according to the value of T _l (k):

• If T _l (k)=0, all the packets in MAC queue can be transmitted to the client during the time interval T _b (k)+T _mac(k). Therefore, to keep the continuous playback of client k, the next segment should be transmitted to client during the interval $T_{b}(k)+T_{\text {mac}}(k)-\widehat {T}_{\text {mac}}(k)$. Let T _seg(k) be the duration of a segment for client k. Then, the coding rate that the client should recommend can be computed by

  $$ {\text{pr}}_{k}=\widehat{r}_{k} \frac{T_{b}(k)+T_{\text{mac}}(k)-\widehat{T}_{\text{mac}}(k)}{T_{\text{seg}}(k)} $$

  (15)

• If T _l (k)>0, client k would suffer interruption after the time interval T _b (k)+T _mac(k). Since the left packets also take some time $\widehat {T}_{l}(k)$ to be transmitted and played, the next segment should arrive at client during the time interval $\widehat {T}_{l}(k)$. Then, the recommended coding rate can be acquired by

  $$ {\text{pr}}_{k}=\widehat{r}_{k} \frac{T_{l}(k)}{T_{\text{seg}}(k)} $$

  (16)

In general, the recommended coding rate can be determined by

$$ \text{pr}_{k} = \left\{ \begin{array}{l} \widehat{r}_{k} \frac{T_{l}(k)}{T_{\text{seg}}(k)},\;if\;T_{l} (k)>0 \\ \widehat{r}_{k} \frac{T_{b}(k)+T_{\text{mac}}(k)-\widehat{T}_{\text{mac}}(k)}{T_{\text{seg}}(k)},\;\text{otherwise} \\ \end{array} \right. $$

(17)

Obviously, the recommended coding rate may be not equal to the real coding rate of the segments stored at the media server. Thus, the client should determine which segment should be selected based on the recommended rate. In this paper, we will pick the representation with the most close coding rate to pr _k from all the pre-encoded representations. The encoding information of the representation can be found in MPD.

Assume that there be ε _k representations of the video sequence requested by client k and let PR _k indicate the set of those representations, $\phantom {\dot {i}\!}\text {PR}_{k}=\{\text {PR}_{k,1}, \cdots, \text {PR}_{k,\varepsilon _{k}}\}$ with PR _k,i <PR _k,i+1. If the recommended rate pr _k is lower or higher than the coding rates of all the encoded representations, the DASH client will pick the representation with the lowest or the highest coding rate of PR _k . The principle of the final decision on the selected representation sp _k can be summarized as follows:

$$ \text{sp}_{k} = \left\{ \begin{array}{l} \text{PR}_{k,1},\;\text{if}\;\text{pr}_{k} < \text{PR}_{k,1}\\ \text{PR}_{k,i},\;\text{if}\;\text{PR}_{k,i} \leq \text{pr}_{k} < \text{PR}_{k,i+1}\\ \text{PR}_{k,\varepsilon_{k}},\;\text{if}\;\text{pr}_{k} \geq \text{PR}_{k,\varepsilon_{k}}\\ \end{array} \right. $$

(18)

From Eq. (18), we can determine which segment should be requested. As long as the segment going to be requested is obtained, the unique URL of the segment can be constructed by using the metadata contained in MPD. Then, the request will be sent to the media server by HTTP GET command and the media server will pick the segment with the unique URL and send it to the DASH client.

3 Results and discussion

We provide our experimental results and discussion in this section. All the experiments are carried out in LTE system with the Network Simulation 3 (NS3) simulator [34]. eNodeB can obtain the CQIs through the feedback of all UEs. Each scheduling period called TTI (Transmission Time Interval) consists of two consecutive RBs and the time slot duration of each RB is 0.5 ms. Therefore, CQIs will be periodically updated and transmitted to eNodeB in the feedback information every 1 ms. It is assumed that the power be equally allocated among the RBs. Since we need the ACK information to predict the playback buffer status of the DASH client, the RLC mode is set to be AM (acknowledged mode). All the video sequence have been encoded into a series of the video segments with five different coding rates (presentations). Each segment consists of 60 frames with the time duration of 2 s. The detailed system configuration is shown in Table 5. The encoding information about the average coding rate, PSNR, and the corresponding presentation is shown in Table 6. For each segment, there is no need to receive all the video packets especially the packets with little effect on the received video quality. In our simulation, each video packet should be transmitted to the client before its deadline which depends on the maximum allowed waiting time of each video packet. We define T a _k as the maximum allowed waiting time for the packet of client k. It means that from the time the packet should be started to be played, the client only wait T a _k for the arrival of the video packet at most. In our simulation, T a _k =5 s,1≤k≤K. Using this parameter, we can determine which packet has the chance to be transmitted and which should be discarded automatically. The clients can dynamically switch their request presentations at the times that they send their segment requests.

Table 5 Description of the experimental environment

Table 6 The encoding information of different video sequences

“PROPOSED” is the algorithm which consists of both the proposed packet scheduling and resource management scheme. “AR” (adaptive request) only indicates the adaptive segment request scheme. When AR is not used at the client, the clients will send the segment request with a fixed time interval and the rate adaption scheme is to request the segment whose rate is the most closest to the estimated transmission rate. The algorithm proposed in this paper will be compared with the following four algorithms:

• QTRMA in [15]: It investigated QoE optimization approaches for adapting the adaptive HTTP video delivery. They considered the client buffer status and proposed a novel playback buffer-dependent approach that determines for each client the streaming rate for future video segments according to its buffer time and the achievable QoE under current radio conditions.

• ADCLA in [18]: It developed a packet importance model based on the information extracted from the compressed video and proposed a content-aware scheduling scheme for resource allocation. The waiting time in MAC queue is used to calculate the packet urgency.

• MAXCI in [35]: It is a purely channel-dependent scheduling scheme, in which the clients with good channel states always occupy the channel most of time even if they are well ahead of their video decoding deadlines.

• PF in [36]: It allocates the limited network resources by considering both the experienced channel states and the past client throughput. Its goal is to maximize the system throughput and to guarantee fairness among end clients.

In Fig. 4, we provide the average PSNR of each frame of all the clients for different algorithms respectively. From this figure, we can see that our proposed algorithm can outperform other existing algorithms for all the clients. This indicates that considering the packet importance and the packet urgency can make the packet scheduling and resource allocation strategy more reasonable. Due to taking into consideration the packet importance, ADCLA+AR has higher PSNR than MAXCI+AR and PF+AR. We can also observe that QTRMA has more stable performance than other algorithms. This is because for QTRMA, there is no packet loss, the client will wait until all the video packets of each segment arrive at the client. No packet loss means not so serious quality fluctuation during one segment. But transmitting all the video packets may lead to requesting the segment with low bit rate which will also degrade the value of the average PSNR. In fact, the packet with little importance should be discarded not scheduled.

Fig. 4

Average PSNR of all the clients for different algorithms

Figures 5 and 6 shows the corresponding playback time of each frame of each client and the average playback time of each frame of all the clients. In the simulation, each client should experience a rebuffering time interval until all the packets of two segments are received. From the two figures, we can see that some continuous playback can be guaranteed for all the clients with different algorithms. From Fig. 6, we can deserve that the client can finish the whole video in a shortest time among all the clients by employing our proposed method. It means that the clients suffer the fewest rebuffering events and shortest interruption time which brings most satisfactory playback experience to the clients. From Fig. 5, we can discover that the performance of our proposed method is not always the best among the algorithms for each frame. This is because the objective of our proposed algorithm is to maximize the sum of the utilities of all the clients not aiming at optimizing an individual client. We can also see that the algorithm QTRMA yields longer interruption time than the PROPOSED and ADCLA+AR. This is because QTRMA has to start to play until all the video packets of each segment arrive at the client while the former algorithms will wait for only T a _k . It is unreasonable to wait for the arrival of the packets with little importance especially in a wireless network with a bad channel state.

Fig. 5

Playback time of each frame of each client for different algorithms

Fig. 6

Average playback time of all the clients for each frame for different algorithms

To examine the performance of the packet scheduling strategy of our proposed scheme, we present the average packet loss rate of the clients for each frame in Fig. 7. From the figure, we can see that the packet loss rate can be kept to zero for some time because a rebuffering process exists for all the videos at the beginning of the playback. In addition, the packet loss rate can be restricted in a lowest range by using our proposed algorithm. The number of the frames with no packet loss is larger than other compared algorithms too. Lower packet loss is equal to the higher PSNR especially when we consider the packet utility in the packet scheduling and resource management. The packet with higher utility will be scheduled with higher priority. Reasonable packet scheduling strategy can improve the received video quality by allocating the limited resource to the clients who really need. Together with the results in Figs. 4 and 6, we can conclude that our proposed algorithm can acquire higher video quality as well as more continuous playback experience.

Fig. 7

Average packet loss rate of all the clients for each frame for different algorithms

The simulation result of the average PSNR of all the clients for each frame for different packet scheduling and resource management algorithms integrating with our proposed segment request scheme AR is shown in Fig. 8, and the interruption time experienced by each client by employing our proposed segment request scheme AR is shown in Fig. 9. Note that “PROPOSED-AR” means the resource allocation scheme used is the algorithm proposed in this paper but the segment will be requested with a fixed time interval. Since QTRMA has its unique segment request scheme, we did not employ AR for it. From Fig. 8, we can see that the video quality based on AR is higher than that without AR for the most of the frames. This is because the segment adaption scheme can make the segment switch more flexible and adaptive to the varying channel states. Without rate adaptation, the rate of the requested segment may exceed the transmission rate that the wireless channel can support. By using AR, the limited resource can be allocated to the packets which are suitable to be transmitted. As shown in Fig. 9, AR can bring the playback experience improvement apparently for all the packet scheduling and resource management schemes. Different schemes earn different achievements. In summary, QoE of the clients can be improved by employing AR.

Fig. 8

The comparison of the average PSNR of all the clients for each frame for different packet scheduling and resource management algorithms by employing our proposed segment request scheme AR or not. a Average PSNR of all the clients for each frame for PROPOSED vs. PROPOSED-AR. b Average PSNR of all the clients for each frame for ADCLA+AR vs. ADCLA. c Average PSNR of all the clients for each frame for MAXCI+AR vs. MAXCI. d Average PSNR of all the clients for each frame for PF+AR vs. PF

Fig. 9

The comparison of interruption time experienced by each client for different packet scheduling and resource management algorithms by employing our proposed segment request scheme AR or not. a Interruption time experienced by each client for PROPOSED vs. PROPOSED-AR. b Interruption time experienced by each client for ADCLA+AR vs. ADCLA. c Interruption time experienced by each client for MAXCI+AR vs. MAXCI. d Interruption time experienced by each client for PF+AR vs. PF

4 Conclusions

A content and buffer status aware packet scheduling and resource management framework is proposed for DASH video streaming over LTE system. We first modify the transmission flow to make the whole framework tolerant of the loss of unimportant packets. Then, depending on the encoding information extracted from the encoder, we calculate the priorities of the video packets. Based on the feedback information from the clients we can estimate the packet urgency. The priorities of the video packets, the feedback channel states and the packet urgencies are considered to optimize the packet scheduling and RB assignment. Instead of requesting the video segments with fixed time interval, we propose an adaptive segment request time determination strategy which can effectively control the status of the client buffer and MAC queue so as to reduce the packet loss rate caused by the overflow of the client buffer and MAC queue. To get a better trade-off between the received video quality and playback continuity, we further propose a novel rate adaption algorithm which depends on the status of the client buffer, the MAC queue and the predicted transmission rate. Numerical simulation results show that our proposed client buffer and content aware packet scheduling framework can provide superior performance in terms of the PSNR of the received video and playback continuity.