1.

Introduction

In general inter-predicted video coding, the motion vector (MV) provides a spatial offset of a block in the current picture to a block in the reference picture. Therefore, more MV information is needed to improve the accuracy of the inter prediction. To minimize the number of bits required to represent MV information, the H.264/AVC standard applies a predictive coding method by using predictive motion vectors (PMVs), which are calculated as the median of three spatially neighboring MVs.¹ The median PMV is effective at reducing the required number of compressed MV bits, since it is very similar to the MV in most cases. However, the median PMV is not always optimal for minimizing the number of MV bits. If a more precise PMV exists than the median PMV, there is a chance that even more bits can be saved.

To overcome this problem, several approaches have been taken.^{2, 3, 4} In Chen and Willson’s work,² MVs that are located in spatially and temporally neighboring blocks are additionally considered to select a more precise PMV. A spatial or temporal PMV is selected according to the value of the predictors. However, this method cannot ensure selection of the optimal PMV. In Kim and Ra’s work,³ an optimal PMV is selected by using a distance measure function between the MV and PMV. In this method, however, additional information is required to decide which candidate PMV is an optimal one. Finally, in Laroche ’s work,⁴ several candidate PMVs are generated by a combination of spatial and temporal neighboring MVs. A more precise PMV is then selected by a rate distortion (RD) competing scheme. As in Kim and Ra’s work,³ in some cases this method requires additional information to determine which predictor is to be used. This means that the benefit of using the optimal PMV may not be fully realized, because the choice of the optimal PMV by the encoder must be signaled to the decoder.

Therefore, in this paper, we propose a new motion vector coding method with optimal PMV selection (MVOP) to use the optimal PMV without the need for additional signaling information. First, the encoder defines the set of possible candidate PMVs by using neighboring MVs. To minimize bits of the MV information, an optimal PMV is selected among the candidate PMV set. If the decoder can estimate the optimal PMV by using decoder-side estimation, the encoder selects it as an optimal PMV. Otherwise, the encoder selects the median PMV in the same manner as the H.264/AVC standard. In the worst case of the proposed method, only $1\text{bit}$ of additional information is required to signal whether the decoder can estimate the optimal PMV or not. Simulation results show that the proposed method reduces the average Bjontegaard delta bit rate (BDBR) by about 2.97% and increases the average Bjontegaard delta peak signal-to-noise ratio (BDPSNR) by about $0.14\mathrm{dB}$ compared with the H.264/AVC standard.

2.

Proposed Method

2.1.

PMV Candidate Set

As shown in Fig. 1, the candidate set (CS) is defined to select a more precise PMV than the median PMV. The CS, which is a group of possible and distinct candidate PMVs for the current block, is composed of a combination of horizontal and vertical components of spatial neighboring MVs. The CS is defined by

Eq. 1

$$\mathrm{CS}=\text{combination}\text{of}\{\mathbf{m}{\mathbf{v}}^L,\mathbf{m}{\mathbf{v}}^U,\mathbf{m}{\mathbf{v}}^R\}=\{({\mathrm{mv}}_x^L,{\mathrm{mv}}_y^L),({\mathrm{mv}}_x^L,{\mathrm{mv}}_y^U),\dots ,({\mathrm{mv}}_x^R,{\mathrm{mv}}_y^U),({\mathrm{mv}}_x^R,{\mathrm{mv}}_y^R)\},$$

where ${\mathrm{mv}}_x^L$ and ${\mathrm{mv}}_y^L$ are the horizontal and vertical components of $\mathbf{m}{\mathbf{v}}^L$ , respectively. In this paper, spatially neighboring MVs are considered in the CS in the same manner as in the H.264/AVC standard.

Fig. 1

Candidate PMV set.

2.3.

Optimal PMV Estimation at the Decoder Side

To estimate an optimal PMV at the decoder with known information of the DMV $\mathbf{d}\mathbf{m}{\mathbf{v}}^C$ , template matching⁵ is applied with a matching criterion function $g(\cdot )$ :

Eq. 4

$$g(\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C\mid \mathbf{d}\mathbf{m}{\mathbf{v}}^{\mathbf{C}})=\sum _{i∊\mathrm{TMS}}{[\mathrm{Ref}(\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C+\mathbf{d}\mathbf{m}{\mathbf{v}}^C,i)-\mathrm{Cur}\left(i\right)]}^2,$$

where Ref refers to the reference picture and Ref $(\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C+\mathbf{d}\mathbf{m}{\mathbf{v}}^C,i)$ denotes a pixel indexed by $i$ with respect to location $\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C+\mathbf{d}\mathbf{m}{\mathbf{v}}^C$ at the reference picture. The index $i$ indicates a pixel location at the template matching set (TMS), and Cur $\left(i\right)$ denotes a value at the location indexed by $i$ in the current picture. The TMS is a set of spatially adjacent upper, diagonal, and left regions around the given block as shown in Fig. 2.

Fig. 2

Decoder-side PMV estimation using template matching.

In the decoding process, all possible PMV candidates in the CS are tested by template matching those pixels indicated by the TMS to find the optimal PMV having the minimum matching error as

Eq. 5

$$\mathbf{p}\mathbf{m}{\mathbf{v}}^{C\left(\mathrm{dec}\right)}=\underset{\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C∊\mathrm{CS}}{\arg \min }g(\mathbf{p}\mathbf{m}\mathbf{v}{\mathbf{c}}^C\mid \mathbf{d}\mathbf{m}{\mathbf{v}}^C).$$

Note that this derivation can also be made in the encoder by the decoding loop. If the following condition is satisfied, the decoder can estimate the optimal PMV autonomously at the decoder side:

Eq. 6

$$\mathbf{p}\mathbf{m}{\mathbf{v}}^{C\left(\mathrm{dec}\right)}=\mathbf{p}\mathbf{m}{\mathbf{v}}^{C\left(\mathrm{opt}\right)}.$$

In this case, it is sufficient to signal the decoder to find the optimal PMV autonomously to reconstruct the MV out of the DMV, which avoids extra bit transmission to signal a particular PMV, in contrast to Laroche ’s work.⁴

3.

Experimental Results

To evaluate the performance of the proposed method, we modified the reference software of the H.264/AVC standard. Joint model (JM) version 12.2 reference software was used for modification and comparison. All sequences [“Coastguard,” “Foreman,” “Carphone,” and “TableTennis” (QCIF, $15\mathrm{Hz}$ ) and “Coastguard,” “Foreman,” “Paris,” and “TableTennis” (CIF, $30\mathrm{Hz}$ )] have their first 300 frames encoded with four quantization parameters (QPs) of 28, 32, 38, and 40. To obtain a more precise MV, no fast motion estimation process was used. The performance of the proposed method was evaluated in terms of the BDBR and BDPSNR.⁶ Those quantities give the average bit rate and PSNR difference of the proposed method compared to the H.264/AVC standard, which always uses median PMV to encode MVs.

Table 1 shows the BDBR and BDPSNR of the proposed method compared to the H.264/AVC standard. As described in Table 1, the experimental results show that the proposed method decreases the number of bits compared with the H.264/AVC standard by about 2.97% on average. This is because the proposed method selects a more precise PMV than the H.264/AVC standard. In particular, some sequences with fast, nonlinear motion such as “Foreman” and “TableTennis” show better performance when a precise PMV is used. With higher QP values, the proposed method works better. This is because motion vector takes a larger bit portion at the lower bit rate, and thus there is more space to improve the coding efficiency.

Table 1

Coding performance of the proposed method.

4.

Conclusions

In this paper, we have proposed a new motion vector coding method using optimal PMV selection. By selecting the optimal PMV, which requires minimal bits to encode MV information, the proposed method can decrease the number of compressed MV bits compared with H.264/AVC. In particular, the proposed method is effective on sequences with fast and nonlinear motion activities. If more candidate PMVs are used, the proposed method can be even more effective without requiring additional signaling information.