Narrow Results

Video coding is an imperative part of the modern day communication system. Furthermore, it has vital roles in the fields of video streaming, multimedia, video conferencing and much more. Scalable Video Coding (SVC) is an emerging research area, due to its extensive application in most of the multimedia devices as well as public demand. The proposed coding technique is capable of eliminating the Spatio-temporal regularity of a video sequence. In Discrete Bandelet Transform (DBT), the directions are modeled by a three-directional vector field, known as structural flow. Regularity is decided by this flow where the data entropy is low. The wavelet vector decomposition of geometrically ordered data results in a lesser extent of significant coefficients. The directions of geometrical regularity are interpreted with a two-dimensional vector, and the approximation of these directions is found with spline functions. This paper deals with a novel SVC technique by exploiting the DBT. The bandelet coefficients are further encoded by utilizing Set Partitioning in Hierarchical Trees (SPIHT) encoder, followed by global thresholding mechanism. The proposed method is verified with several benchmark datasets using the performance measures which gives enhanced performance. Thus, the experimental results bring out the superiority of the proposed technique over the state-of-arts.

Versatile video coding (VVC) aims to achieve high compression but also issues like varying content/network conditions. Existing rate control (RC) methods struggle to achieve optimal quality under these complex scenarios. This paper proposes a novel RC scheme for VVC based on a linear model. The Lagrange minimization multiplier is introduced under bit budget constraints, allowing optimized bit allocation. RC optimization is formulated as a convex solution, and is derived into the optimal quantization parameter (QP) for RC. Experimental analysis demonstrates the proposed linear model-based RC algorithm performances are better compared to other state-of-the-art methods due to their use of a linear model and optimal QP determination.

This paper presents a new Computational-RAM (C-RAM) architecture for real-time mesh-based video motion tracking. The motion tracking consists of two operations: mesh-based motion estimation and compensation. The proposed motion estimation architecture is presented in Part 1 and the proposed motion compensation architecture is presented in Part 2. The motion estimation architecture stores two frames and computes motion vectors for a regular triangular mesh structure as defined by MPEG-4 Part 2.¹ The motion estimation architecture uses the block-matching algorithm (BMA) to estimate the vertical and horizontal motion vectors for each mesh node. Parallel and pipelined implementations have been used to overcome the huge computational requirements of the motion estimation process. The two frames are stored in embedded S-RAMs generated with Virage™ Memory Compiler. The proposed motion estimation architecture has been prototyped, simulated and synthesized using the TSMC 0.18 μm CMOS technology. At 100 MHz clock frequency, the proposed architecture processes one CIF video frame (i.e., 352×288 pixels) in 1.48 ms, which means it can process up to 675 frames per second. The core area of the proposed motion estimation architecture is 24.58 mm² and it consumes 46.26 mW.

This paper presents a new Computational-RAM (C-RAM) architecture for real-time mesh-based video motion tracking. In Part 1, the motion estimation part of the proposed architecture is presented. Here in Part 2, a new C-RAM mesh-based motion compensation architecture is presented. The input data to the architecture is the mesh nodes motion vectors and the reference frame and the output data is the compensated (i.e., predicted) frame. The architecture uses the affine transformation for warping the deformed patches in the reference frame into the undeformed patches in the current frame. The architecture computes the affine parameters using a multiplication-free algorithm. The reference and current frames are stored in embedded S-RAMs generated with Virage™ Memory Compiler. The proposed motion compensation architecture has been prototyped, simulated and synthesized using the TSMC 0.18 μm CMOS technology. Using 100 MHz clock frequency, the proposed architecture processes one CIF video frame (i.e., 352×288 pixels) in 0.59 ms, which means it can process up to 1694 frames per second. The core area of the proposed motion compensation architecture is 28.04 mm² and it consumes 31.15 mW.

Current multimedia design processes suffer from the excessively large time spent on testing new IP-blocks with references based on large video encoders specifications (usually several thousands lines of code). The appropriate testing of a single IP-block may require the conversion of the overall encoder from software to hardware, which is difficult to complete in the short time required by the competition-driven reduced time-to-market demanded for the adoption of a new video coding standard. This paper presents a new design flow to accelerate the conformance testing of an IP-block using the H.264/AVC software reference model. An example block of the simplified 8 × 8 transformation and quantization, which is adopted in FRExt, is provided as a case study demonstrating the effectiveness of the approach.

An architecture proof-of-concept which adapts the throughput datapath based on the anticipation of computational demand in dynamic environments is demonstrated and evaluated for a motion estimation (ME) engine. The input signal characteristics are exploited to anticipate the time varying computational complexity as well as to instantiate dynamic replicas (DRs) to realize fault-resilience. The scheme employs amorphous processing elements (APEs) which either perform as active elements (AEs) to maintain quality/throughput, serve as DRs to increase reliability levels, or hibernate passively as reconfigurable slack (RS) available to other tasks.

Experimental results from a hardware platform for field programmable gate array (FPGA)-based video encoding demonstrate power efficiency and fault-tolerance of the ME engine. A significant reduction in power consumption is achieved ranging from 83% for low-motion-activity scene's to 12.5% for high motion activity video scenes. The scenes motion activity is utilized to improve redundancy for the purpose of priority based diagnosis of the computing modules. In addition, a graceful degradation strategy is developed to recover from hard errors by adapting the search range of candidate motion vectors (MVs). This adaptive hardware scheme is shown to automatically demote the faulty resources in FPGA devices based on streaming performance.

In this paper, we propose a rate control (RC) scheme for versatile video coding (VVC) to enhance coding performance. First, we propose a rate–distortion (R–D) model at the coding tree unit (CTU) level to describe the R–D relationship. Second, we adjust the quantization parameter (QP) and Lagrange multiplier at the coding unit (CU) level according to visual features to improve the rationality of local coding results. Finally, we propose a model parameter updating strategy to guarantee bitrate accuracy. The experimental results demonstrate that our RC method has better R–D performance with similar bitrate accuracy and better visual quality compared to the default RC strategy in VVC reference software VTM 10.2.

This paper presents the enhanced frame-based video coding scheme. The input source video to the enhanced frame-based video encoder consists of a rectangular-sized video and shapes of arbitrarily shaped objects on video frames. The rectangular frame texture is encoded by the conventional frame-based coding technique and the video object's shape is encoded using the contour-based vertex coding. It is possible to achieve several useful content-based functionalities by utilizing the shape information in the bitstream at the cost of a very small overhead to the bit-rate.

Advancements in artificial intelligence have significantly increased the use of images and videos in machine analysis algorithms, predominantly neural networks. However, the traditional methods of compressing, storing and transmitting media have been optimized for human viewers rather than machines. Current research in coding images and videos for machine analysis has evolved in two distinct paths. The first is characterized by End-to-End (E2E) learned codes, which show promising results in image coding but have yet to match the performance of leading Conventional Video Codecs (CVC) and suffer from a lack of interoperability. The second path optimizes CVC, such as the Versatile Video Coding (VVC) standard, for machine-oriented reconstruction. Although CVC-based approaches enjoy widespread hardware and software compatibility and interoperability, they often fall short in machine task performance, especially at lower bitrates. This paper proposes a novel hybrid codec for machines named NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to − 43.20% and − 26.8% Bjøntegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks according to the common test conditions designed by the VCM study group in MPEG standardization activities. Furthermore, to improve reconstruction quality, we introduce a human-focused branch into our codec, enhancing the visual appeal of reconstructions intended for human supervision of the machine-oriented main branch.

In this paper, the classical algorithm of 3D wavelet transform is improved and a new video coding based on the improved algorithm and motion compensation is proposed. The main difference between the classical algorithm and the new algorithm is the temporal decomposition by wavelet transform. With the new algorithm, video sequence is firstly separated into two groups according to the parity of each frame. Then the improved algorithm for temporal decomposition is applied to each group separately which resulting several high frequency frames and one low frequency frame in each group. After that, the motion compensation is only performed in the two low frequency frames in order to eliminate the interframe redundancies. Experiments results show that the proposed scheme has better coding performance compared with other schemes based on 3D-WT.

Most fast block motion estimation algorithms so far have adopted the sum of absolute difference (SAD) as matching criterion. But due to the limitation of SAD, these algorithms are not generally efficient. This paper firstly will give analyses of the SAD criterion. A SAD-optimized fast search algorithm is proposed from the prediction of initial search center, hybrid search template, and adaptive search modes for the video sequences with different motion. Experimental results show that the proposed algorithm performs faster than the MVFAST and the PMVFAST, whereas similar prediction quality is still maintained.

In this paper, an improved interframe coding scheme consisting of both the adaptive directional motion search and improved mode selection is proposed to reduce the computational complexity of video coding. The motion direction is divided into four regions, and different multi-hexagon templates are designed for each region to enhance directional search, which can improve the search accuracy. Subsequently, the improved mode decision that integrates the detection of all zero blocks is proposed for further computation reduction. Experimental results demonstrate that the proposed scheme can reduce encoding time by an average of 60% while maintaining almost the same coding efficiency as existing fast interframe coding schemes.

In this paper, fast mode decision strategies based on motion coherence for P-frame coding are proposed to accelerate the video encoding procedure. Firstly, mode prejudgment strategy based on neighborhood macroblocks (MBs) is introduced. According to the coherence of adjacent movement, the left and up MBs in current frame and the current-position and right MBs in previous frame are used to predict the mode of current MB. Secondly, mode combination strategy based on motion coherence is implemented to combine blocks in sub-type with similar motion vectors (MVs) into a block in macro-type. Lastly, the overall fast mode decision scheme is proposed and the performance of the proposed scheme is verified through comparative experimental results using JM reference software.

Fractional pixel motion estimation (FPME) and fractional pixel interpolation (FPI) account for relatively significant amount of computational time in video coding. Fewer search positions FPME checks, the lesser fractional pixels have to be interpolated. Thus, developing fast FPME algorithms is equivalent to save computation on FPME and FPI. Most of fast FPME algorithms are dedicated to motion vector prediction for finding an optimized position to start the successive search. However, these algorithms overlook the minimal cost prediction which can stop the refinement search as early as possible. In this paper, we propose a dual prediction based algorithm for FPME. Spatial and temporal correlations of neighboring blocks are being used to predict motion vectors as well as the optimal costs. According to our experiments, the algorithm not only achieves significant speedup but also provides good R-D performance compared with other algorithms.