In this paper, we present the work on implementation of a half-D1interlaced MPEG-4 encoder with Equator Technology DSP chip, BSP-15. The BSP-15 DSP consists mainly of a VLIW core, Co-processors, and media I/O interfaces. The encoder utilizes several BSP-15 functional blocks in parallel. In general, the VLIW performs pixel procesing that is computationally intensive. The VLx coprocessor completes variable length coding. Further parallelism is obtained by pre-loading data cache and doubling data buffers. Given the DSP processing power and real time requirements, a complexity control scheme is implemented. A frame-level quantization scheme with quality and rate control is employed. The current implementation for video at 30 fps consumes about 90% of the chip performance at a bit rate ~2Mbps.

The MPEG-4 standard has already been established, well accepted and is being widely used by content creators, developers, manufacturers, providers of services and end users. Industry leaders have come together to guarantee interoperability. Amongst the video standards, the MPEG-4 video Advanced Simple Profile (ASP) has found wide acceptance and the need for real-time low cost MPEG-4 video ASP encoding solutions isappropriate for certain markets. Embedded multimedia systems, such as programmable VLIW-based chips offer flexibility and functionality, and are very suitable platforms for deployment of real-time multimedia applications. The Equator BSP-15 is one such single chip platform. Enabling technologies for such embedded multimedia systems require very careful and often creative considerations of cost versus performance. To this end, we have designed and developed a real-time MPEG-4 video ASP encoder at Level 5 on an Equator BSP-15 chip. This paper describes our experiences with the design considerations and some implementation issues along with some proposed solutions. The experimental results show the feasibility of these solutions to obtain very good performance on low-cost embedded systems.

Complexity scalability attempts to scale the required resources of an algorithm with the chose quality settings, in order to broaden the application range. In this paper, we present complexity-scalable MPEG encoding of which the core processing modules are modified for scalability. Scalability is basically achieved for the computational complexity by varying the number of computed DCT coefficients and the number of evaluated motion vectors, while other modules are designed such that they scale with the previous parameters. Resource usage such as power-consuming memory accesses scale accordingly. The interdependencies of the scalable modules and the system performance are evaluated. Experimental results show scalability giving a smooth change in complexity and coresponding video quality. The elapsed execution time of the scalable encoder, reflecting the computational complexity, can be gradually reduced to roughly 50% of its execution time when operating at high quality. The video quality scaled betwee 21.5 dB and 38.5 dB PSNR for different sequences targeting 1500 kbps. The implemented encoder and the scalability techniques can be successfully applied in mobile systems based on MPEG video compression, and also has advantages in multi-tasking environments like high-end TV sets. The obtained scalability techniques can be applied to other coding standards like MPEG-4 and H.264 as well.

Quarter pixel precision motion compensation became a common tool in advanced video compression standards such as MPEG-4 Advanced Simple Profile (ASP), H.264 because of its ability to represent motion more accurately than half pixel precision. MPEG-4 ASP uses 8-tap symmetric filter in finding interpolated sample values at half pixel resolution in quarter pixel precision motion compensation. This new tool increases the computational complexity of the fractional pixel motion estimation on the encoder side compared to the integer pixel motion estimation, in terms of computation of interpolated sample values as well as number of fractional pixel search points. This paper presents a way to reduce the complexity of quarter pixel motion estimation by reducing the filter tap length. Reduction in computation complexity and variation in PSNR values with different filters are presented.

Low power dissipation and fast processing time are crucial requirements for embedded multimedia devices. This paper presents a technique in video coding to decrease the power consumption at a standard video decoder. Coupled with a small dedicated video internal memory cache on a decoder, the technique can substantially decrease the amount of data traffic to the external memory at the decoder. A decrease in data traffic to the external memory at decoder will result in multiple benefits: faster real-time processing and power savings. The encoder, given prior knowledge of the decoder’s dedicated video internal memory cache management scheme, regulates its choice of motion compensated predictors to reduce the decoder’s external memory accesses. This technique can be used in any standard or proprietary encoder scheme to generate a compliant output bit stream decodable by standard CPU-based and dedicated hardware-based decoders for power savings with the best quality-power cost trade off. Our simulation results show that with a relatively small amount of dedicated video internal memory cache, the technique may decrease the traffic between CPU and external memory over 50%.

In this paper, a method for designing low-power video schemes is presented. Algorithms that imply a very low dissipation are required for new applications where the energy source is limited, e.g. mobile phones including a camera and video features. Whereas it can be observed that video standards are mainly designed around coding efficiency, we propose to take into account power consumption characteristics directly when designing the algorithm. More precisely, we give some guidelines for the design of low-power video codecs in the scope of modern hardware architectures and we introduce the notion of power scalability. We present an original encoder based on so-called 'Collocated Motion Estimation' designed using the proposed methodology. Experimental results show that we remain close to the coding efficiency of the reference H.264 baseline encoder while the power consumption is largely reduced in our solution. Moreoever this encoder is scalable in memory transfer and computational complexity.

In video coding and streaming over wireless communication network, the power-demanding video encoding operates on the mobile devices with limited energy supply. To analyze, control, and optimize the rate-distortion (R-D) behavior of the wireless video communication system under the energy constraint, we need to develop a power-rate-distortion (P-R-D) analysis framework, which extends the traditional R-D analysis by including another dimension, the power consumption. Specifically, in this paper, we analyze the encoding mechanism of typical video encoding systems and develop a parametric video encoding architecture which is fully scalable in computational complexity. Using dynamic voltage scaling (DVS), a hardware technology recently developed in CMOS circuits design, the complexity scalability can be translated into the power consumption scalability of the video encoder. We investigate the rate-distortion behaviors of the complexity control parameters and establish an analytic framework to explore the P-R-D behavior of the video encoding system. Both theoretically and experimentally, we show that, using this P-R-D model, the encoding system is able to automatically adjust its complexity control parameters to match the available energy supply of the mobile device while maximizing the picture quality. The P-R-D model provides a theoretical guideline for system design and performance optimization in wireless video communication under energy constraint, especially over the wireless video sensor network.

It is possible to improve the features supported by devices with embedded systems by increasing the processor computing power, but this always results in higher costs, complexity, and power consumption. An interesting alternative is to use the growing networking infrastructures to do remote processing and visualization, with the embedded system mainly responsible for communications and user interaction. This enables devices to behave as if much more “intelligent” to users, at very low costs and power. In this article we explain how compression can make some of these solutions more bandwidth-efficient, enabling devices to simply decompress very rich graphical information and user interfaces that had been rendered elsewhere. The mixture of natural images and video with text, graphics, and animations simultaneously in the same frame is called compound video. We present a new method for compression of compound images and video, which is able to efficiently identify the different components during compression, and use an appropriate coding method. Our system uses lossless compression for graphics and text, and, on natural images and highly detailed parts, it uses lossy compression with dynamically varying quality. Since it was designed for embedded systems with very limited resources, and it has small executable size, and low complexity for classification, compression and decompression. Other compression methods (e.g., MPEG) can do the same, but are very inefficient for compound content. High-level graphics languages can be bandwidth-efficient, but are much less reliable (e.g., supporting Asian fonts), and are many orders of magnitude more complex. Numerical tests show the very significant gains in compression achieved by these systems.

Wyner-Ziv coding refers to lossy source coding with side information at the decoder. Recently some practical applications of Wyner-Ziv coding to video compression have been studied due to its advantage of error robustness over standard video coding standards. Based on recent theoretical result on successive Wyner-Ziv coding, we propose in this paper a practical layered Wyner-Ziv video codec using the DCT, nested scalar quantizer (NSQ), and irregular LDPC code based Slepian-Wolf coding (or lossless source coding with side information). The DCT is applied as an approximation to the conditional KLT, which makes the components of the transformed block conditionally independent given the side information. NSQ is a binning scheme that facilitates layered bit-plane coding of the bin indices while reducing the bit rate. LDPC code based Slepian-Wolf coding exploits the correlation between the quantized version of the source and the side information to achieve further compression. Different from previous works, an attractive feature of our proposed system is that video encoding is done only once but decoding allowed at many lower bit rates without quality loss.

The mobile phone is undergoing a rapid evolution from a voice and limited text-messaging device to a complete multimedia client. RISC processors are predominantly used in these devices due to low cost, time to market and power consumption. The growing demand for signal processing performance on these platforms has triggered a convergence of RISC, CISC and DSP technologies on to a single core/system. This convergence leads to a multitude of challenges for optimal usage of available processing power. Voice codecs, which have been traditionally implemented on DSP platforms, have been adapted to sole RISC platforms as well. In this paper, the issues involved in optimizing a standard vocoder for RISC-DSP convergence platform (DSP enhanced RISC platforms) are addressed. Our optimization techniques are based on identification of algorithms, which could exploit either the DSP features or the RISC features or both. A few algorithmic modifications have also been suggested. By a systematic application of these optimization techniques for a GSM-AMR (NB) codec on ARM9E core, we could achieve more than 77% improvement over the baseline codec and almost 33% over that optimized for a RISC platform (ARM9T) alone in terms of processing cycle requirements. The optimization techniques outlined are generic in nature and are applicable to other vocoders on similar 'application-platform’ combinations.

In this paper we describe some of the research issues and challenges in image-based location awareness and navigation. We will describe two systems being developed at Purdue University as testbeds for our ideas. The main system architecture combines image processing, mobility, wireless communication, and location awareness. We will describe two fundamental scenarios for using images to aid in mobile navigation problems. The first provides the ability to use a locally acquired image to determine the identity of an object, for example a building, as one roams in an area. The second problem is the use of images in a database to aid in vehicle navigation. The solution to these problems use location information, such as GPS signals, to compare and search location-annotated images in a database. We believe location information can improve the accuracy in image database search.

In this paper, we consider the problem of transmitting compressed video over wirless LAN under transmission power constraints, where retransmission is adopted as the error control scheme. For different retry limits, transmitter energy per bit is adjusted to keep a constant video quality at the receiver, which costs different transmission power. We propose an algorithm to minimize transmission power by carefully choosing the maximum number of retransmission times and transmission energy level based on the quality and delay requirement. We also examine the effect of distance on the choice of the optimal points.

We describe a MPEG-7 Meta-Data enhanced Audio-Visual Encoder system that targets DVD recorders. We extract features in the compressed domain with both video and audio, which allows us to add the meta-data extraction without altering the hardware architecture of the encoder core. Our feature extraction algorithms are simple, and thus implementable through a simple combination of software and hardware on the integrated DVD chip. The primary application of the meta-data is video summarization, which enables rapid browsing of stored video by the end user. The simplicity of our summarization and feature extraction algorithms enables incorporation of the powerful functionality of smart content navigation through content summarization, into the DVD recorder at a low cost.

In this paper we propose a novel, post-processing system for compressed video sources. The proposed system explores the interaction between artifact reduction and sharpness/resolution enhancement to achieve optimal video quality for compressed (e.g. MPEG-2) sources. It is based on the Unified Metric for Digital Video Processing (UMDVP), which adaptively controls the post-processing algorithms according to the coding characteristics of the decoded video. The experiments carried out on several MPEG-2 encoded video sequences have shown significant improvement in picture quality compared to a system without the UMDVP control and to a system that did not exploit the interaction between artifact reduction and video enhancement. The UMDVP as well the proposed post-processing system can be easily adapted for different coding standard, such as MPEG-4, H.26x.

Elliptic curve cryptography (ECC) is an excellent candidate for secure embedded multimedia applications due to its small key size and high security protection. The performance profiling of the ECC implementation, such as execution time and data cache stalls, on TriMedia TM1300 and Intel Pentium 4 is conducted in this research. Based on this study, we identify the main bottlenecks of the EEC implementation, and propose some favorable micro-architecture for this application. Moreover, several integer multiplication schemes are presented for the TM1300 processor for performance enhancement. In particular, the FIR-based multiplication is built with the special FIR instruction provided by TM1300. The performance improvement of the proposed schemes is reported and discussed. Overall, we aim at providing a good understanding of the system architecture of secure embedded multimedia applications, hardware and software cryptography implementation with ECC as an example.

This paper proposes an adaptive block-size motion alignment technique in 3D wavelet coding to further exploit temporal correlations across pictures. Similar to B picture in traditional video coding, each macroblock can motion align from forward and/or backward for temporal wavelet de-composition. In each direction, a macroblock may select its partition from one of seven modes - 16x16, 8x16, 16x8, 8x8, 8x4, 4x8 and 4x4 - to allow accurate motion alignment. Furthermore, the rate-distortion optimization criterions are proposed to select motion mode, motion vectors and partition mode. Although the proposed technique greatly improves the accuracy of motion alignment, it does not directly bring the coding efficiency gain because of smaller block size and more block boundaries. Therefore, an overlapped block motion alignment is further proposed to cope with block boundaries and to suppress spatial high-frequency components. The experimental results show the proposed adaptive block-size motion alignment with the overlapped block motion alignment can achieve up to 1.0 dB gain in 3D wavelet video coding. Our 3D wavelet coder outperforms the MC-EZBC for most sequences by 1~2dB and we are doing up to 1.5 dB better than H.264.

Spatial video segmentation is usually performed based on motion between two frames. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation can be interpreted as a volume "carved out" by a moving object in the image sequence domain, or the so-called "object tunnel". In this paper, we extend this concept to explicit modeling of occlusion events in space-time. In addition to the modeling of object evolution, we also explicitly model occluded and newly-exposed areas in the background by means of "occlusion volume", a new space-time concept. A voxel belongs to occlusion volume if its intensity is consistent with past intensities along its motion trajectory but inconsistent with future intensities (reversed for "exposed volume"). We propose a variational formulation of the problem that we solve using the multiphase level set method. We show encouraging experimental results for synthetic and natural image sequences.

We present a computer vision tool to improve the clinical outcome of patients undergoing radiation therapy for prostate cancer by improving irradiation technique. While intensity modulated radiotherapy (IMRT) allows one to irradiate a specific region in the body with high accuracy, it is still difficult to know exactly where to aim the radiation beam on every day of the 30~40 treatments that are necessary. This paper presents a geometric model-based technique to accurately segment the prostate and other surrounding structures in a daily serial CT image, compensating for daily motion and shape variation. We first acquire a collection of serial CT scans of patients undergoing external beam radiotherapy, and manual segmentation of the prostate and other nearby structures by radiation oncologists. Then we train shape and local appearance models for the structures of interest. When new images are available, an iterative algorithm is applied to locate the prostate and surrounding structures automatically. Our experimental results show that excellent matches can be given to the prostate and surrounding structure. Convergence is declared after 10 iterations. For 256 x 256 images, the mean distance between the hand-segmented contour and the automatically estimated contour is about 1.5 pixels (2.44 mm), with variance about 0.6 pixel (1.24 mm).

This paper presents a novel approach to human body posture recognition based on the MPEG-7 contour-based shape descriptor and the widely used projection histogram. A combination of them was used to recognize the main posture and the view of a human based on the binary object mask obtained by the segmentation process. The recognition is treated as a typical pattern recognition task and is carried out through a hierarchy of classifiers. Therefore various structures both hierachical and non-hierarchical, in combination with different classifiers, are compared to each other with respect to recognition performance and computational complexity. Based on this an optimal system design with recognition rates of 95.59% for the main posture, 77.84% for the view and 79.77% in combination is achieved.

In this paper, we propose a new variational formulation for simultaneous multiple motion segmentation and occlusion detection in an image sequence. For the representation of segmented regions, we use the multiphase level set method proposed by Vese and Chan. This method allows an efficient representation of up to 2^L regions with L level-set functions. Moreover, by construction, it enforces a domain partition with no gaps and overlaps. This is unlike previous variational approaches to multiple motion segmentation, where additional constraints were needed. The variational framework we propose can incorporate an arbitrary number of motion transformations as well as occlusion areas. In order to minimize the resulting energy, we developed a two-step algorithm. In the first step, we use a feature-based method to estimate the motions present in the image sequence. In the second step, based on the extracted motion information, we iteratively evolve all level set functions in the gradient descent direction to find the final segmentation. We have tested the above algorithm on both synthetic- and natural-motion data with very promising results. We show here segmentation results for two real video sequences.

We propose a novel algorithm to partition an image with low depth-of-field (DOF) into focused object-of-interest (OOI) and defocused background. The proposed algorithm consists of three steps. In the first step, we transform the low DOF image into an appropriate feature space for the partition. This is conducted by computing higher-order statistics (HOS) for all pixels in the low DOF image. Next, the obtained feature space, which is called HOS map, is simplified by removing small dark holes and bright patches using a morphological filter by reconstruction. Finally, the OOI is extracted by applying region merging and adaptive thresholding to the simplified image. Experimental results show that the proposed method yields more accurate segmentation results than the previous methods.

The use of partial differential equations in image processing has become an active area of research in the last few years. In particular, active contours are being used for image segmentation, either explicitly as snakes, or implicitly through the level set approach. In this paper, we consider the use of the implicit active contour approach for segmenting scientific images of pollen grains obtained using a scanning electron microscope. Our goal is to better understand the pros and cons of these techniques and to compare them with the traditional approaches such as the Canny and SUSAN edge detectors. The preliminary results of our study show that the level set method is computationally expensive and requires the setting of several different parameters. However, it results in closed contours, which may be useful in separating objects from the background in an image.

A rate-distortion optimized scheme for interactive light field streaming is proposed. The light field data set is transformed into blocks of wavelet coefficients; each block is coded as a scalable bitstream and stored at the sender. To render a frame, the receiver issues a request for relevant data. Based on the request, the estimated state of the data already at the receiver, the network characteristics, and the desired transmission rate, the sender customizes the outgoing packets in order to minimize the distortion experienced at receiver. Experimental results show that the proposed rate-distortion optimized scheme improves the rendering quality by 0.5~2.1 dB in PSNR over a heuristic scheme at the same rate. Alternatively, it reduces the required bit-rate by 10%~25% over the heuristic scheme at the same rendering quality.

Object-oriented coding in the MPEG-4 standard enables the separate processing of foreground objects and the scene background (sprite). Since the background sprite only has to be sent once, transmission bandwidth can be saved. This paper shows that the concept of merging several views of a non-changing scene background into a single background sprite is usually not the most efficient way to transmit the background image. We have found that the counter-intuitive approach of splitting the background into several independent parts can reduce the overall amount of data. For this reason, we propose an algorithm that provides an optimal partitioning of a video sequence into independent background sprites (a multi-sprite), resulting in a significant reduction of the involved coding cost. Additionally, our algorithm results in background sprites with better quality by ensuring that the sprite resolution has at least the final display resolution throughout the sequence. Even though our sprite generation algorithm creates multiple sprites instead of a single background sprite, it is fully compatible with the existing MPEG-4 standard. The algorithm has been evaluated with several test-sequences, including the well-known Table-tennis and Stefan sequences. The total coding cost could be reduced by factors of about 2.7 or even higher.

In this paper, we describe a new lossless compression algorithm for color video sequences. Our approach is to exploit both spatial and temporal redundancies in the video sequence by developing a new adaptive algorithm. By adaptive selection between spatial and temporal prediction we show that our scheme is better than state-of-the-art lossless compression algorithms.

In this work, we consider a generic definition of video object, which is a group of pixels with temporal motion coherence. The generic video object (GVO) is the superset of the conventional video objects discussed in the literature. Because of its motion coherence, the GVO can be easily recognized by the human visual system. However, due to its arbitray spatial distribution, the GVO cannot be easily detected by the existing algorithms which often assume the spatial homogeneousness of the video objects. In this work, we introduce the concept of extended optical flow and develop a dynamic programming framework for the GVO detection. Using this mathematical optimization formulation, whose solution is given by the the Viterbi algorithm, the proposed object detection algorithm is able to discover the motion path of the GVO automatically and refine its spatial location progressively. We apply the GVO detection algorithm to extract and remove the so-called "video mask" signals in the video sequence. Our experimental results show that this type of vision-guided video pre-processing significantly improves the compression efficiency.

Set Partitioned Embedded bloCK coder (SPECK) has been found to perform comparable to the best-known still grayscale image coders like EZW, SPIHT, JPEG2000 etc. In this paper, we first propose Color-SPECK (CSPECK), a natural extension of SPECK to handle color still images in the YUV 4:2:0 format. Extensions to other YUV formats are also possible. PSNR results indicate that CSPECK is among the best known color coders while the perceptual quality of reconstruction is superior than SPIHT and JPEG2000. We then propose a moving picture based coding system called Motion-SPECK with CSPECK as the core algorithm in an intra-based setting. Specifically, we demonstrate two modes of operation of Motion-SPECK, namely the constant-rate mode where every frame is coded at the same bit-rate and the constant-distortion mode, where we ensure the same quality for each frame. Results on well-known CIF sequences indicate that Motion-SPECK performs comparable to Motion-JPEG2000 while the visual quality of the sequence is in general superior. Both CSPECK and Motion-SPECK automatically inherit all the desirable features of SPECK such as embeddedness, low computational complexity, highly efficient performance, fast decoding and low dynamic memory requirements. The intended applications of Motion-SPECK would be high-end and emerging video applications such as High Quality Digital Video Recording System, Internet Video, Medical Imaging etc.

Transform coding plays a central role in image and video coding technologies. Various transforms, whether fixed or adaptive, have been utilized for video compression. However, these transforms are inherently sub-optimal in a coding-efficiency sense, given that their design does not explicitly take into account the entire transform coding model. Thus, commonly used transforms such as the DCT have desirable properties, but were designed without the full consideration of the other transform coding elements including quantization and entropy coding. Although these transforms have good performance in a rate-distortion sense, as demonstrated by today's image and video coding standards, we show that superior transforms can be designed based on the complete transform coding model. We present a new class of transforms called coding-adaptive transforms, that are derived under an optimality criterion which incorporates the elements of transform coding and is formulated for coding efficiency. Additionally, characteristics of the new transforms such as adaptivity, generalization, and robustness are discussed.

This paper presents a Rate-Distortion based optimization procedure for video coding that explicitly incorporates a perceptual distortion measure. A flexible MINMAX (minimizing the maximum distortion) optimization procedure based on Linear Programming (LP) is used to minimize the variation in distortion across frames. The distortion measure utilizes a multi-channel human vision model to generate a map of the perceived distortion at each pixel in a frame. This map is then collapsed into a single frame-level distortion. Lagrangian optimization is employed within each frame to assign macroblock-level quantization step-sizes that minimize this frame-level distortion. Coding results with MPEG-2 show a large reduction in peak perceived distortion with respect to TM5 as well as lower variation in distortion across the frames in the video sequence. Although results are presented for a specific metric, the LP based optimization procedure easily extends to other distortion measures.

Considering nodes energy and channel bandwidth limitations in a multiview images network, avoiding inter-node communication in the coding scheme is necessary and makes the communication efficient, in comparison with conventional coding methods with internode communication. In our system, we consider a multiview images network as an array of nodes on a line with the same distance to each other. Each node in this system includes a camera to capture with a limited processing and communication abilities. We propose a multiview images coding without requiring inter-node communication, to gain the advantage of correlation at decoder side for two network configurations, but the main concept of the proposed coding for both network configurations is the same. The two network configurations are distinguished based on presence of parent node in each cluster, which sends full information to central node (i.e. joint decoder). The other nodes called children nodes and send their partial information to central node. In this paper, a coding scheme is proposed for multiview images network with parent node based on the number of parent nodes (i.e. “one parent node cluster” and “two parent node cluster”) in each cluster. Not that, because we are involved with multiview images, the decoding procedure is searching for correspondence between partially sent multiview images data. In coding scheme with parent node, if each parent node fails the decoding task of children nodes will be failed or the error caused by corresponding search during the decoding of children nodes is increased if network uses other parent nodes to decode the cluster without parent node. To make the network robust in case of node failure, we avoid parent nodes, and developed a network configuration where all nodes are children nodes (i.e. "without parent node cluster"). For such a network, we also propose a coding algorithm for multiview images that the joint decoder at central node is able to decode the partial information of each children node with side information of other children node. So, in case of losing any viewpoint, joint decoder is still able to decode other received partial information of children nodes. Finally, we have compared coding scheme of network with and without inter-node communication for different cluster configuration mentioned herein, and “all parent node cluster” configuration, considering communication rate in the network, symmetry in communication and decoding quality of children node, and their robustness in case of nodes failure.

We investigate and compare the performance of several three-dimensional (3D) embedded wavelet algorithms on lossless 3D image compression. The algorithms are Asymmetric Tree Three-Dimensional Set Partitioning In Hierarchical Trees (AT-3DSPIHT), Three-Dimensional Set Partitioned Embedded bloCK (3D-SPECK), Three-Dimensional Context-Based Embedded Zerotrees of Wavelet coefficients (3D-CB-EZW), and JPEG2000 Part II for multi-component images. Two kinds of images are investigated in our study -- 8-bit CT and MR medical images and 16-bit AVIRIS hyperspectral images. First, the performances by using different size of coding units are compared. It shows that increasing the size of coding unit improves the performance somewhat. Second, the performances by using different integer wavelet transforms are compared for AT-3DSPIHT, 3D-SPECK and 3D-CB-EZW. None of the considered filters always performs the best for all data sets and algorithms. At last, we compare the different lossless compression algorithms by applying integer wavelet transform on the entire image volumes. For 8-bit medical image volumes, AT-3DSPIHT performs the best almost all the time, achieving average of 12% decreases in file size compared with JPEG2000 multi-component, the second performer. For 16-bit hyperspectral images, AT-3DSPIHT always performs the best, yielding average 5.8% and 8.9% decreases in file size compared with 3D-SPECK and JPEG2000 multi-component, respectively. Two 2D compression algorithms, JPEG2000 and UNIX zip, are also included for reference, and all 3D algorithms perform much better than 2D algorithms.

A progressive 3D geometry coding scheme based on the octree structure is proposed in this work, which achieves better coding efficiency than the state-of-the-art geometric codec known as the kd-tree-based codec. Given a 3D mesh, the quantized 3D vertices are first partitioned into an octree structure. The octree is then traversed from the root and gradually to the leaves and, during the traversal, each 3D cell in the tree front is subdivided along three orthogonal directions. For each cell subdivision, the order of subdivision directions is adaptively chosen, the neighborhood-prediction is used and the vertex number distribution is efficiently encoded. The final output bit stream contains relevant information associated with each cell subdivision. It is shown in experimental results that the coding cost is around 15.6 bits per vertex (bpv) for 12-bit coordinate quantization and 5.6 bpv for 8-bit coordinate quantization on average.

In critical applications of image compression that are sensitive to information loss, lossless compression techniques are usually employed. The compression ratios obtained from lossless techniques are low. Hence we need different schemes that give quantitative guarantees about the type and amount of distortion incurred, viz. near-lossless compression methods. In this paper, we explore hybrid techniques for near-lossless image compression based on error bound per pixel and apply them to Digital Terrain Elevation Data (DTED). We develop multi-stage quantization methods and other hybrid schemes using SPIHT as an embedded lossy coder. The methods developed are scalable with a control on maximum allowable deviation per pixel in the reconstruction. We compare the results obtained by these methods with JPEG-LS, the standard for lossless and near-lossless compression. Results show that these methods usually perform better than JPEG-LS in terms of compression performance and have additional features useful for progressive transmission.

We present a new approach to video coding that applies video analysis based on global motion features. A super-resolution mosaic is build for each frame to be encoded from a number of previously transmitted frames. This super-resolution mosaic is used to detect macroblocks that are only affected by global motion. For such macroblocks no prediction error is transmitted. They are purely reconstructed by prediction from the super-resolution mosaic which results in significant bitrate savings. Our results indicate total bitrate savings of 20% and more compared to a state of the art H.264/AVC codec at the same visual quality.

A multistage mode decision method to reduce the complexity of Intra prediction in H.264/JVT is proposed in this work. The rate-distortion optimized (RDO) mode decision is adopted by H.264 to achieve the optimal coding gain at the cost of a very high computational complexity. Our research goal is to reduce the complexity of intra prediction without significant RD performance loss. The proposed fast intra prediction algorithm uses two features, i.e. the sum of absolute residual coefficients (SATD) and the sum of absolute gradients (SAG), and a simplified RDO method to determine the best mode for 4x4 blocks. It incorporates an early termination mechanism at various intermediate decision steps to avoide unnecessary computations when a good decision can be made at an earlier stage. It is demonstrated by experimental results that the proposed scheme performs approximately 10 to 30 times faster than the current RDO mode decision (JM7.3a) with little degradation in the coding gain.

In this paper we introduce a new Inter Macroblock type within the H.264 (or MPEG-4 AVC) video coding standard that can further improve coding efficiency by exploiting the temporal correlation of motion within a sequence. This leads to a reduction in the bits required for encoding motion information, while retaining or even improving quality under a Rate Distortion Optimization Framework. An extension of this concept within the skip macroblock type of the same standard is also presented. Simulation results show that the proposed semantic changes can lead to up to 7.6% average bitrate reduction or equivalently 0.39dB quality improvement over the current H.264 standard.

In this paper, we present a novel H.264 video decoder where memory transfers and energy consumption are significantly reduced. A power-friendly solution is indeed required in mobile applications, where autonomy is a key feature. Whereas low-power design is usually regarded as a pure architectural and implementation problem, our solution is based on an algorithmic approach. We first observe that in modern hardware architectures, computational complexity plays a second role in terms of energy dissipation, which is dominated by memory transfers. We thus identify the motion compensation module as the most consuming part of the standard H.264 decoder, due to its numerous memory accesses to the reference frame(s). Through an algorithmic modification that uses concurrent embedded compression techniques and relies on the data structure of the memory, we reduce memory transfers, and hence power dissipation, while maintaining the visual quality. Experimental results prove that the new method is close to the reference H.264 baseline decoder both in terms of objective and subjective measurements. In the meantime, memory transfers have been reduced by 55% in average, implying power savings which lengthen the battery life of the mobile device, increase the reliability of the chip and lower production costs.

H.264 is the emerging video coding standard, which aims at compressing high-quality video contents at low bit-rates. While its new encoding and decoding processes are similar to many previous standards, the new standard includes a number of new features and thus requires much more computation than most existing standards do. The complexity of H.264 standard poses a large amount of challenges to implementing the encoder/decoder in real-time via software on personal computers. Even after 2~3x performance improvement with media instruction on modern general-purpose processors and another 2~4x improvement from algorithmic optimization, the H.264 encoder is still too complicated to be implemented in real-time on a single processor. Based on the detailed analysis of the possibilities of parallelism in H.264 encoder, we proposed an efficient multithreading implementation of the H.264 video encoder. In order to guarantee enough concurrency of the whole system, an elaborate macroblock and inter-frame parallel scheduling scheme is presented. In addition, our macroblock-based multithreading scheme achieves almost no video quality losses in contrast to other parallelization schemes. Our results show that the multithreaded encoder can obtain another 3.96x speed-up on a four-processor system or 4.6x speed-up on a four-processor system with Hyper-Treading Technology. The techniques demonstrated in this work can be applied not only to H.264, but also to other video/image coding/decoding applications on personal computers.

We present a method for true-motion estimation assisted by feature point correspondences. First the difference between true-motion estimation and motion estimation for coding applications is explained, and an earlier published efficient true-motion estimation algorithm, called 3DRS, is summarized. Then the convergence property of this algorithm is discussed. We present a method for improving the convergence, by using feature point correspondences and show that a significant quality increase can be obtained for sequences containing high velocities.

Predictive motion estimation techniques have become widely used in video coding systems due to their reasonable coding performance and low computational requirement compared to the brute force full search method. However, most existing predictive motion estimation algorithms focus mainly on local motions and consider motion predictions from only neighboring image blocks (either spatially or temporally). In this paper, a reliable global motion predictor or GMP is introduced to improve predictive motion estimation for sequences with fast camera motions. A simple global motion model is applied for global motion estimation. The derived global motion vector for each macroblock is then used as the GMP during the predictive motion estimation for all possible block modes or types, together with up to 8 local motion predictors for P frame motion estimation and up to 12 local motion predictors for B frame motion estimation. The comparison between the coding performance with and without GMP shows very significant differences; roughly 1-2 dB’s improvement is achieved by GMP for sequences with very fast camera motions. The computational overhead for GMP is very small, less than 2% of the total computation cycles in our H.264 encoder.

In this paper we present a statistical analysis of motion prediction with drift in video coding. The drift effect occurs when the video decoder does not have access to the same reference information used in the encoder in a hybrid video codec using motion prediction. Although the drift effect has been known to the video research community for a long time, there has not been a systematic theoretical treatment of this mechanism. Generally the performance of motion prediction with drift is evaluated experimentally. In this paper we derive a closed-form expression for the drift error. Based on this result, we derive an efficient rate distortion optimization given the statistical knowledge of the channel.

We propose a non-iterative, globally optimal dense motion field estimation technique based on a multiresolutional probability model. We consider the field to be estimated in terms of its wavelet coefficients and carry out the estimation in the field’s wavelet transform domain. Our approach models interscale dependencies of the wavelet coefficients and allows for smooth, edge, and occluded regions in the field. We obtain segmentations of the field and our results show that the field estimates yield accurate depictions of scene motion. The globally optimal nature of our estimation framework allows it to be applicable in scenes exhibiting large motion and in settings of ill-posed motion. Hence, our algorithms can also be used to determine accurate initializations for optical flow type estimation techniques, which use more sophisticated models but can only obtain locally optimal solutions that are heavily dependent on initial conditions. The performance is illustrated on several examples.

An enhanced version of the fast inter-prediction mode decision and variable block size motion estimation scheme for the H.264 video encoder is proposed in this work. In particular, we focus on the rate estimation model and thresholds for early termination. They are adaptively updated to enhance the coding speed and the rate-distortion performance. More details about the threshold setting and the mode decision processes are also provided. The improved algorithm can achieve a speed-up factor ranging from 70 to 150 times as compared to the full-search algorithm.

Lifting-based implementations of various discrete wavelet transforms applied in the temporal direction under motion compensation have recently become a very powerful tool in video compression research. We present in this paper a theoretical analysis of motion compensation in both transversal and lifted implementations of such transforms. We derive conditions for perfect reconstruction in the case of motion-compensated transversal discrete wavelet transform. We also derive conditions on motion transformation assuring that a motion-compensated lifting scheme is exactly equivalent to its transversal counterpart. In general, these conditions require that motion transformation allow composition and be invertible. Unfortunately, many motion models do not obey these properties, thus inducing subband decomposition errors (prior to compression). We propose an alternative approach to motion compensation in the case of Haar transform. This new approach poses no constraints on motion; motion-compensated lifted Haar transform exactly implements its transversal implementation, and the latter obeys perfect reconstruction, both regardless of motion transformation used. This new approach, however, does not extend to the 5/3 or any higher-order discrete wavelet transform.

Motion estimation algorithms are a key component for multimedia systems and optimization of these algorithms is still a topic of current research. Promising approaches try to integrate into the motion estimation process besides pure grey level similarities further types of information, contained in the image. Due to the moderate quality of this additional information the integration has to be performed rather conservatively in order to reduce the risk of an even dramatic degradation of the vector field quality in some cases. Up to now there is no robust algorithm available, which yields a noticeable improvement for all types of motion and image scenes, without causing a loss of quality in critical situations. Within the scope of this contribution the application of high performance segmentation for the enhancement of motion vector fields is analyzed. Starting from these results a new iterative concept for object based motion estimation is developed, which combines the results of a classic motion estimation with the information of image segmentation and features a high robustness against segmentation errors. The results of this new algorithm are analyzed on the basis of different objective evaluation criterions and compared to classic motion estimation algorithms.

In order to achieve high video coding efficiency, a new motion estimation and compensation algorithm is proposed based on Genetic Algorithm. This algorithm exploits the uniformity and correlation in the properties of the cluster of blocks called Super-Block. These Super-Blocks have adaptive boundaries that are used to partially generate initial population for fast convergence to global minimum. Rest of the population is generated using pure Random Number Generator (RNG). This population then generates offspring which then competes within itself by the virtue of it’s fitness to survive into the next generation. The fitness value in each generation is calculated by comparing the reference frame with the predicted frame. The algorithm stops after convergence or when maximum generations are reached. This algorithm compares well against conventional algorithms like FSA (Full Search Algorithm), One-Step Method or N-Step Method in terms of number of searches, complexity, robustness and scalability.

This paper deals with the problem of estimating multiple motions at points where these motions are overlaid. We present a new approach that is based on block-matching and can deal with both transparent motions and occlusions. We derive a block-matching constraint for an arbitrary number of moving layers. We use this constraint to design a hierarchical algorithm that can distinguish between the occurrence of single, transparent, and occluded motions and can thus select the appropriate local motion model. The algorithm adapts to the amount of noise in the image sequence by use of a statistical confidence test. The algorithm is further extended to deal with very noisy images by using a regularization based on Markov Random Fields. Performance is demonstrated on image sequences synthesized from natural textures with high levels of additive dynamic noise.

This paper investigates video coding with wavelet transforms applied in the temporal direction of a video sequence. The wavelets are implemented with the lifting scheme in order to permit motion compensation between successive pictures. We improve motion compensation in the lifting steps and utilize complementary motion-compensated signals. Similar to superimposed predictive coding with complementary signals, this approach improves compression efficiency. We investigate experimentally and theoretically complementary motion-compensated signals for lifted wavelet transforms. Experimental results with the complementary motion-compensated Haar wavelet and frame-adaptive motion compensation show improvements in coding efficiency of up to 3 dB. The theoretical results demonstrate that the lifted Haar wavelet scheme with complementary motion-compensated signals is able to approach the bound for bit-rate savings of 2 bits per sample and motion-accuracy step when compared to optimum intra-frame coding of the input pictures.

We propose an overlapped block motion compensation method, called adaptive windowing technique, to improve the performance of variable block-size motion compensation in H.264. First, we restrict the number of neighboring blocks to be overlapped. Then we design adaptive overlapping windows, where each weight is set to be inversely proportional to the distance between the current pixel and the neighboring block. The weights can be computed effciently in both the encoder and the decoder. Also, to further improve the prediction performance, we introduce the notion of reliability of a motion vector based on the block size, and fine-tune the weights according to the reliability. Extensive simulation results show that the proposed algorithm improves the performance of H.264 both objectively and subjectively.

In current interframe video compression systems, the encoder performs predictive coding to exploit the similarities of successive frames. The Wyner-Ziv Theorem on source coding with side information available only at the decoder suggests that an asymmetric video codec, where individual frames are encoded separately, but decoded conditionally (given temporally adjacent frames) could achieve similar efficiency. We propose a transformdomain Wyner-Ziv coding scheme for motion video that uses intraframe encoding, but interframe decoding. In this system, the transform coefficients of a Wyner-Ziv frame are encoded independently using a scalar quantizer and turbo coder. The decoder uses previously reconstructed frames to generate side information to conditionally decode the Wyner-Ziv frames. Simulation results show significant gains above DCT-based intraframe coding and improvements over the pixel-domain Wyner-Ziv video coder.

Recently we have witnessed a growing interest in the development of the subband/wavelet coding (SBC) technology, partly due to the superior scalability of SBC. Scalable coding provides great synergy with the universal media access applications, where media content is delivered to client devices of diverse types through heterogeneous channels. In this respect, SBC system provides flexibility in realizing different ways of media scaling, including scaling dimensions of SNR, spatial, and temporal. However, the selection of specific scalability operations given the bit rate constraint has always been ad hoc - a systematic methodology is missing. In this paper, we address this open issue by applying our content-based optimal scalability selection framework and adopting subjective quality evaluation. For this purpose we firstly explore the behavior of SNR-Spatial-Temporal scalability using Motion Compensated (MC) SBC systems. Based on the system behavior, we propose an efficient method for the optimal selection of scalability operator through content-based prediction. Our experiment results demonstrate that the proposed method can efficiently predict the optimal scalability operation with an excellent accuracy.

Leaky prediction layered video coding (LPLC) partially includes the enhancement layer in the motion compensated prediction loop, by using a leaky factor between 0 and 1, to balance the coding efficiency and error resilience performance. In this paper, rate distortion functions are derived for LPLC from rate distortion theory. Closed form expressions are obtained for two scenarios of LPLC, one where the enhancement layer stays intact and the other where the enhancement layer suffers from data rate truncation. The rate distortion performance of LPLC is then evaluated with respect to different choices of the leaky factor, demonstrating that the theoretical analysis well conforms with the operational results.

The increasing diversity of the characteristics of the terminals and networks that are used to access multimedia content through the internet introduces new challenges for the distribution of multimedia data. Scalable video coding will be one of the elementary solutions in this domain. This type of coding allows to adapt an encoded video sequence to the limitations of the network or the receiving device by means of very basic operations. Algorithms for creating fully scalable video streams, in which multiple types of scalability are offered at the same time, are becoming mature. On the other hand, research on applications that use such bitstreams is only recently emerging. In this paper, we introduce a mathematical model for describing such bitstreams. In addition, we show how we can model applications that use scalable bitstreams by means of definitions that are built on top of this model. In particular, we chose to describe a multicast protocol that is targeted at scalable bitstreams. This way, we will demonstrate that it is possible to define an abstract model for scalable bitstreams, that can be used as a tool for reasoning about such bitstreams and related applications.

Multiple Description Video Coding (MDC) and Layered Coding (LC) are both error-resilient source coding techniques used for transmission over error-prone channels. Both techniques generate multiple streams. The streams generated by MDC correspond to different descriptions of the same source whereas the streams produced by LC are differentiated as base and enhancement layer streams. Moreover whereas the MDC streams are independently decodable the decoding of the enhancement layer stream is dependent on the decoding of the base layer stream. In this work we concentrate on specific MDC and LC schemes, i.e. Multi-State Video Coding (MSVC) and Temporal Layered Coding (TLC). MSVC was introduced by John Apostolopoulos and it was showed that if each frame is transmitted in a separate packet and if motion information for each lost frame is also lost, MSVC outperforms Single Layer Coding (SC) in recovering from single as well as burst losses. Here we compared MSVC with TLC as an extension of SC based on transmission simulations over lossy channels under the assumption that the motion vectors are always available. Using different coding modes and specific reconstruction methods average reconstructed frame PSNR (peak signal to noise ratio) is measured and compared. Results show that when motion vectors are received TLC performs better than MSVC for every coding option tested. The performance difference is bigger for low motion sequences.

Contrary to predictive schemes such as hybrid video coding systems, orthonormal transform coding systems are immune to error accumulation in case of desynchronization between encoder and decoder. Therefore, these systems allow for drift-free data adaptation at bit stream level, thus, scalability. In t+2D interframe wavelet video coding, wavelet-based motion-compensated temporal filtering is employed, followed by spatial wavelet decomposition and bit plane coding. This allows for temporal, spatial, and SNR scalability. While motion compensation seems to be essential in this scheme to achieve excellent coding performance, it causes local violation of the orthonormality of the temporal transform. Particularly, motion compensated interframe wavelet systems employ predictive coding for certain occlusion areas. In case of reference mismatch between encoder and decoder, error accumulation occurs in these regions. In this paper we present an approach to adapt the encoder operating point for predictively coded regions, effectively eliminating the reference mismatch adaptively. An iterative algorithm for computation of the decoder reference at the encoder side is presented for t+2D systems. We show that this approach significantly increases overall coding performance, gaining up to 1 dB in PSNR. Furthermore, the optimized quantization algorithm presented in an earlier work can be applied more effectively, leading to more even noise distribution.

We present a novel scheme for error-resilient digital video broadcasting,using the Wyner-Ziv coding paradigm. We apply the general framework of systematic lossy source-channel coding to generate a supplementary bitstream that can correct transmission errors in the decoded video waveform up to a certain residual distortion. The systematic portion consists of a conventional MPEG-coded bitstream, which is transmitted over the error-prone channel without forward error correction.The supplementary bitstream is a low rate representation of the transmitted video sequence generated using Wyner-Ziv encoding. We use the conventionally decoded error-concealed MPEG video sequence as side information to decode the Wyner-Ziv bits. The decoder combines the error-prone side information and the Wyner-Ziv description to yield an improved decoded video signal. Our results indicate that, over a large range of channel error probabilities, this scheme yields superior video quality when compared with traditional forward error correction techniques employed in digital video broadcasting.

The new Motion JPEG 2000 standard is providing with some compelling features. It is based on an intra-frame wavelet coding, which makes it very well suited for wireless applications. Indeed, the state-of-the-art wavelet coding scheme achieves very high coding efficiency. In addition, Motion JPEG 2000 is very resilient to transmission errors as frames are coded independently (intra coding). Furthermore, it requires low complexity and introduces minimal coding delay. Finally, it supports very efficient scalability. In this paper, we analyze the performance of Motion JPEG 2000 in error-prone transmission. We compare it to the well-known MPEG-4 video coding scheme, in terms of coding efficiency, error resilience and complexity. We present experimental results which show that Motion JPEG 2000 outperforms MPEG-4 in the presence of transmission errors.

In this paper, a robust image transmission scheme based on JPEG2000 is proposed for packet erasure channels. Error resilience functionalities provided by JPEG2000 are utilized to control the source coding efficiency and the robustness according to channel conditions. Furthermore, together with the proposed interleaving scheme, some erasures can be recovered. Experimental results show the effectiveness of the scheme.

The robustness of the recently ratified video compression standard H.264/MPEG-4 AVC against channel errors is evaluated with the focus on rate distortion matters. After a brief introduction of the standard and an explanation of its error-resistant features, it is investigated how the error resilience tools of H.264 can be deployed best for packet-wise transmission as in ATM, H.323, and IP-based services. Further, the performances of two error concealment strategies for use in an H.264-conform decoder are compared to each other.

The imaging sensors are able to provide intuitive visual information for quick recognition and decision. However, imaging sensors usually generate vast amount of data. Thus, processing of image data collected in the sensor network for the purpose of energy efficient transmission poses a significant technical challenge. In particular, when a cluster of imaging sensors is activated to track certain moving target, multiple sensors may be collecting similar visual information simultaneously. With correlated image data, we need to intelligently reduce the redundancy among the neighboring sensors so as to minimize the energy for transmission, the primary source of sensor energy consumption. We propose in this paper a novel collaborative image transmission scheme for wireless sensor networks. First, we apply a shape matching method to coarsely register images to find out maximal overlap in order to exploiting the spatial correlation between images acquired from neighboring sensors. A transformation is generated according to the matching results. We encode the original image and the difference between the transformed image and reference image. Then, we transmit the coded bit stream together with the transformation parameters. This will significantly reduce the transmission energy comparing with transmitting two individual images independently. To exploiting the temporal correlation among images in the same sensor, we assume that the imaging sensors and the background scenes remain stationary over the data acquisition period. For a given image sequence, we transmit background image only once. A simple background subtraction method is employed to detect targets. Whenever targets are detected, only the regions of target and their spatial locations are transmitted to the monitoring center. At the monitoring center, the whole image can be reconstructed by fusing the background and the target image as well as its spatial location to further reduce energy consumption. Experimental results show that the transmission energy can be greatly reduced.

In this paper we consider the problem of robust image coding and packetization for the purpose of communications over slow fading frequency selective channels and channels with a shaped spectrum like those of digital subscribe lines (DSL). Towards this end, a novel and analytically based joint source channel coding (JSCC) algorithm to assign unequal error protection is presented. Under a block budget constraint, the image bitstream is de-multiplexed into two classes with different error responses. The algorithm assigns unequal error protection (UEP) in a way to minimize the expected mean square error (MSE) at the receiver while minimizing the probability of catastrophic failure. In order to minimize the expected mean square error at the receiver, the algorithm assigns unequal protection to the value bit class (VBC) stream. In order to minimizes the probability of catastrophic error which is a characteristic of progressive image coders, the algorithm assigns more protection to the location bit class (LBC) stream than the VBC stream. Besides having the advantage of being analytical and also numerically solvable, the algorithm is based on a new formula developed to estimate the distortion rate (D-R) curve for the VBC portion of SPIHT. The major advantage of our technique is that the worst case instantaneous minimum peak signal to noise ratio (PSNR) does not differ greatly from the averge MSE while this is not the case for the optimal single stream (UEP) system. Although both average PSNR of our method and the optimal single stream UEP are about the same, our scheme does not suffer erratic behavior because we have made the probability of catastrophic error arbitarily small. The coded image is sent via orthogonal frequency division multiplexing (OFDM) which is a known and increasing popular modulation scheme to combat ISI (Inter Symbol Interference) and impulsive noise. Using dual adaptive energy OFDM, we use the minimum energy necessary to send each bit stream at a particular probability of bit error. An added benefit of OFDM over serial transmission schemes is that some degree of progressiveness of SPIHT is preserved by transmitting both the VBC and LBC streams in parallel.

We address the problem of robust streaming of high-quality video over wireless local area networks in a home environment. By robust streaming, we mean maintaining the highest possible video quality and preventing interruptions to the video under varying bandwidth conditions, which may be due to distance, interference, obstructions, and existence of multiple streams. We propose an application-layer approach where we provide algorithms for dynamic on-line network bandwidth estimation and dynamic on-line adaptation of video rate according to the available network bandwidth. The proposed system employs a packet scheduler, and a video rate control and adaptation mechanism at the sender, and bandwidth measurement and feedback mechanisms at the receiver. Our bandwidth estimation approach uses the actual video data in real time by transmitting it in packet bursts; hence, separate test traffic is not required. Since the proposed method operates at the application layer, it is flexible and applicable to different local area network types and implementations. We propose an extension to multiple streams by providing an algorithm for joint rate allocation to multiple video streams over a network enabling network-adaptive simultaneous streaming of high-quality video.

We consider an unconventional procedure for communicating to the server the receipt of media packets for Internet video streaming. Instead of separately acknowledging each media packet as it arrives, we periodically send to the server a single acknowledgment packet, denoted rich acknowledgment, that contains information about all media packets that have arrived at the client by the time the rich acknowledgment is sent. We investigate rate-distortion optimized sender-driven streaming that employs rich acknowledgments. Performance gains of up to 1.3 dB for streaming packetized video content are observed over rate-distortion optimized sender-driven systems that employ conventional acknowledgments.

Progressive coding of 3D textured graphic models using a joint mesh-texture optimization technique is investigated in this work. The mesh and texture data of a model are first fed into their respective compression modules to result in a series of levels of details. Then, for a given viewpoint, a rate-distortion surface is generated using these mesh and texture data. Afterwards, an optimal path over the rate-distortion surface is determined by the steepest descent algorithm. To achieve progressive transmission, the mesh and texture are transmitted in a certain ratio along the optimal path to provide the best visual quality for the given viewpoint. For further generalization, a layered sampling algorithm is proposed to deal with an arbitrary viewing angle. The performance of the proposed joint mesh-texture progressive coding algorithm is demonstrated by experimental results.

In this paper we propose an image coding scheme based on Block Turbo Codes (BTC) for protecting SPIHT coded images transmitted over wireless channels. An Unequal Error Protection (UEP) scheme using different product codes is also presented. The optimization of the UEP scheme under a transmission budget constraint is investigated. Performance of the proposed system are first evaluated over Gilbert-Elliott (GE) channels. They are then compared to the well-known system of Sherwood and Zeger, through numerical simulations over a single path Rayleigh fading channel. Then, we present an optimized mobile image transmission framework involving a multiuser asynchronous DS-CDMA transmission system over Rayleigh multipath channels. Simulation results for the 512x512 Lena image, transmitted over the ITU Vehicular A channel, show that a significant gain in PSNR is obtained by using the SIC/RAKE multiuser detector in comparison with a conventional detection by the RAKE receiver. Furthermore, performance are also improved when introducing the UEP scheme.

Flat panel displays, such as liquid crystal displays (LCDs), typically emit light during the whole frame time. In contrast, traditional cathode ray tubes (CRTs) emit light as very short pulses, which gives the CRT a better dynamic resolution. As a consequence, LCDs suffer from motion artifacts, which are visible as a blurring of moving objects. Based on a straightforward frequency domain analysis that takes into account the eye tracking of the viewer, we propose a method for reducing these artifacts. This method, `motion compensated inverse filtering', uses motion vectors to apply a pre-correction to the video data. As such, we are able to recover the sharpness of moving images on LCDs to a large extent.

Yen's generalisation of the sampling theorem has been proposed as the theoretical solution for de-interlacing by Delogne and Vandendorpe. The solution results in a vertical interpolation filter with coefficients that depend on the motion vector value, which uses samples that exist in the current field and additional samples from a neighbouring field shifted over (part of) a motion vector. We propose a further generalisation, where we design vector-adaptive inseparable 2D filters, which use samples from the current and the motion compensated previous field that are not available for all vectors on a vertical line. The resulting inseparable filters give a better interpolation quality at a given number of input pixels. We will show that the algorithm can be made robust against the sensitivity to inaccurate motion vectors.

The motion-compensated temporal filtering is an essential part of a scalable wavelet-based video coding scheme, which applies a temporal wavelet transform in the motion direction over the frames of a video sequence. The lifting structure of the temporal filter bank tackled in this paper involves a predict operator which makes use of two motion vector fields to bidirectionally predict frames from their neighbouring ones. We show in this paper that there exists an optimal algorithm allowing to estimate jointly these two motion vector fields subject to an optimization criterion directly related to the coding of the detail subbands. We provide an iterative suboptimal form of this algorithm implementing this approach. We show that this algorithm provides substantial gains in terms of PSNR, with the same complexity as a separate estimation of the two motion vector fields.

In a Standard Definition (SD) Television system, the Y:U:V video format 4:2:2 with chrominance sub-sampling is widely used. With the advent of High Definition (HD) television, the 4:4:4 format is required for high-performance TV. High-quality up-sampling methods have been developed to perform a resolution conversion from Standard Definition (SD) signal to HD signal. Although these algorithms have been designed for spatial scaling of luminance, they may be adapted and used to up-sample the low-resolution components U,V (4:2:2) to a high-resolution UV-colour format (4:4:4). In this paper, a content-adaptive up-scaling method for chrominance is proposed, with interpolation filters that adapt to the local structure of both luminance and chrominance data. Optimal filters were computed from a large video data set in different colour formats, such that original high-resolution data in a 4:4:4 format was reconstructed from low-resolution colour data, on the basis of the Least Mean Square (LMS) criterion. By combining edge information of both luminance and chrominance, the edge in the chrominance signal can be detected more accurately, thus exploiting the wider bandwidth of the luminance signal.

Most of the researches are concentrated on motion detection in gray value image sequences and the methods for motion detection are based on background subtraction or on temporal gray value derivatives. The methods based on background subtraction, including auto-adaptive ones, meet difficulties in presence of illumination changes and of slowly moving objects and need to be re-initialized from time to time. The methods based on temporal derivatives are in general sensible to noise. Color images containing much richer information than the gray value ones, it would be interesting to use them to better detect moving objects. In this paper, we address the problem of motion detection in color image sequences and the problems of illumination changes and shadow elimination. Our motion detection method is based on fuzzy segmentation of the color difference image in help of non-symmetrical π membership functions. The elimination of false moving objects detected due to illumination change is realized by combining the background subtraction method with the temporal derivative method and motion continuity. Shadows are removed by comparing the color of mobile pixels detected in the current frame with that in the precedent frame in HSL color space. Experimental results are reported.

For analog color television standards such as PAL and NTSC, the transmission of color (C) takes place within the band available for the luminance (Y). At the television receiver, the required separation of Y and C can only be imperfect as both components now share the same frequency space. Modern televisions apply so-called comb-filters that exploit the opposite sub-carrier phase of correlated samples to separate both components. However, cross-talk artifacts and loss of resolution will occur in situations where no sufficiently correlated samples meet the strict opposite phase requirement. In this paper, a novel Y/C separation method is presented that is able to use samples with non-opposite sub-carrier phases.

Previously we have presented a method for selective image sharpness enhancement. Our method is based on the simultaneous nonlinear reaction-diffusion time-evolution equipped with a nonlinear diffusion term, a reaction term and an isotropic peaking term, and it can sharpen only degraded edges blurred by several causes without increasing the visibility of nuisance factors such as random noise. This paper applies our simultaneous nonlinear reaction-diffusion method to removal of image blurs due to image motion. The motion blur is not only shift-variant but also anisotropic. To adapt our simultaneous nonlinear reaction-diffusion method for motion de-blurring, we replace the isotropic Laplacian operator, included in the isotropic peaking term of our prototypal method, with the anisotropic operator considering the direction of the estimated image motion. Preliminarily experiments using artificially generated test images show that our method achieves excellent motion de-blurring.

We describe an advanced Human Computer Interaction (HCI) model that employs photo-realistic virtual humans to provide digital media users with information, learning services and entertainment in a highly personalized and adaptive manner. The system can be used as a computer interface or as a tool to deliver content to end-users. We model the interaction process between the user and the system as part of a closed loop dialog taking place between the participants. This dialog, exploits the most important characteristics of a face-to-face communication process, including the use of non-verbal gestures and meta communication signals to control the flow of information. Our solution is based on a Virtual Human Interface (VHI) technology that was specifically designed to be able to create emotional engagement between the virtual agent and the user, thus increasing the efficiency of learning and/or absorbing any information broadcasted through this device. The paper reviews the basic building blocks and technologies needed to create such a system and discusses its advantages over other existing methods.

We propose a novel multistage facial feature extraction approach using a combination of 'global' and 'local' techniques. At the first stage, we use template matching, based on an Edge-Orientation-Map for fast feature position estimation. Using this result, a statistical framework applying the Active Shape Model (ASM) is initialized and deformed to fit the real face image. In our proposal, we use a 2-D pattern search-and-fitting scheme guiding the deformation process, which provides more robustness and faster convergence than the traditional ASM. Our proposed approach for feature extraction shows good results dealing with a test set composed of faces images which are quite dissimilar with the faces used for the statistical training of the face model. The convergence area of our proposed technique almost quadruples compared to the ASM, while the amount of faces doubles for which the convergence is reached. The total processing for feature extraction takes less than 1 second for 250x250 face images on a Pentium-IV PC (1.7GHz).

The lack of eye contact in video conference degrades the user experience. This problem has been known and studied for many years. There are hardware-based solutions to the eye gaze problem; however, these specialized systems are not generally accessible. This paper assumed the depth of the scene taken from the monocular camera is normal distributed on the XY plane on the real world, the plane parallel to the view plane. The three dimensional normal curve is initially estimated from the face model of the user. By performing rotation operation on the normal curve, the orientation of the face is rectified. The approach suggested in this paper is fast and effective. It has the advantages of 3D modelling, but could save the steps on complex registration, texture mapping and rendering.

Technical advance in creating, storing digital media in daily life enables computers to capture human life and remember it as people do. A critical point with digitizing human life is how to recall bits of experience that are associated by semantic information. This paper proposes a technique for structuring dynamic digital object based on MPEG-21 Digital Item (DI) in order to recall human’s memory and providing interactive TV service on family tree albuming system as one of its applications. DIs are a dynamically reconfigurable, uniquely identified, described by a descriptor language, logical unit for structuring relationship among multiple media resources. Digital Item Processing (DIP) provides the means to interact with DIs to remind context to user, with active properties where objects have executable properties. Each user can adapt DIs’ active properties to tailor the behavior of DIs to match his/her own specific needs. DIs’ technologies in Intellectual Property Management and Protection (IPMP) can be used for privacy protection. In the interaction between the social space and technological space, the internal dynamics of family life fits well sharing family albuming service via family television. Family albuming service can act as virtual communities builders for family members. As memory is shared between family members, multiple annotations (including active properties on contextual information) will be made with snowballing value.

Several color matching algorithms are proposed to merge two or more video inputs of smaller sizes into one single video output of a larger size with a wider field of view for the video mosaic application. The main challenge is to remove apparent seam lines between image boundaries. All developed algorithms share the same basic idea with different implementation details. That is, color differences between input images are first compensated using either histogram equalization or polynomial-based contrast stretching techniques. Then, a linear filtering technique is adopted to remove seam lines between image boundaries. The algorithms are developed in both the pixel and the DCT domains. The compressed-domain processing is attractive since it reduces the computational complexity. It is shown by experimental results that the color matching problem can be solved satisfactorily even in the compressed domain.

Color reproduction systems using multispectral imaging techniques make it possible to accurately reproduce the color of the original object under various viewing illuminants. In this paper, a multispectral image compression method for high fidelity color reproduction is proposed in consideration of color degradation. In the proposed method, a spectral transform and a nonlinear quantization designed to reduce colorimetric error are combined with the discrete wavelet transform in JPEG2000. Through the experiments using some 16-band multispectral images, it is confirmed that the proposed method reduces the average and the maximum color differences in L*a*b* color space in comparison with the conventional methods.

Scalable video coding via motion-compensated spatio-temporal wavelet decompositions has gained a great interest in transmission over heterogeneous networks due to the flexibility of the resulting bitstream to accomodate various network conditions as well as user capabilities and demands. Meanwhile, the adaptation of the bitstream to the available bandwidth can lead to discarding the finest detail subbands during the transmission. The loss of these subbands would result in a low quality, oversmoothed reconstructed sequence. In this paper, we present a statistical spatio-temporal model between the wavelet coefficients and we show its efficiency in the prediction of the high frequency subbands and in the quality enhancement of the scalable video.

Imagery sensors have been one indispensable part of the detection and recognition systems. They are widely used to the field of surveillance, navigation, control and guide, et. However, different imagery sensors depend on diverse imaging mechanisms, and work within diverse range of spectrum. They also perform diverse functions and have diverse circumstance requires. So it is unpractical to accomplish the task of detection or recognition with a single imagery sensor under the conditions of different circumstances, different backgrounds and different targets. Fortunately, the multi-sensor image fusion technique emerged as important route to solve this problem. So image fusion has been one of the main technical routines used to detect and recognize objects from images. While, loss of information is unavoidable during fusion process, so it is always a very important content of image fusion how to preserve the useful information to the utmost. That is to say, it should be taken into account before designing the fusion schemes how to avoid the loss of useful information or how to preserve the features helpful to the detection. In consideration of these issues and the fact that most detection problems are actually to distinguish man-made objects from natural background, a fractal-based multi-spectral fusion algorithm has been proposed in this paper aiming at the recognition of battlefield targets in the complicated backgrounds. According to this algorithm, source images are firstly orthogonally decomposed according to wavelet transform theories, and then fractal-based detection is held to each decomposed image. At this step, natural background and man-made targets are distinguished by use of fractal models that can well imitate natural objects. Special fusion operators are employed during the fusion of area that contains man-made targets so that useful information could be preserved and features of targets could be extruded. The final fused image is reconstructed from the composition of source pyramid images. So this fusion scheme is a multi-resolution analysis. The wavelet decomposition of image can be actually considered as special pyramid decomposition. According to wavelet decomposition theories, the approximation of image (formula available in paper) at resolution 2j+1 equal to its orthogonal projection in space , that is, where Ajf is the low-frequency approximation of image f(x, y) at resolution 2j and , , represent the vertical, horizontal and diagonal wavelet coefficients respectively at resolution 2j. These coefficients describe the high-frequency information of image at direction of vertical, horizontal and diagonal respectively. Ajf, , and are independent and can be considered as images. In this paper J is set to be 1, so the source image is decomposed to produce the son-images Af, D1f, D2f and D3f. To solve the problem of detecting artifacts, the concepts of vertical fractal dimension FD1, horizontal fractal dimension FD2 and diagonal fractal dimension FD3 are proposed in this paper. The vertical fractal dimension FD1 corresponds to the vertical wavelet coefficients image after the wavelet decomposition of source image, the horizontal fractal dimension FD2 corresponds to the horizontal wavelet coefficients and the diagonal fractal dimension FD3 the diagonal one. These definitions enrich the illustration of source images. Therefore they are helpful to classify the targets. Then the detection of artifacts in the decomposed images is a problem of pattern recognition in 4-D space. The combination of FD0, FD1, FD2 and FD3 make a vector of (FD0, FD1, FD2, FD3), which can be considered as a united feature vector of the studied image. All the parts of the images are classified in the 4-D pattern space created by the vector of (FD0, FD1, FD2, FD3) so that the area that contains man-made objects could be detected. This detection can be considered as a coarse recognition, and then the significant areas in each son-images are signed so that they can be dealt with special rules. There has been various fusion rules developed with each one aiming at a special problem. These rules have different performance, so it is very important to select an appropriate rule during the design of an image fusion system. Recent research denotes that the rule should be adjustable so that it is always suitable to extrude the features of targets and to preserve the pixels of useful information. In this paper, owing to the consideration that fractal dimension is one of the main features to distinguish man-made targets from natural objects, the fusion rule was defined that if the studied region of image contains man-made target, the pixels of the source image whose fractal dimension is minimal are saved to be the pixels of the fused image, otherwise, a weighted average operator is adopted to avoid loss of information. The main idea of this rule is to store the pixels with low fractal dimensions, so it can be named Minimal Fractal dimensions (MFD) fusion rule. This fractal-based algorithm is compared with a common weighted average fusion algorithm. An objective assessment is taken to the two fusion results. The criteria of Entropy, Cross-Entropy, Peak Signal-to-Noise Ratio (PSNR) and Standard Gray Scale Difference are defined in this paper. Reversely to the idea of constructing an ideal image as the assessing reference, the source images are selected to be the reference in this paper. It can be deemed that this assessment is to calculate how much the image quality has been enhanced and the quantity of information has been increased when the fused image is compared with the source images. The experimental results imply that the fractal-based multi-spectral fusion algorithm can effectively preserve the information of man-made objects with a high contrast. It is proved that this algorithm could well preserve features of military targets because that battlefield targets are most man-made objects and in common their images differ from fractal models obviously. Furthermore, the fractal features are not sensitive to the imaging conditions and the movement of targets, so this fractal-based algorithm may be very practical.

This paper presents an algorithm to predict interpolative motion compensation (MC) mode without computing interpolative SADs (Sum of Absolute Difference) for B-picture coding in an MPEG-2 video encoder. Firstly, an initial MC mode is selected among forward frame/field MC and backward frame/field MC for each macroblock (MB) in a Bpicture. This is accomplished by comparing four minimum SADs corresponding to the above-mentioned MC modes, and choosing a minimum SAD among them. Secondly, if the SAD corresponding to the selected mode is less than a specific threshold, we set the selected MC mode to the final MC mode. Otherwise, an MC mode of the current MB is determined between interpolative frame MC and interpolative field MC. If the sum of SADs of forward frame MC mode and backward frame MC mode is less than that of SADs of forward field MC mode and backward field MC mode, the final MC mode is set to an interpolative frame MC. Otherwise, the final MC mode is set to an interpolative field MC mode. Experimental results show that the proposed algorithm is comparable with as brute-force MC mode selection method as in conventional MPEG-2 video encoding. In addition, the proposed algorithm may noticeably reduce the memory bandwidth in implementation of MPEG-2 video encoder.

In this paper, we analyze some properties of the orthogonal Gaussian-Hermite moments and propose a new method to detect the moving objects using the orthogonal Gaussian-Hermite moments. For the segmentation of moving objects in such moment images, we propose the fuzzy relaxation method (FRM) and 3D morphological relaxation method (3DMRM). The experiment results are reported, which show the robustness of our methods.

This paper presents a practical framework for creating and visualizing interactive 3-D media using a system of uncalibrated projector-cameras. The proposed solution uses light patterns that temporally encode the projector’s coordinate system to solve the traditionally challenging multiframe correspondence problem by straightforward decoding instead of computational multiframe optimization. Two sets of coded light patterns (black/white stripes and colored 2x2 blocks, both of varying spatial resolutions) are presented and compared. The resulting correspondences are directly used as a compelling form of interactive 3-D media through described techniques including three-frame view synthesis, multiframe view synthesis using multiple three-frame groupings, and even single-camera view interpolation. It is shown that adapting the rendering order of the correspondences with respect to the projector’s coordinate system ensures the correct visibility for the synthesized views. Experimental results demonstrate that the framework works well for various real-world scenes, even including those with multiple objects and textured surfaces. The framework, along with the resulting correspondences, also has implications in many other computer vision and image processing applications, especially those that require multiframe correspondences.

This paper addresses an adaptive motion vector prediction algorithm to improve the performance of video encoder. The block-based motion vector is characterized by the local statistics so that the coefficients of LS-based linear motion predictor can be optimized. However, it requires very expensive computational cost. The proposed algorithm using LS approach with spatially varying motion-directed property adaptively controls the coefficients of the motion predictor and reduces the computational cost as well as the motion prediction error. Experimental results show the capability of the proposed algorithm.

In this paper, based on the characteristics of Turbo Codes and Trellis Coded Modulation (TCM), an improved method Turbo Trellis Coded Modulation (Turbo TCM) with two different mapping strategies is presented over AWGN (add white gauss noise) channels. The performance of different mapping strategies of Turbo TCM in AWGN channels for image transmission is analyzed and investigated. Furthermore, taking 16QAM modulation of Turbo TCM as the example, two different signal constellations are compared and computer simulation results are showed.

Magnetic resonance image analysis by computer is useful to aid diagnosis of malady. We present in this paper a automatic segmentation method for principal brain tissues. It is based on the possibilistic clustering approach, which is an improved fuzzy c-means clustering method. In order to improve the efficiency of clustering process, the initial value problem is discussed and solved by combining with a histogram analysis method. Our method can automatically determine number of classes to cluster and the initial values for each class. It has been tested on a set of forty MR brain images with or without the presence of tumor. The experimental results showed that it is simple, rapid and robust to segment the principal brain tissues.

Identifying moving objects from a video sequence is a fundamental and critical task in many computer-vision applications. A common approach is to perform background subtraction, which identifies moving objects from the portion of a video frame that differs significantly from a background model. There are many challenges in developing a good background subtraction algorithm. First, it must be robust against changes in illumination. Second, it should avoid detecting non-stationary background objects such as swinging leaves, rain, snow, and shadow cast by moving objects. Finally, its internal background model should react quickly to changes in background such as starting and stopping of vehicles. In this paper, we compare various background subtraction algorithms for detecting moving vehicles and pedestrians in urban traffic video sequences. We consider approaches varying from simple techniques such as frame differencing and adaptive median filtering, to more sophisticated probabilistic modeling techniques. While complicated techniques often produce superior performance, our experiments show that simple techniques such as adaptive median filtering can produce good results with much lower computational complexity.

This paper proposes a fully digital auto-focusing algorithm for restoring the image with differently out-of-focused objects, which can restore background as well as all objects. In this paper, we assume that out-of-focus blur is isotropic such as circle of confusion (COC) or two-dimensional Gaussian blur. Therefore, the proposed algorithm can segment and estimate the point spread function (PSF) by using the size of ramp in the one-dimensional step response. The proposed algorithm can be developed by object-based image segmentation and restoration algorithm. Experimental results show that the proposed object-based image restoration algorithm can efficiently remove the space-variant out of focus blur from the image with multiple blurred objects.

In this paper, we propose a novel scalable video coding algorithm based on adaptively weighted multiple reference frame method. To improve the coding efficiency in enhancement layer, double reference picture and double motion vector algorithms are employed for motion compensated prediction. In addition, the enhancement layer is predicted by the adaptively weighted sum of the double motion compensated frames in the enhancement layer and the the current frame in the base layer, according to the input video characteristics. By employing adaptive reference selection scheme at the decoder, the proposed method can reduce the drift problem significantly. experimental results show that the proposed algorithm yields higher PSNR performance compared with the fixed weighted scheme and the H.263+ for various packet loss rate channel conditions.

Camera motion estimation is very important for indexing and retrieving video information. In this paper, we propose a robust camera motion estimation and classification algorithm. Our camera motion estimation algorithm consists of optical flow estimation, iterative RANSAC (RANdom SAmple Consensus) multiple motion estimation, and long-term camera motion estimation through a shortest-path search. In this approach, we first estimate multiple global affine motions from the computed optical flow field for every frame in the video sequence. Then, the long-term camera motion is determined from searching a shortest path in a graph of cascaded nodes of global motions. After the camera motion is determined for the whole video, we apply an artificial neural network to classify the camera motion type. This neural network is trained from a large set of different types of camera motion data. We show accurate camera motion classification results through experiments on real videos.

In embedded multimedia applications, improving video quality under constraints of bandwidth and storage is an important problem. In this paper, we discuss the relationship among picture resolution, quantization precision and subjective quality in video coding for embedded systems. Then we propose a principle of tradeoff between picture resolution and quantization precision. Video coding based on the tradeoff principle can achieve higher subjective quality at low bitrates, and significantly reduce the burden of decoders. Experimental results on both MPEG-2 codec and H.264 codec prove that the tradeoff principle is valuable and feasible for embedded systems.

In this paper, early-stop and Motion Vector (MV) re-use approaches are proposed for the MPEG-2 to H.264 transcoding to reduce the computation of the variable block-size motion estimation. By combining the two approaches, the number of MV search points is reduced by more than 80% without significantly affecting the video quality. The proposed approaches can also be used in fast variable block-size motion estimation for the H.264 video encoding.

In the framework of multimedia communication over heterogeneous networks and variable quality of service, there are strong needs in robust and scalable video coding schemes. In order to reduce robustness problems of traditional finegrain scalable (FGS) video coding scheme, in this paper, we present a scalable texture-based video coding scheme for robust functionalities. To this purpose, the different frames are coded using an analysis-synthesis coding scheme with a progressive refinement layer. The robustness of the scheme is increased thanks to the ability to decode independently the different frames of the video sequence.

In this paper, we present a novel hybrid image coding scheme for real-time applications of computer screen video transmission. Based on the Mixed Raster Content (MRC) multilayer imaging model, the background picture is compressed with lossy JPEG, and the foreground layer consisting of text and graphics is compressed with a block-based lossless coding algorithm, which integrates shape-based coding, palette-based coding, palette reuse, and LZW algorithm. The key technique is to extract text and graphics from background pictures accurately and with low complexity. Shape primitives, such as lines, rectangles, and isolated pixels with prominent colors, are found to be significant clues for textual and graphical contents. The shape-based coding in our lossless algorithm provides intelligence to extract the computer-generated text and graphics elegantly and easily. Experimental results demonstrate the efficiency and low complexity of our proposed hybrid image coding scheme.

In this paper, we introduce a new supervised learning method of a Bayesian network for user preference models. Unlike other preference models, our method traces the trend of a user preference as time passes. It allows us to do online learning so we do not need the exhaustive data collection. The tracing of the trend can be done by modifying the frequency of attributes in order to force the old preference to be correlated with the current preference under the assumption that the current preference is correlated with the near future preference. The objective of our learning method is to force the mutual information to be reinforced by modifying the frequency of the attributes in the old preference by providing weights to the attributes. With developing mathematical derivation of our learning method, experimental results on the learning and reasoning performance on TV genre preference using a real set of TV program watching history data.

The fact that perception of facial beauty may be a universal concept has long been debated amongst psychologists and anthropologists. In this paper, we performed experiments to evaluate the extent of beauty universality by asking a number of diverse human referees to grade a same collection of female facial images. Results obtained show that the different individuals gave similar votes, thus well supporting the concept of beauty universality. We then trained an automated classifier using the human votes as the ground truth and used it to classify an independent test set of facial images. The high accuracy achieved proves that this classifier can be used as a general, automated tool for objective classification of female facial beauty. Potential applications exist in the entertainment industry and plastic surgery.

Block based texture synthesis algorithms have shown better results than others as they help to preserve the global structure. Previous research has proposed several approaches in the pixel domain, but little effort has been taken in the synthesis of texture in a multiresolution domain. We propose a multiresolution framework in which coefficient-blocks of the spatio-frequency components of the input texture are efficiently stitched together to form the corresponding components of the output texture. We propose two algorithms to this effect. In the first, we use a constant block size throughout the algorithm. In the second, we adaptively split blocks so as to use the largest possible block size in order to preserve the global structure, while maintaining the mismatched error of the overlapped boundaries below a certain error tolerance. Special consideration is given to minimization of the computational cost, throughout the algorithm designs. We show that the adaptation of the multiresolution approach results in a fast, cost-effective, flexible texture synthesis algorithm that is capable of being used in modern, bandwidth-adaptive, real-time imaging applications. A collection of regular and stochastic test textures is used to prove the effectiveness of the proposed algorithm.

Human computer interaction is an active area of research that focuses on how humans can better work with machines. In order for this to happen, there must be an enabling technology that creates a natural interface between man and machine. In this paper an enabling technology in the form of a camera based man-machine interface is described. This system uses smart cameras to analyze a scene directly in front of a computer display. The analysis determines where a user has touched the display and then treats that information is a mouse click thereby allowing the computer to be controlled. There are significant technological problems that have been overcome in order to make the system robust enough for commercialization. System architecture and operation is described and some interesting advantages of using machine vision are presented. The interface is also shown to be compatible with very large computer displays thus creating a very natural interface with familiar usage paradigms.

Video encoding and streaming over wireless ad hoc network operates under severe conditions, such as time-varying channel characteristics with bursty errors, limit power for data transmission, and dynamic topology of the self-organized network, stringent time delay for packet delivery, etc. Due to the dynamic topology, complex mechanism, and time-varying nature of the wireless ad hoc network, the network system and the streaming service often exhibit unpredictable behaviors. The ultimate goal in the wireless video streaming service is to provide the end user with the best possible video presentation quality. The video streaming quality, often measured by the end-to-end picture distortion, is affected by the scene coding characteristics S of the input video, and the configuration of the network parameters N which includes bandwidth, transmission power, bit error ratio, delay, etc. This brings up the following important and challenging issue: given an input video with scene characteristics S and a network configuration N, what is the average video streaming quality the receiver could expect the system to provide. To address this issue, in this work, we examine the behaviors and constraints of the major components in the streaming system, and propose an across-layer framework to model, control, and optimize the end-to-end video streaming quality.

In this paper, we consider the compression of high-definition video sequences for bandwidth sensitive applications. We show that down-sampling the image sequence prior to encoding and then up-sampling the decoded frames increases compression efficiency. This is particularly true at lower bit-rates, as direct encoding of the high-definition sequence requires a large number of blocks to be signaled. We survey previous work that combines a resolution change and compression mechanism. We then illustrate the success of our proposed approach through simulations. Both MPEG-2 and H.264 scenarios are considered. Given the benefits of the approach, we also interpret the results within the context of traditional spatial scalability.

An approach to extract traffic events by integrating the low-level, middle-level, and high-level feature extraction modules is developed in this research. To be more specific, the low-level module extracts features such as motion, size, and location. The middle-level module builds a bridge between the road surface plane in the real world and the captured image plane by geometric analysis. Finally, the high-level module looks for traffic events such as "traffic jam", "lane change", and "traffic rule violation", which require the understanding of the video contents in a specific knowledge domain. In the high-level module, various traffic events are related to motion characteristics obtained from the middle-level module. It is demonstrated by experimental results that the proposed system can achieve robust traffic event extraction. The effectiveness of the proposed technique is analyzed. Conventional traffic event extraction methods demand the knowledge of capturing conditions for camera calibration. This requirement can be greatly relaxed in our proposed scheme.

A lossless compression algorithm of 3D point data is proposed in this work. QSplat is one of the efficient rendering methods for 3D point data. In QSplat, each point is assigned a sphere, and the geometry and normal data are stored in the hierarchical structure of bounding spheres. To compress QSplat data, child spheres are sorted based on their limit radii to constrain the indices for the geometry data. Then, the radii and the positions of spheres are encoded separately using the reduced index sets. Also, each normal is encoded using the parent normal context, and the normal indices are reduced by the normal cone information. Simulation results show that the proposed algorithm achieves a high compression ratio by combining the reduced index sets with the context-based entropy coding.

In this paper, computation reduction algorithm of motion estimation is developed for MPEG4 video encoders. Our focus is on the computation reduction for block based integer and half pixel search. A method is proposed to reduce the search points and computation with little extra tradeoff and small quality downgrade. In MPEG4, the motion estimation is mainly composed of three major parts: 1) The Macroblock based integer pixel search. 2) The block based integer pixel search. 3) The half pixel search. The Macroblock based integer pixel search usually consumes the most computation power. Therefore many fast search algorithms are employed instead of the full search. In contrast, as to 2), 3), since the search range is quite small, usually the full search method is used. With many novel algorithms the computation requirements in 1) decrease greatly. Hence, the percentage of 2) and 3) are increasing and come to an important position for computation reduction. The method proposed in this paper jointly considered the ME, DCT and quantization. It takes advantage of the idea in both MB and block layer. The computation reduction is obvious from experiment. Because the full search algorithm is still in use, the quality downgrade is little compared to the old computation reduction algorithms.

This paper presents an accurate bit rate and quality control algorithm for 3D multiview sequences. We remodel the quadratic rate-distortion model for 3D multiview sequences based on their picture types. The proposed method consists of two levels for more accurate bits rate control. In a frame level, it sets quantization parameters based on HVS (Human Visual System) and reduces error bits by remodeling rate-quantization. In a MB level, rate control is activated to provide stricter buffer regulations and higher bit rate encoding based on the target bits and the quantization parameters calculated at the frame level. The proposed algorithm shows improvements in rate control as well as in PSNR compared to the conventional methods. It also provides more accurate rate control and higher image quality than TM5 and TMN8 at various bits rates.

In the paper we advocate image compression technique in the scope of distributed source coding framework. The novelty of the proposed approach is twofold: classical image compression is considered from the positions of source coding with side information and, contrarily to the existing scenarios, where side information is given explicitly, side information is created based on deterministic approximation of local image features. We consider an image in the transform domain as a realization of a source with a bounded codebook of symbols where each symbol represents a particular edge shape. The codebook is image independent and plays the role of auxiliary source. Due to the partial availability of side information at both encoder and decoder we treat our problem as a modification of Berger-Flynn-Gray problem and investigate a possible gain over the solutions when side information is either unavailable or available only at decoder. Finally, we present a practical compression algorithm for passport photo images based on our concept that demonstrates the superior performance in very low bit rate regime.

In this paper, we present an adaptive motion search range decision algorithm for low-power video encoder. The performance, computation, and power consumption of block matching motion estimation algorithms in video coding standards depends on the motion search range. The motion search range to restrict the motion candidates controls the trade-off between the motion accuracy and the encoder complexity. The proposed algorithm adaptively determines the motion search range by local statistics of the neighbor blocks, resulting in dramatic reduction of computational cost of video encoder without the loss of coding efficiency. Experimental results show that the proposed algorithm speeds up encoding time by 1.5-2.8 times, and reduces power consumption to 2.5-7.0 times.

Video coding is used under the hood of a lot of multimedia applications, such as video conferencing, digital storage media, television broadcasting, and internet streaming. Recently, new standards-based and proprietary technologies have emerged. An interesting problem is how to evaluate these different video coding solutions in terms of delivered quality. In this paper, a PSNR-based approach is applied in order to compare the coding potential of H.264/AVC AHM 2.0 with the compression efficiency of XviD 0.9.1, DivX 5.05, Windows Media Video 9, and MC-EZBC. Our results show that MPEG-4-based tools, and in particular H.264/AVC, can keep step with proprietary solutions. The rate-distortion performance of MC-EZBC, a wavelet-based video codec, looks very promising too.

JPEG 2000 is a flexible standard for coding many different types of images. This flexibility will be increased in the near future with the JP3D standard, an extension of JPEG 2000 to three dimensions. This paper presents a set of techniques to code three-dimensional textured parametric surfaces, common in Computer Graphics applications, using JPEG 2000 and possibly using extensions from the newer JP3D standard. These coding techniques generate compressed files of much smaller size than conventional file formats for storing three-dimensional objects. This is because the proposed coding techniques are able to exploit the strong correlation that exists between neighboring surface points in a parametric representation. Furthermore, it is possible to encode parametric surfaces which change through time, something that is not available in many conventional file formats. The reversible compression mode of JPEG 2000 ensures that threedimensional surfaces can be recovered without error after decompression. This is an important requirement for surfaces used in medical imaging, for example. This paper will demonstrate how a large variety of surfaces can be coded with JPEG 2000. The techniques presented here impose only one, but important, constraint on the surfaces: they must obey a parametric representation.

For real-time imaging in surveillance applications, image fidelity is of primary importance to ensure customer confidence. The obtained image fidelity is a result from amongst others dynamic range expansion and video signal enhancement. The dynamic range of the signal needs adaptation, because the sensor signal has a much larger range than the standard CRT display. The signal enhancement should accommodate for the widely varying light and scene conditions and user scenarios of the equipment. This paper proposes a new system to combine dynamic range and enhancement processing, offering a strongly improved picture quality for surveillance applications. The key to our solution is that we use Non-Linear Processing (NLP) with a so-called Constrained Histogram Range Equalization (CHRE). The NLP transforms the digitized high-dynamic luminance sensor signal such that details of the low-luminance parts are enhanced, while avoiding detail losses in the high-luminance areas. The CHRE technique enhances visibility of the global contrast for the camera signal without significant information loss in the statistically less relevant areas. Evaluations of this proposal have shown clear improvements of the perceptual image quality. An additional advantage is that the new scheme is adaptable and allows the concatenation of further enhancement techniques without sacrificing the obtained picture quality improvement.

Compression of digitized video highly depends on, and varies with, the signal to be compressed. The relation of quantizer, distortion and rate needs to be modeled when control in a system involving video compression is asked for. An experimental analysis of the optimized video codec AVC of ITU-T and MPEG shows that its rate behavior can be modeled accurately enough to predict bit-rate on macroblock-level. Given information of allocated rates from a pre-encoding analysis step, the bit-rate profile at any different quantizer setting for the video can be predicted. Experiments, comparing predicted rate against actual encoding rate show good performance of the model.These reflect the performance of the model in rate-control schemes. A simple example pre-analysis rate-control based on the model will determine beforehand a possible rate-profile that the actual encoding should be able to follow with small quality variations. For a signal of varying complexity, a varying number of bits will be used to obtain constant quality. Such variations are limited by peak bit-rates and buffer-sizes that are defined by hypothetical reference decoders in AVC.

The aim of this paper is to present a methodology to generate a partition of an image and a hierarchical region merging scheme to improve the meaningfulness of the segmentation, by reducing excessive object fragmentation. The segmentation method is based on the watershed transform applied to the image gradient magnitude. Prior to the actual segmentation, the image is smoothed to decrease the amount of detail detected by the watershed transform. To further improve the segmentation result, we use an iterative region merging process that uses a graph to represent the image partitions. In this process the most similar pair of adjacent regions is sequentially merged according to a predefined similarity metric. We investigate the use of a combined region merging criterion that takes into account both the intensity similarity and the contrast at the boundary of two adjacent regions. Results obtained illustrate the good combined performance of this segmentation and merging methods and the usefulness of the combined similarity function.

Multi-hypothesis motion compensated prediction (MHMCP) predicts a block from a weighted sum of multiple reference blocks in the frame buffer. By efficiently combining these reference blocks, MHMCP can provide less prediction errors so as to reduce the coding bit rates. Although MHMCP was originally proposed to achieve high coding efficiency, it has been observed recently that MHMCP can also enhance the error resilient property of compressed video. In this work, we investigate the error propagation effect in the MHMCP coder. More specifically, we study how the multi-hypothesis number as well as hypothesis coefficients influence the strength of propagating errors. Simulation results are given to confirm our analysis. Finally, several design principles for the MHMCP coder are derived based on our analysis and simulation results.

This paper presents a joint source coding and networking scheme for video delivery over ad hoc wireless local area networks. The objective is to improve the end-to-end video quality with the constraint of the physical network. The proposed video transport scheme effectively integrates several networking components including load-aware multipath routing, class based queuing (CBQ), and scalable (or layered) video source coding techniques. A typical progressive video coder, 3D-SPIHT, is used to generate multi-layer source data streams. The coded bitstreams are then segmented into multiple sub-streams, each with a different level of importance towards the final video reconstruction. The underlay wireless ad hoc network is designed to support service differentiation. A contention sensitive load aware routing (CSLAR) protocol is proposed. The approach is to discover multiple routes between the source and the destination, and label each route with a load value which indicates its quality of service (QoS) characteristics. The video sub-streams will be distributed among these paths according to their QoS priority. CBQ is also applied to all intermediate nodes, which gives preference to important sub-streams. Through this approach, the scalable source coding techniques are incorporated with differentiated service (DiffServ) networking techniques so that the overall system performance is effectively improved. Simulations have been conducted on the network simulator (ns-2). Both network layer performance and application layer performance are evaluated. Significant improvements over traditional ad hoc wireless network transport schemes have been observed.

In this paper, we develop a tree-structured predictive partial matching (PPM) scheme for progressive compression of PointTexture images. By incorporating PPM with tree-structured coding, the proposed algorithm can compress 3D depth information progressively into a single bitstream. Also, the proposed algorithm compresses color information using a differential pulse coding modulation (DPCM) coder and interweaves the compressed depth and color information effciently. Thus, the decoder can reconstruct 3D models from the coarsest resolution to the highest resolution from a single bitstream. Simulation results demonstrate that the proposed algorithm provides much better compression performance than a universal Lempel-Ziv coder, WinZip.

In this paper, we develop a tree-structured predictive partial matching (PPM) scheme for progressive compression of PointTexture images. By incorporating PPM with tree-structured coding, the proposed algorithm can compress 3D depth information progressively into a single bitstream. Also, the proposed algorithm compresses color information using a differential pulse coding modulation (DPCM) coder and interweaves the compressed depth and color information effciently. Thus, the decoder can reconstruct 3D models from the coarsest resolution to the highest resolution from a single bitstream. Simulation results demonstrate that the proposed algorithm provides much better compression performance than a universal Lempel-Ziv coder, WinZip.

Currently, the demand for larger screens and high-definition (HD) television-sets is rapidly increasing. To display standard definition (SD) video on high-resolution television sets, spatial up-scaling techniques are necessary to increase the resolution. One technique that makes this possible is linear up-scaling complemented by non-linear sharpness enhancement, such as Luminance Transient Improvement (LTI). In this paper, three different two-dimensional (2D) implementations of the LTI algorithm are analyzed in terms of image quality and computational complexity. Results show that taking the local edge orientation into account is essential for obtaining high image quality. Furthermore, it is shown that the process of operating perpendicular to an edge can be replaced by a simplified alternative with a minimal loss of image quality.

This paper addresses two issues related to motion estimation using the block matching algorithms (BMA): (1) determining the reliability of the motion vectors of each block, and (2) imposing smoothness constraint to the motion vector field. We introduce a new robust reliability measure to represent the confidence level of the motion vector from the cost function distribution and propose a novel algorithm that incorporates smoothness constraint into the motion vector field evaluation by implementing a priority queue structure based on the reliability measure. In this framework, a smooth motion vector field is evaluated in a single pass without going through iterations typical of many existing optical flow estimation algorithms. Hence it is fast and can easily be incorporated into real-time applications for video compression as well as image segmentation.

It is important to smooth the quality variability and improve the utilization of the available network bandwidth in a streaming session from a scalable streaming server to the client over the network with bandwidth variation. In this paper, we propose an on-line smoothing scheduling algorithm for real-time scalable media stream with the variable network bandwidth and packet loss. This algorithm adopts a rate-distortion optimized framework and real-time scheduling scheme to select and schedule the packets according to the network status. It attempts to minimize the quality variability in client while at the same time maximizing the utilization of the variable network bandwidth. Experiments show that, compared with frame-based scheduling algorithm, our proposed on-line smoothing algorithm not only improves the quality in decoded video frames but also make the quality variation smoother.

Many image compression standards such as JPEG, MPEG or H.263 are based on the discrete cosine transform (DCT), quantization, and Huffman coding. Quantization error is the major source of image quality degradation. The current dequantization method assumes the uniform distribution of DCT coefficients. Therefore the reconstruction value is the center of each quantization interval. However DCT coefficients are regarded to follow Laplacian probability density function (pdf). We derive an optimal reconstruction value in closed form assuming Laplacian pdf, and show the effect of the correction on image quality. We estimate the Laplacian pdf parameter for each DCT coefficient, and obtain a correction for reconstruction value from the proposed theoretical predictions. The corrected value depends on the Laplacian pdf parameter and the quantization step size Q. The effect of PSNR improvement due to the change in dequantization value is about 0.2 ~ 0.4 dB. We also analyze the reason for the limited improvements.

In this paper, we present a novel energy compaction method, the selective block reordering, which is used with SPIHT (SBR-SPIHT) coding for low rate video coding to enhance the coding efficiency for motion-compensated residuals. The inter-frame coding basically includes three major parts - motion estimation, motion compensation, and motion-compensated residual coding. The motion estimation and overlapped block motion compensation (OBMC) methods of H.263 are used to reduce the temporal redundancy. The motion-compensated residuals are encoded in the wavelet domain. The block-mapping reorganization utilizes the wavelet zerotree relationship that jointly presents the wavelet coefficients from the lowest subband to high frequency subbands at the same spatial location, and allocates each wavelet tree with all descendents to form a wavelet block. The block reordering based on the threshold scan rearranges the significant blocks in the descending order of the energy. Then, the block reordering technique reorders the wavelet sub-blocks recrusively, according to the energy of each sub-block, to yield the maximum energy compaction that allows the SPIHT coding to operate efficiently on the motion-compensated residuals. Simulation results demonstrate that SBR-SPIHT outperforms H.263 by 1.28~0.69 dB on average for various video sequences at very low bit-rates, ranging from 48 to 10 kbps.

We propose applying an adaptive context-tree weighting (CTW) method in the H.264 video coders. We first investigate two different ways to incorporating the CTW method into an H.264 coder and compare the coding effectiveness of using the method with that of using the context models specified in the H.264 standard. We then describe a novel approach for automatically adapting the CTW method based on the syntactic element to be coded and the encoding parameters. We show that our CTW-based arithmetic coding method yields similar or better compression results compared with the context-based adaptive arithmetic coding method used in H.264, without having to specify so many context models.

In the past, many schemes have been proposed for progressive image transmission using unequal error protection (UEP) or unequal loss protection (ULP). However, most existing UEP/ULP schemes do not consider the minimum image quality requirement and usually have high computation complexity. In this paper, we propose a layered ULP (L-ULP) scheme for progressive image transmission over packet loss channels, which is able to solve the mentioned problems of existing ULP schemes by smartly choosing the layers. The numerical results show that the proposed L-ULP scheme is quite promising for fast image transmission over packet loss networks.

This paper proposes a transcoding technique with low computational complexity, by means of eliminating high order DCT coefficients. Although the computational complexity of the DCT coefficient reduction technique is very low, this technique induces severe drift errors and produces poor-quality blockish pictures; artifacts are especially noticeable in interlaced video. Therefore, we propose a fast transcoding method of interlaced video which can reduce these artifacts. Our technique detects problem blocks, and allocate them with higher order DCT coefficients into the new MPEG stream. As a result of using our proposed technique, experiments show that artifacts caused by drift errors are mostly reduced, and the computational complexity is very low. Only 4.1% of all the luminance blocks are detected as problem blocks. Hence, most blocks are filled with the minimum set of DCT information required in the transcoded stream. Moreover, we develop a networked video browsing system by using our proposed transcoding method to play multiple streams in a client viewer, simultaneously.

In this paper, we propose a new method for an objective measurement of video quality based on edge degradation. One of the most important requirements for an objective method for video quality measurement is that it should provide consistent performances over a wide range of video sequences that are not used in the designing stage. By analyzing subjective scores of various video sequences, we found that the human visual system is sensitive to degradation around edges. In other words, when edge areas of a video are blurred, evaluators tend to give low scores to the video even though the overall mean squared error is not so large. Based on this observation, we propose an objective video quality measurement method that measures degradation around edges. In the proposed method, we first apply an edge detection algorithm to videos and find edge areas. Then, we measure degradation of those edge areas by computing mean squared error. From this mean squared error, we compute the PSNR and use it as video quality metric. Experimental results show that the proposed method compares favorably with the current objective methods for video quality measurement. Furthermore, when the proposed method is applied to test video sequences that are not used in the designing stage, it still consistently provides satisfactory performances.

This paper proposes a low-complexity sub-pixel refinement to motion estimation based on full-search block matching algorithm (BMA) at integer-pixel accuracy. This algorithm eliminates the need to produce interpolated reference frames, which is may be too memory- and processor- intensive, for some real-time mobile applications. The algorithm assumes the BMA is done at pixel resolution and the (sum-of-absolute-differences) SADs of the candidate motion vector and its neighbouring vectors are available for each block. The proposed method than models the SAD distribution around the candidate motion vector and its neighbouring points. Actual minimum point at sub-pixel resolution is then computed according to the model used. 3 variations of the parabolic model are considered and simulations using the H.263 standard encoder on several test sequences reveal an improvement of 1.0 dB over integer-accuracy motion estimation. Albeit its simplicity, some test cases come close to the results obtained by actual interpolated reference frames.

This paper describes the complete procedure of framework design on a video coding system for error-prone heterogeneous network environment. First of all, system requirements are analyzed and the targets of system design are emphasized on three categories: 1. improve video performance under the constraints of network bandwidth and computational complexity; 2. provide error robust mechanism when packet loss occur; 3. add capability of layered video coding for heterogenous network transmission. Then, ITU-T H.263+ recommendation is introduced, especially about its 16 optional enhanced coding modes: feature, effect and benefit. System design on mode selection is somewhat a trade-off, one hand is improvement on subjective quality of video or other system targets, the other hand is influence on time delay and complexity (computational load, data dependency, ease of implementation, etc.). Five enhanced coding modes are preferred in strong reason for our purpose: Advanced INTRA Coding mode, Deblocking Filter mode, Modified Quantization mode, Slice Structured mode and Temporal, SNR and Spatial Scalability mode. At last, some useful and important system elements beyond H.263+ recommendation are discussed: RTP packetization, error concealment and error tracking. Application shows that all the targets are easily reached by the implementation based on this design.

In this paper a novel robust video transmission scheme is proposed that can not only decode the compressed video stream in two directions but also correct channel errors with limit burst length. Standard video coders usually utilize variable length coding (VLC) to obtain high compression ratio, which makes the compressed bit stream very vulnerable to channel errors. Even one bit error may lead to loss of synchronization with the following bits undecodable and result in error propagation. In order to limit error propagation due to VLC we put forward a new video coding scheme named as two-way decodable variable length data blocks (TDVLDB), which makes the compressed bit stream bi-directionally decodable through reversal and XOR operations. Great subjective and object quality improvements have been found in our simulation results compared with other resilient schemes, such as the reversible variable length codes (RVLC) scheme, the partial backward decodable bit stream (PBDBS) scheme and the early resynchronization (ER) scheme.

In this paper we present tests and results of an automatic color fading restoration process for digitized movies. The proposed color correction method is based on the ACE model, an unsupervised color equalization algorithm based on a perceptual approach and inspired by some mechanisms of the human visual system. This perceptual approach is local, robust and does not need any user region selection or any other user supervision. However the model has a small number of parameters that has to be set once before the filtering. The tests presented in this paper aim to study these parameters and find their effect on the final result.

This paper investigates the motion prediction techniques used in hybrid video coding. We first present a unified interpretation of motion prediction in terms of the prediction of motion threads. It is demonstrated that most current motion prediction techniques can be regarded as linear predictors of motion threads. Based on this new interpretation of motion prediction, we discuss the optimal motion predictor in the framework of Markov universal prediction. We define Markov predictability in a way that it upper bounds the optimal prediction performance in perfect reconstruction scenario. Since most current video applications use lossy coding, this results in imperfect reconstructions of the motion threads used in prediction. However, the optimality with the above perfect reconstruction scenario still holds in this case in an almost sure sense.

FGS (Fine Granularity Scalability) is a scalable coding technique which can provide flexibility and good performance for Internet video streaming. However, FGS is not suitable for wireless video streaming. This is mainly because the low coding efficiency of FGS does not fit the limited bandwidth of wireless networks. In this paper, we jointly consider mode selection and UEP (unequal error protection) for FGS video transmission over wireless channels. In particular, we provide two modes for encoding the FGS enhancement layer of each video frame, i.e., with prediction or without prediction. The mode selection depends on the capability of UEP while the solution for UEP depends on the vulnerability of source data. We construct an overall end-to-end rate-distortion (R-D) function. Based on this end-to-end R-D function, we are able to find the optimal solutions for both mode selection and UEP so that an optimal tradeoff between efficiency and robustness can be achieved. Experimental results demonstrate the proposed system is able to significantly improve the end-to-end video quality for wireless FGS video coding and transmission.

Multiple camera based surveillance systems provide us with a more robust tracking of objects. To take advantage of additional cameras, it is necessary to establish geometrical relationship between the cameras and relationship between an object and a camera. In recent years several techniques have been proposed, which estimate only the relation of a dominant ground plane between different views instead of fully geometrical relation of camera-object-camera. They are however neither fully automatic nor suitable for a non-planar ground. We propose a fully automatic calibration algorithm which can cope with complex environment, including non-planar ground. The proposed algorithm automatically tracks and matches objects between different views, determines the overlapped region, and aligns each piece-wisely segmented plane between two views. The proposed calibration algorithm minimizes the effect of occlusion and improves the accuracy of 3D measurements by using multiple views. The algorithm can also provide us large field of view by concatenating a series of cameras.

In this study, we propose a new multi-frame method to compute a smooth and boundary preserving optical flow estimate. The approach is based on the bilateral filtering, a fast edge preserving smoothing tool. Inspired by the variational origin of the bilateral filter, we construct an energy functional that takes into account the image brightness conservation constraint and the bilateral smoothness of the optical flow. Minimization of this energy functional is performed using the Gauss Seidel iterations. Experimental results on synthetic and real image sequences demonstrate that the proposed approach yields an optical flow that is smooth and yet motion boundary preserving.

JPEG2000 is the latest still image coding standard. It was designed to overcome the limitations of the original JPEG standard and provide high quality images at low bit rates. The JPEG2000 algorithm is fundamentally based on the Discrete Wavelet Transform (DWT) and Embedded Block Coding with Optimal Truncation (EBCOT). Both of the algorithms are computationally intenstive and require significant memory bandwidth. In this paper we propose a JPEG2000 hardware/software co-processing architecture that complements existing JPEG2000 software packages with efficient dedicated hardware units for EBCOT tier-1 coding, enabling significant speedup.

The possibility of perceptual compression using live eye-tracking has been anticipated for some time by many researchers. Among the challenges of real-time eye-gaze based perceptual video compression is how to handle the fast nature of eye movements with a relative complexity of video transcoding and also take into the account a delay associated with transmission in the network. Such delay requires an additional consideration in perceptual encoding because it increases the size of the area that requires high quality coding. In this paper we present a hybrid scheme, one of the first to our knowledge, which combines eye-tracking with fast in-line scene analysis to drastically narrow down the high acuity area without the loss of eye-gaze containment.

In this paper, a novel pre-compression rate-distortion optimization algorithm is proposed, which can reduce computation power and memory requirement of JPEG 2000 encoder. It can reduce the wasted computational power of the entropy coder (EBCOT Tier-1) and unnecessary memory requirement for the code-stream. Distortion and rate of coding passes are calculated and estimated before coding, and therefore truncation point is selected before coding. Experimental results show that the computation time of EBCOT Tier-1 and memory requirement for the code-stream can be greatly reduced,especially at high compression ratio. The quality of the proposed algorithm is slightly lower than that of post-compression rate-distortion optimization algorithm.

A motion-compensated wavelet video coder is presented that uses adaptive mode selection (AMS) for each macroblock (MB). The block-based motion estimation is performed in the spatial domain, and an embedded zerotree wavelet coder (EZW) is employed to encode the residue frame. In contrast to other motion-compensated wavelet video coders, where all the MBs are forced to be in INTER mode, we construct the residue frame by combining the prediction residual of the INTER MBs with the coding residual of the INTRA and INTER_ENCODE MBs. Different from INTER MBs that are not coded, the INTRA and INTER_ENCODE MBs are encoded separately by a DCT coder. By adaptively selecting the quantizers of the INTRA and INTER_ENCODE coded MBs, our goal is to equalize the characteristics of the residue frame in order to improve the overall coding efficiency of the wavelet coder. The mode selection is based on the variance of the MB, the variance of the prediction error, and the variance of the neighboring MBs' residual. Simulations show that the proposed motion-compensated wavelet video coder achieves a gain of around 0.7-0.8dB PSNR over MPEG-2 TM5, and a comparable PSNR to other 2D motion-compensated wavelet-based video codecs. It also provides potential visual quality improvement.

H.264/AVC is a video codec developed by the Joint Video Team (JVT); a cooperation between the ITU-T VCEG (Video Coding Experts Group) and ISO/IEC MPEG (Moving Picture Experts Group). This new video coding standard has some new features that allow to get significant improvements in coding efficiency. This improved coding efficiency leads to an overall more complex algorithm which has high demands regarding memory usage and processing power. Complexity, however, is an abstract concept and cannot be measured in a simple manner. In this paper we present a method to obtain an accurate and more in-depth view on the internals of the JVT/AVC decoder. By decoding several bit streams having different encoding parameters, various program characteristics were measured. On these measurements, principal components analysis was performed to get a different view on these measurements. Our results show that the various encoding parameters have a clear impact on the low level behavior of the decoder. Moreover, our methodology allows us to give an explanation for the observed dissimilarities.

Region of interest coding is an important feature in JPEG2000 and it is accomplished by pre-emphasizing the wavelet coefficients participating in the reconstruction of the ROI. In the general scaling-based method, a number of extra bits appear in the right side of the least significant bit of the ROI samples after the shifting process. These bits need be coded at the time of bit-plane coding, but they are discarded by the decoder. The usual procedure is for the encoder to set these "don't care" bits to zero. In this paper, we propose a method that exploits the state of the JPEG2000 entropy coder to set the values of these bits in a more intelligent way. The method can be implemented with only a small increase in coder complexity and improvements of up to 6% in the number of bits required to represent a code-block has been observed. The bit-stream remains JPEG2000 compliant.