This PDF file contains the editorial “Guest Editorial: Special Section on Image/Video Compression and Processing for Visual Communications” for JEI Vol. 7 Issue 03

As the applications of digital imagery expand in resolution and pixel fidelity there is a greater need for more efficient compression and extraction of images and subimages. No longer is it sufficient to compress and decompress an image for a specific target device. The ability to handle many types of image data, extract images at different resolutions and quality, lossless and lossy, zoom and pan, and extract regions-of-interest are the new measures of image compression system performance. Compression with reversible embedded wavelets (CREW) is a high-quality image compression system that is progressive from high compression to lossless, and pyramidal in resolution. CREW supports regions-of-interest, and multiple image types, such as bi-level and continuous-tone. This paper describes the CREW system and format, shows how the correct data can be quickly extracted from a CREW file to support a variety of target devices, describes the mechanisms needed for panning, zooming, and fixed-size compression, and explains the superior performance on bi-level and graphic images.

We present a new image compression technique called "DjVu" that is specifically geared towards the compression of highresolution, high-quality images of scanned documents in color. This enables fast transmission of document images over low-speed connections, while faithfully reproducing the visual aspect of the document, including color, fonts, pictures, and paper texture. The DjVu compressor separates the text and drawings, which need a high spatial resolution, from the pictures and backgrounds, which are smoother and can be coded at a lower spatial resolution. Then, several novel techniques are used to maximize the compression ratio: the bi-level foreground image is encoded with AT&T’s proposal to the new JBIG2 fax standard, and a new wavelet-based compression method is used for the backgrounds and pictures. Both techniques use a new adaptive binary arithmetic coder called the ZP-coder. A typical magazine page in color at 300 dpi (dots per inch) can be compressed down to between 40 and 60 kbytes, approximately 5–10 times better than JPEG for a similar level of subjective quality. A real-time, memory efficient version of the decoder was implemented, and is available as a plug-in for popular web browsers.

An emerging trend in the banking industry is to digitize checks for storage and transmission. An immediate requirement for efficient storage and transmission is check image compression. General purpose compression algorithms such as JPEG and wavelet-based methods produce annoying ringing or blocking artifacts at high compression ratios. In this paper, a layered approach to check image compression is proposed in which a check image is represented in several layers. The first layer describes the foreground map; the second layer specifies the gray levels of foreground pixels; the third layer is a lossy representation of the background image; and the fourth layer describes the error between the original and the reconstructed image of the first three layers. The layered coding approach produces images of better quality than traditional JPEG and wavelet coding methods, especially in the foreground, i.e., the text and graphics. In addition, this approach allows progressive retrieval or transmission of different image layers.

We develop a new image representation which combines support for coding and content-based retrieval. The algorithm minimizes a weighted sum of the expected compressed file size and query response time. Our approach leads to a progressive refinement retrieval by successively reducing the number of searched files as more bits are read. Furthermore, no distance computations are required during a query. Only simple bit pattern comparisons are required. In our experimental results based on a 1.4 Mbyte image database, the number of bits read during a typical retrieval was less than 1000 bytes (less than 0.08% of the database). The approach supports compressed data modification and low-bit-rate high quality browsing.

The exchange of visual information in heterogeneous networks requires image compression algorithms that provide high quality images at several spatial resolutions and bit rates. Scalable coding provides images at several resolutions and bit rates, while perceptual coding incorporates human visual system characteristics. This paper presents an image compression algorithm that provides both scalable and perceptual coding to produce an embedded stream suitable for transmission over packet networks. Scalable and perceptual coding are combined through intraband coding of perceptually significant image regions. Images are segmented into smooth, edge, and detailed regions, and these regions are then independently intraband coded and included in the stream in decreasing order of visual importance with respect to both region type and scale. Visual importance is based on the results of two psychophysical studies analyzing sensitivity to noise in different image regions and their roles in recognition. Locally adaptive quantization, based on study results, is also included in the compression algorithm. The resulting embedded stream provides spatial scalability and region-selective coding and performs well in the presence of packet loss.

In order to establish a synergy between the compression scheme and the intended processing of the compressed image, we develop a family of compression algorithms that can adapt to the specialized requirements imposed by the application. The focus of this work is on wavelet-based compression for still images. A family of eight compression schemes is proposed to address specialized multimedia requirements such as (i) partial evaluation of wavelet coefficients, (ii) asymptotic losslessness of the algorithms, (iii) tree-structure in the selected coefficients, and (iv) optimization of either energy compaction or coefficient retention. Experimental results, including fraction of coefficients evaluated, rate-distortion performance, and coefficient retention are presented.

Compression of three-dimensional (3D) or four-dimensional medical image data has now become imperative for clinical picture archiving and communication systems (PACS), telemedicine, and telepresence networks. While lossless compression is often desired, lossy compression techniques are gaining acceptance for medical applications, provided that clinically important information can be preserved in the coding process. We present a coherently three-dimensional method for volumetric image compression using 3D wavelet transform and 3D zerotree coding. First, the volumetric image data is decomposed using 3D separable wavelet filterbanks. In this study, we adopt a three-level decomposition to form a 22-band multiresolution pyramid of an octree. Then, to exploit the dependencies among the subband coefficients resulting from 3D wavelet decomposition, a 3D zerotree coding scheme is utilized. To take advantage of the near-Laplacian distributions of the subband coefficients and the efficiency of zerotree coding, a pseudouniform quantization is adopted. The proposed volumetric image compression scheme is applied to two sets of real CT medical data. Significant coding gains have been achieved which demonstrate the effectiveness of the proposed volumetric image compression scheme for medical as well as other applications.

Predictive and multiresolution techniques for near-lossless image compression based on the criterion of maximum allowable deviation of pixel values are investigated. A procedure for near-lossless compression using a modification of lossless predictive coding techniques to satisfy the specified tolerance is described. Simulation results with modified versions of two of the best lossless predictive coding techniques known, CALIC and JPEG-LS, are provided. Application of lossless coding based on reversible transforms in conjunction with prequantization is shown to be inferior to predictive techniques for near-lossless compression. A partial embedding two-layer scheme is proposed in which an embedded multiresolution coder generates a lossy base layer, and a simple but effective context-based lossless coder codes the difference between the original image and the lossy reconstruction. Results show that this lossy plus near-lossless technique yields compression ratios close to those obtained with predictive techniques, while providing the feature of a partially embedded bit-stream.

In image compression applications, the choice of a method is often linked to its computational complexity and storage cost. Lattice vector quantization (LVQ) is very fast and does not require the construction of the codebook like LBG-based algorithms. This particularity is very attractive for low bit rate applications. However, LVQ indexing is a key problem for transmission and storage purposes. We recently proposed a new indexing algorithm that provides a good tradeoff between computational complexity and storage requirements. This paper is concerned with the design of a specific multi-DSP architecture for this new method in a real time and low bit rate coding context.

MPEG-4 aims at providing standardized core technologies allowing efficient storage, transmission, and manipulation of video data in multimedia environments. In order to achieve this broad goal, rather than a solution for a narrow set of applications, functionalities common to clusters of applications are under consideration. The encoded representation, along with the set of tools and algorithms provided by MPEG-4 will enable these functionalities. The current focus of the MPEG-4 video group is on the development of a video verification model (VM). A VM is a precise definition of encoding and decoding algorithms which may be thought of as tools addressing specific functionalities. Typically, members of the MPEG-4 video group propose several tools addressing different functionalities. The VM serves as a common platform for testing and comparing the proposed tools. The tools are compared by means of so-called core experiments. As the result of each core experiment becomes available, the VM is updated, so that the tool determined to be the best in a given core experiment replaces the current corresponding tool present in the VM. This paper gives an overview of version 8.0 of the video VM in MPEG-4. In doing so, issues, algorithms, and major tools used in the development of this future video standard are discussed.

Two-layer video using SNR scalability can be used to offer error resilience to video applications where channel resources are unreliable. In the event of information loss or corruption, increased priority and/or protection can be given to the base layer bitstream in order to ensure minimum base layer picture quality. However, SNR scalability is disadvantaged by a significant increase in total bit-rate compared to an equivalent single-layer coder operating at the same quality. In this paper we identify the DCT bit allocation of the enhancement layer as the primary cause of inefficiency citing two specific areas where coding efficiency is compromised. From this, the unique probability characteristics exhibited by the enhancement layer are used to develop an alternative approach to enhancement layer DCT coding. Exploiting the conditional and adaptive coding features facilitated by arithmetic coding, four enhancement layer coding schemes are proposed and compared to the standard coder. The results show that a hybridized arithmetic coder combining run-length and base layer conditioning coding techniques emerges as the best approach, delivering up to 20–25% better compression in the enhancement layer and outperforming even the single-layer equivalent through much of its operating range.

To find good trade-offs between the rate-distortion efficiency of motion-compensated, discrete-cosine-transform video coding and its computational complexity, we explore a scheme which uses an adaptive displacement vector field for motion compensation. The sampling density of the field is nonuniform and is adapted to the image scene using a Lagrangian optimization algorithm. The algorithm is formulated to minimize an overall distortion for a given bit rate constraint on the video frame. The algorithm takes into account the contributions to the overall bit rate and distortion due to motion vector and prediction residual coding. Our simulation results demonstrate that, for various common video sequences, the proposed scheme outperforms a popular H.263 test model by about 0.5 dB of peak signal-to-noise ratio. Significant reduction in coding artifacts is observed, and visual quality improvement is highly palpable. The work demonstrates that a suitably crafted rate-distortion optimization scheme can improve performance without exacerbating complexity.

In this paper, we present an efficient video coding technique based on a variable block-sizing (VBS) technique and the wavelet transform. Quadtrees are used for VBS segmentation on the wavelet-transformed frame which is decomposed into pyramid multiresolution subbands. Feature areas are extracted during the growth of the quadtree on each subband. Motion estimation/compensation and vector quantization are performed on only the extracted feature areas in the subband. Since the VBS technique is very effective in lowering overall coding rates, the proposed coder is designed for very low bit-rate video coding. Simulation results are presented at 14.4 kbits/s for videoconferencing sequences. To evaluate the performance of the proposed coder, comparisons are made with results obtained by the H.263 standard.

The channel sharing efficiency in cell-loss conditions is important during the transmission of video signals over asynchronous transfer mode networks. Several possible approaches using two-layer video coding methods have been proposed, in which video information is transmitted over the network with different levels of protection with respect to cell loss. This paper proposes a two-layer coding method using a pyramid structure, in which several realizations are presented and their performance comparison is shown. To minimize the image quality degradation by cell loss, a cell-loss concealment method using a pyramid structure is proposed and its performance is compared with that of conventional methods.

In traditional video coding, motion estimation is used to generate motion compensated difference frames and a texture coding method is then used to code the motion compensated difference frames. A general problem in using a training-based vector quantization technique for coding motion compensated difference frames is the training-coding loop. Vector wavelet coding (VWC) has been shown to provide good performance for image coding. In order to implement VWC for video coding, the training-coding loop has to be examined. This paper reports the results of our study on a method of dealing with this training-coding loop.

A dynamic search window adjustment for block-matching algorithm (BMA) based on the block similarity is presented to reduce the computational complexity of full search BMA. The adjustment of the size of the search window is performed in three steps: (1) set a new search origin based on the block similarity and the displaced block difference (DBD), (2) adjust the size of search window in inverse proportion to block similarity, and (3) update the thresholds for accommodation to a given image sequence. The technique can be easily applied to full search BMA and several fast search algorithms to get more efficiency and to reduce a possibility of falling into a local minimum. Experimental results show that the proposed technique has a good MSE performance and reduces the number of search points substantially.

In this paper we propose a new method for invisibly watermarking high-quality color and gray-scale images. This method is intended for use in image verification applications to determine whether the content of an image has been altered, due perhaps, to the act of a malicious party. It consists of both a watermarking process which stamps a watermark into a source image without visual degradation, and a watermark extraction process which extracts a watermark from a stamped image. The extracted watermark can be compared with the embedded watermark to determine whether the image has been altered. The processing used in the watermarking and watermark extraction processes will be presented in this paper. In addition, we shall describe some modifications that provide better security, and an adaptation of the scheme to watermark JPEG images. Experimental results are reported. Some advantages of this technique over other invisible watermarking techniques for verification will be discussed; these include a high degree of invisibility, color preservation, ease of extraction, and a high degree of protection against the retention of a correct watermark after alteration by a malicious party.

This paper proposes a content-based temporal video segmentation system that integrates syntactic (domain-independent) and semantic (domain-dependent) features for automatic management of video data. Temporal video segmentation includes scene change detection and shot classification. The proposed scene change detection method consists of two steps: detection and tracking of semantic objects of interest specified by the user, and an unsupervised method for detection of cuts, and edit effects. Object detection and tracking is achieved using a region matching scheme, where the region of interest is defined by the boundary of the object. A new unsupervised scene change detection method based on two-class clustering is introduced to eliminate the data dependency of threshold selection. The proposed shot classification approach relies on semantic image features and exploits domain-dependent visual properties such as shape, color, and spatial configuration of tracked semantic objects. The system has been applied to segmentation and classification of TV programs collected from different channels. Although the paper focuses on news programs, the method can easily be applied to other TV programs with distinct semantic structure.

We propose a novel method for designing linear, space-invariant pre/post filters for DCT-based image coders that minimize the mean square error. We use a linear gain-plus-additive noise model to describe the effects of a class of entropy-coded uniform threshold quantizers. Using this quantizer model and a homogeneous random field model for the input source, we derive an explicit expression for the power spectral density of the reconstruction error in an improved transform coding system that includes pre/post filters. Linear pre/post filters are subsequently designed that minimize the overall mean square error for a given input source and target rate. The postfilter coefficients are designed at the transmitter, and are sent to the receiver as part of the coded data. Such pre/post filters improve the frequency domain behavior of the system, when coding random fields. However, when coding natural images, one must account for spatial errors as well, such as blocking artifacts. We therefore develop an a posteriori method of designing linear postfilters that uses the actual image data and maximizes the PSNR performance of the system. We present simulation results that show that pre/post filters can help DCT-based coders to match the performance of subband/wavelet coders. We design short, adaptive, linear postfilters for the Baseline JPEG coder that surpass the PSNRperformance of other post-processing methods described in the literature.

The most straightforward approach in obtaining a down-converted image sequence is to decimate each frame after it has been fully decoded. To reduce memory requirements and other costs incurred by this approach, a down-conversion decoder would perform a decimation within the decoding loop. In this way, predictions are made from a low-resolution reference which has experienced considerable loss of information. Additionally, the predictions must be made from a set of motion vectors which correspond to the full-resolution image sequence. Given these conditions, it is desirable to optimize the performance of the motion compensation process. In this paper we show that the optimal set of filters for performing the low-resolution motion compensation is dependent on the choice of down-conversion filter. The motion compensation filters are determined as the optimal solution of a least squares problem. This problem is formulated in the context of two general classes of down-conversion techniques: one which is dependent on a single block, and another which is dependent on multiple blocks. General solutions for each class of down-conversion are provided. To demonstrate the usefulness of these results, a sample set of motion compensation filters for each class of down-conversion is calculated, and the results are incorporated into a low-resolution decoder. In comparison to a sub-optimal motion compensation scheme, the optimal motion compensation filters realize a drastic reduction in the amount of drift. Simulation results also reveal that the filters which were based on multiple block down-conversion can reduce the amount of prediction drift found in the single block down-conversion by as much as 35%.

In this paper, we wish to present a process enabling us to mark digital pictures with invisible and undetectable secret information. This so-called watermarking process is intended to be the basis of a complete copyright protection system. It consists of constructing a band-limited image from binary sequences with good correlation properties and in modulating some randomly selected carriers. The security relies on the secrecy of these carrier frequencies, which are deduced from a unique secret key. Then the amplitude of the modulated images is modified according to a masking criterion based on a model of the Human Visual System. The adding of the modulated images to the original is supposed to be invisible. The resulting image fully identifies the copyright owner since he is the only one able to detect and prove the presence of the embedded watermark thanks to his secret key. This paper also contains an analysis of the robustness of the watermark against compression and image processing.

In this paper, contrast enhancement and image segmentation are investigated using a class of morphological nonincreasing filters that can be considered toggle mappings. These nonincreasing filters are built using the traditional morphological gradients. These filters have interesting properties and give essential contrast to the images. We apply them sequentially between two or more given parameters in order to obtain intermediate results. This approach improves the control of the filtering process and provides other tools for contrasting images. We presented several new properties and one study of the invariant set of these filters. Using these new propositions, we show that it is possible to obtain better results in image segmentation when we apply the watershed transformation. Also, we propose an algorithm for segmenting images using this class of nonincreasing filters. The method is applied in a geodesic way using two different criteria for segmenting an image. We relate our results with a recent method in mathematical morphology called the flat zone approach and we compare our approach with another method in image processing, the so-called quadtree approach.

This paper describes an improved error diffusion algorithm for the purpose of digitally halftoning images. In one variation of the algorithm an error signal is calculated by the difference between a visually perceived input value and a visually perceived output value. This is accomplished by applying a causal visual blur function to both the input and output images. This approach has the advantage that it minimizes the appearance of worm artifacts in the output image, while simultaneously eliminating the edge artifacts associated with a previous visual error diffusion algorithm. In a second variation of the improved error diffusion algorithm, a local image activity detector is used to adaptively modify the input and output blur filters. This allows the error diffusion parameters to be optimized for different types of image content.

A method is proposed for solving the mapping problem from the three-dimensional color space to the four-dimensional CMYK space of printer ink signals by means of a neural network. The CIE-L*a*b* color system is used as the device-independent color space. The color reproduction problem is considered as the problem of controlling an unknown static system with four inputs and three outputs. A controller determines the CMYK signals necessary to produce the desired L*a*b* values with a given printer. Our solution method for this control problem is based on a two-phase procedure which eliminates the need for UCR and GCR. The first phase determines a neural network as a model of the given printer, and the second phase determines the combined neural network system by combining the printer model and the controller in such a way that it represents an identity mapping in the L*a*b* color space. Then the network of the controller part realizes the mapping from the L*a*b* space to the CMYK space. Practical algorithms are presented in the form of multilayer feedforward networks. The feasibility of the proposed method is shown in experiments using a dye sublimation printer and an ink jet printer.

The observation of a whole space on a 360° angle, as well as the reconstruction of a 3D scene, presumed unknown, presents to us an unquestionable interest in the field of robotics vision. The object of this paper is to describe a panoramic stereovision sensor dedicated to distance estimation and 3D scene reconstruction whose concept simplifies calibration and feature matching. The advantages of the device are numerous and allow us to consider omnidirectional robotics vision under new realistic and robust conditions.

The stereoscopic videoprojector developed at our laboratory is based on the simultaneous projection of the left and right images of a stereo-pair along two orthogonal polarization directions of a laser beam. The horizontal scanning system, which deflects four lines simultaneously, led to the design of a specific acousto-electro-optical modulation device using three independent command variables to control four pixel intensities. This introduces a degradation of the perceived images that we strive to minimize using the information redundancy in the images. In this paper, we first present the principle of operation and the optical modulation device. Then, an optimal solution for the computation of the command variables from the original stereo-pair pixel intensities is proposed and developed. The chosen mathematical criterion is the minimization of the quadratic error between the perceived and original pixel intensities. The study of the visual degradation introduced in the images by the proposed stereo-pair processing leads to the distinction of two kinds of phenomena: the mixing of the pixel intensities in the same image and the mixing of the pixel intensities between the left and right views. The simulations realized with reference images allow a quantitative evaluation of their relative proportions and of the final image quality.

We present algorithms for automatic image annotation and retrieval based on color, shape, texture, and any combination of two or more of these features. Pixel- or region (object)-based color; region-based shape; and block- or region-based texture features have been considered. Automatic region selection has been accomplished by integrating color and spatial edge features. Color, shape, and texture indexing may be knowledge based (using appropriate training sets) or by example. The multifeature integration algorithms are designed to: (i) offer the user a wide range of options and flexibilities in order to enhance the outcome of the search and retrieval operations, and (ii) provide a compromise between accuracy and computational complexity, and vice versa. We demonstrate the performance of the proposed algorithms on a variety of images.

A striping scheme is proposed in which a video program is partitioned into small segments that are stored by distributing them across multiple disks in a video server. In a striped-disk system, the read workload will always be evenly distributed among the component disks as long as the terminals accessing the server perform only normal playback of the video materials. However, if normal playback is combined with visual search capabilities such as jumps and skip searches, then the workload on disks could become momentarily unbalanced or skewed, which could cause interruptions in playback at terminals. In this work, we developed a queueing network model for representing multiple accesses to video materials striped across multiple disks of a video server. An approximate analysis was performed on the model to clarify the traffic characteristics of disks when visual searches are executed. One useful finding for the design of video servers is that, spreading video data over a larger number of disks through striping alleviates momentary workload skewing, which enables a server to support more multiple accesses per disk.

This PDF file contains the editorial “Book Rvw: Video Coding: The Second Generation Approach," edited by Luis Torres and Murat Kunt for JEI Vol. 7 Issue 03