The benefits of an object-oriented approach and pattern based design for complex software systems are well known. But most software engineers believe that these techniques have a run time performance cost that is too high for real-time imaging applications. Unfortunately, no usable data to support or disprove this contention exists. In this work we describe a set of experiments that challenge the conventional wisdom. We report on our results and we further introduce a strategy to thoroughly investigate the issues of run-time performance in any real-time imaging system.

The 3D-Pathology project has created a novel microscope system that realises, for the pathologist at their work bench, real-time analysis of three-dimensional, digital images from transmitted light microscopy (TLM). The system incorporates: 1. A software application that facilitates the role of the pathologist to: a.survey a microscopic specimen, at low resolution; b.identify and digitally capture one or more regions of interest (ROI) within the specimen, at high resolution; c.perform an image reconstruction on the ROI to enhance the contrast and detail; · analyse and manipulate the reconstructed ROI, for diagnostic purposes; · all from the comfort of their desktop personal computer. 2. A hardware prototype that includes: a.a stage platform, to position the specimen; b.a microscope and digital camera assembly, to acquire image data; c.a custom-built interface, to allow the hardware to be controlled from a personal computer. Key to the success of the system is the availability of an appropriate model of the image distortion that occurs within the optical system. Such a model has been devised, based on the assumption that distortion can be described as the convolution of the ideal image and a point spread function which expresses the effects of absorption within the specimen. Current computational techniques for deconvolution are applied to recover a high quality digital representation of the original specimen.

Usually, 3D images are contaminated by a noise that can be modeled as an impulsive, Gaussian, or may be speckle one. In this paper we present the implementation of the three dimensional (3D) filters applied to ultrasound (US) imaging. Such a filtering technique uses different modified order statistics algorithms suppressing noise and preserving the small-size image details. The performance of the presented filters has demonstrated for 3D objects by means of use the objective criteria PSNR and MAE, and subjective one analyzing an error image. The PSNR and MAE criteria for Gaussian noise contamination have showed that Alfa Trimmed Mean and ROM filters presented the better results, and the MM-KNN filter obtained good results for small values of contamination. MMKNN filter was the most efficient in impulsive noise suppression for higher level of contamination, also reducing the speckle noise that is natural for coherent US transducer. The DSP TMS320C6711 was employed for implementation in 3D imaging. The processing times for Alfa Trimmed Mean filter present sufficiently high values. Filters: selection of median and of mean show significantly small times, but for LUM type filters the time values are increasing during the ordering of the voxels. The time values for RM-KNN filters are larger in comparison with other filters, but their performance is sufficiently better, and by selecting an adequate configuration of the voxels the time values can be reduced significantly without losing of good filter quality.

Real-time imaging applications are concerned with efficient and deterministic processing of digital images. These applications are predominantly written using structured programming rather than object-oriented programming with the belief that the former approach has better performance characteristics. Current research shows that this may not be the case and an object-oriented approach may not only result in efficient code but one that is more maintainable and understandable. We look at current techniques for visualizing the code for software applications to determine if they can help predict its qualities such as maintainability, understandability and performance, and suggest ways in which these techniques can be enhanced to meet the specific needs of real-time imaging applications.

In this contribution, we propose a computationally fast algorithm to compute local feature histograms of an image. existing histogram extraction is done by evaluating the distribution of image features such as color, edge, etc. within a local image windows centered each pixels. This approach is computationally very demanding since it requires evaluation of the feature distributions for every possible local window in the image. We develop an accumulated histogram propagation method that takes advantage of the fact that the local windows are overlaps and their feature histograms are highly correlated. Instead of evaluating the distributions independently, we propagate the distribution information in a 2D sweeping fashion. Our simulations prove that the proposed algorithm significantly accelarates histogram extraction and enables computation of e.g. posterier propabilities and likelihood values, which are frequently used for object detection, and tracking, as well as in other vision applications such as calibration and recognition.

This paper describes an algorithm to regulate and control the acquisition of images of a computer vision system used to measure contact wire wear in railways. Real-time implementation of the algorithm is also shown. In this work it will be shown that is possible to base a control strategy on a simple measure derived from the image histogram. The system developed controls acquisition process varying the exposure time of CCD without the necessity to control other parameter of camera/digitiser combination as black and white reference levels of digitiser. In the control strategy proposed, histogram is used as exposure time indicator. A new architecture has been defined to implement the control system in real time. Implementation system has been divided in two processing .On the one hand, a hardware processing in which histogram values extraction is carried out by FPGAs (Field Programmable Gate Array).On the other hand, the calculation of new exposure time based on the histogram is carried out by a PC whose operating system is RTLinux. The control system has been incorporated as a new module in the system used by RENFE (Spanish Railways Company) to measure the contact wire wear and measurement system precision has been increased considerably.

In image and video analysis, distance maps are frequently used. They provide the (Euclidean) distance (ED) of background pixels to the nearest object pixel. In a naive implementation, each object pixel feeds its (exact) ED to each background pixel; then the minimum of these values denotes the ED to the closest object. Recently, the Fast Exact Euclidean Distance (FEED) transformation was launched, which was up to 2x faster than the fastest algorithms available. In this paper, first additional improvements to the original FEED algorithm are discussed. Next, a timed version of FEED (tFEED) is presented, which generates distance maps for video sequences by merging partial maps. For each object in a video, a partial map can be calculated for different frames, where the partial map for fixed objects is only calculated once. In a newly developed, dynamic test-environment for robot navigation purposes, tFEED proved to be up to 7x faster than using FEED on each frame separately. It is up to 4x faster than the fastest ED algorithm available for video sequences and even 40% faster than generating city-block or chamfer distance maps for frames. Hence, tFEED is the first real time algorithm for generating exact ED maps of video sequences.

Forms of surveillance are very quickly becoming an integral part of crime control policy, crisis management, social control theory and community consciousness. In turn, it has been used as a simple and effective solution to many of these problems. However, privacy-related concerns have been expressed over the development and deployment of this technology. Used properly, video cameras help expose wrongdoing but typically come at the cost of privacy to those not involved in any maleficent activity. This work describes the design and implementation of a real-time, privacy-protecting data collection infrastructure that fuses additional sensor information (e.g. Radio-frequency) with video streams and an access control framework in order to make decisions about how and when to display the individuals under surveillance. This video surveillance system is a particular instance of our data collection framework, and here we describe in detail the real-time video processing techniques used in order to achieve tracking of users in pervasive spaces while utilizing the additional sensor data provided by the various instrumented sensors. In particular, we discuss background modeling techniques, object tracking and implementation techniques that pertain to the overall development of this system.

Nowadays, there are many families that live separately. Especially, communication with elderly person living alone is very important. We propose a new tele-surveillance system to support of their communications and to give an alarm for the accident on the elderly person living alone. The systems are set on two sites; the room for the elderly person living alone and the living room of his/her family's house. This system tracks the persons in real-time and analyzes the person's condition. Then, the system transmits the information of their head position and their condition to another site. The computer of recipient site generates the computer graphics (CG) animation of the tracked person (avatar) and display on a monitor. This method reduces traffic on network and keeps the privacy for the tracked person. The tracking part of this system uses the omni-directional image sensor that can capture a surrounding image in whole direction at a time by using a hyperbolic mirror and a video camera. The system detects the person's head in images captured by omni-directional image sensors. Then, the position of the person’s head in a room is computed. We made a prototype system of this proposed system with graphical user interface using touch panels. Experiments and evaluations showed good feasibility of the proposed system.

We present an integrated real-time smart network camera. This system is composed of an image sensor, an embedded PC based electronic card for image processing and some network capabilities. The application detects events of interest in visual scenes, highlights alarms and computes statistics. The system also produces meta-data information that could be shared between other cameras in a network. We describe the requirements of such a system and then show how the design of the system is optimized to process and compress video in real-time. Indeed, typical video-surveillance algorithms as background differencing, tracking and event detection should be highly optimized and simplified to be used in this hardware. To have a good adequation between hardware and software in this light embedded system, the software management is written on top of the java based middle-ware specification established by the OSGi alliance. We can integrate easily software and hardware in complex environments thanks to the Java Real-Time specification for the virtual machine and some network and service oriented java specifications (like RMI and Jini). Finally, we will report some outcomes and typical case studies of such a camera like counter-flow detection.

Human face detection is the first step for a fully automated face recognition system. It is also crucial to video surveillance systems, human computer interface, image/video retrieval systems. We consider color a very useful cue for face detection in color images. So that we propose a fast skin color detector for detecting in skin color patches in images with complex illumination and background. The accuracy and performance of this detector will have great affect on the upcoming feature extraction and verification processing. In our architecture, after applying a novel adaptive lighting compensation to alleviate the correct the illumination, a skin color filter based on normalized RGB color space is used to detect skin tone patches. Then, in order to remove noises and increase accuracy, morphological operations are used to refine the mask generated by the filter. Finally, the refined mask is used to gain the final result. For each step, including lighting compensation, color space modeling and final results, when compared to existing skin-tone color filtering algorithms, our algorithm is proven to be more robust and efficient by algorithm analysis and experiments results. After the processing, data amount is dramatically reduced and later algorithms can start with the skin patches remained.

This paper describes the hardware and software implementation of image segmentation chain on ARM based embedded processor. Image segmentation chain is composed by edge detection operator, thinning and crest restoration operators. Thinnning and restoration steps provide closing and thin contours. The crest restoration processing can be followed by region labeling to get a unique label for each closed region. The seamless ALTERA flow allows designing and implementing efficiently the segmentation architecture into the XA10 Excalibur board. The Altera flow provides design entries for hardware/software, simulation and evaluation steps. The architecture of the edge detection, thinning and crest restoration operators are hardware and software optimized for real-time image processing.

In this paper, we propose 32 parallel image compression circuits for high-speed cameras. The proposed compression circuits are based on a 4 x 4-point 2-dimensional DCT using a DA method, zigzag scanning of 4 blocks of the 2-D DCT coefficients and a 1-dimensional Huffman coding. The compression engine is designed with FPGAs, and the hardware complexity is compared with JPEG algorithm. It is found that the proposed compression circuits require much less hardware, leading to a compact high-speed implementation of the image compression circuits using parallel processing architecture. The PSNR of the reconstructed image using the proposed encoding method is better than that of JPEG at the region of low compression ratio.

Three-dimensional ultrasonic imaging, especially the emerging real-time version of it, is particularly valuable in medical applications such as echocardiography and surgical navigation. A known problem with ultrasound images is their high level of speckle noise. Anisotropic diffusion filtering has been shown to be effective in enhancing the visual quality of 3D ultrasound images and as preprocessing prior to advanced image processing. However, due to its arithmetic complexity and the sheer size of 3D ultrasound images, it is not possible to perform online, real-time anisotropic diffusion filtering using standard software implementations. We present an FPGA-based architecture that allows performing anisotropic diffusion filtering of 3D images at acquisition rates, thus enabling the use of this filtering technique in real-time applications, such as visualization, registration and volume rendering.

A data-driven algorithmic structure on a standard PC was developed for a block-based motion compensated temporal filtering in real time. The major time limiting factor of the algorithm was identified as the irregular memory access mainly caused by the layered multi-resolution representation of the input frames. As a result, data is transferred from main memory to cache multiple times leading to memory-dominated critical paths in execution. In order to improve the cache utilization, the computations have been rearranged to process the complete signal on the cached subset of data. The input frames are now divided into super-lines, which are subsets of data containing the relevant information to calculate one line of motion vectors and to filter the corresponding image lines. Only when a set of data is no longer used nor for motion vector analysis nor for filtering the images themselves it is replaced by data of different layers or lines. Due to these data-driven techniques the cache capacity miss rate is reduced to less than 0.8%. As a result, images are processed at a rate of more than 44 fps on a standard PC (Intel dual-processor Xeon, 1.8 GHz), as opposed to 1 fps in the standard implementation.

This paper presents a flexible hardware architecture for performing the Discrete Wavelet Transform (DWT) on a digital image. The proposed architecture uses a variation of the lifting scheme technique and provides advantages that include small memory requirements, fixed-point arithmetic implementation, and a small number of arithmetic computations. The DWT core may be used for image processing operations, such as denoising and image compression. For example, the JPEG2000 still image compression standard uses the Cohen-Daubechies-Favreau (CDF) 5/3 and CDF 9/7 DWT for lossless and lossy image compression respectively. Simple wavelet image denoising techniques resulted in improved images up to 27 dB PSNR. The DWT core is modeled using MATLAB and VHDL. The VHDL model is synthesized to a Xilinx FPGA to demonstrate hardware functionality. The CDF 5/3 and CDF 9/7 versions of the DWT are both modeled and used as comparisons. The execution time for performing both DWTs is nearly identical at approximately 14 clock cycles per image pixel for one level of DWT decomposition. The hardware area generated for the CDF 5/3 is around 15,000 gates using only 5% of the Xilinx FPGA hardware area, at 2.185 MHz max clock speed and 24 mW power consumption.

The imaging radar uses the high frequency electromagnetic waves reflected from different objects for estimating of its parameters. Pulse compression is a standard signal processing technique used to minimize the peak transmission power and to maximize SNR, and to get a better resolution. Usually the pulse compression can be achieved using a matched filter. The level of the side-lobes in the imaging radar can be reduced using the special weighting function processing. There are very known different weighting functions: Hamming, Hanning, Blackman, Chebyshev, Blackman-Harris, Kaiser-Bessel, etc., widely used in the signal processing applications. Field Programmable Gate Arrays (FPGAs) offers great benefits like instantaneous implementation, dynamic reconfiguration, design, and field programmability. This reconfiguration makes FPGAs a better solution over custom-made integrated circuits. This work aims at demonstrating a reasonably flexible implementation of FM-linear signal and pulse compression using Matlab, Simulink, and System Generator. Employing FPGA and mentioned software we have proposed the pulse compression design on FPGA using classical and novel windows technique to reduce the side-lobes level. This permits increasing the detection ability of the small or nearly placed targets in imaging radar. The advantage of FPGA that can do parallelism in real time processing permits to realize the proposed algorithms. The paper also presents the experimental results of proposed windowing procedure in the marine radar with such the parameters: signal is linear FM (Chirp); frequency deviation DF is 9.375MHz; the pulse width T is 3.2μs; taps number in the matched filter is 800 taps; sampling frequency 253.125*10⁶ MHz. It has been realized the reducing of side-lobes levels in real time permitting better resolution of the small targets.

Consumer demand for fast and accurate zoom tracking has increased in the Digital Still Camera (DSC) market. Consumers desire a DSC that automatically performs zoom tracking in order to maintain the image sharpness when the zoom lens is moved towards wide-angle or tele-angle directions. Zoom tracking involves the estimation of the in-focus motor position over all zooms based on a current in-focus position before the zoom lens is moved in either direction. Normally, a zoom tracking curve is utilized to automatically track the focus motor position when the zoom lens is moved. This paper discusses and compares the real-time implementation of two widely used zoom tracking algorithms, namely geometric zoom tracking (GZT) and adaptive zoom tracking (AZT), on the Texas Instruments (TI) digital media (DM) processor. This processor is a highly integrated, programmable dual-core processor manufactured by TI specifically for the DSC market. Our previously developed rule-based search algorithm is used to perform auto-focusing over the vicinity of the tracked focus motor position when the zoom lens is halted. This is done to regain any loss in accuracy during zoom tracking. Extensive testing was carried out to examine the performance of these algorithms in terms of tracking accuracy and speed. The results show that AZT generates a better tracking accuracy while GZT provides a faster tracking speed.

Spectral imaging becomes more and more interesting not only for agricultural use but also for industrial application. Especially wavelength in the near infrared (NIR) range can be used for materials classification. Today sorting systems for plastics are available in different variations, utilizing single-point spectroscopy and the different characteristics of plastics in the SWIR band. Sorting systems for paper and cardboard will have increased significance because better sorting can increase the price of the secondary material and reduce the need of chemicals in paper production. However, sorting paper qualities is a very difficult task due to the close similarities between the materials. The present work describes the development of an unique industrial inline material sorting system using spectral imaging technique focusing on classification for cellulose based materials such as pulp, paper and cardboard. It deals with the hardware requirements for industrial use of spectral imaging solutions as well as with adjustment and calibration techniques. Due to needed classification speed the software design and classification methods are described under this focus. To cope with the vast amount of spectral data and to implement a stable and reliable classification algorithm for different materials chemometric standard methods are used. The PCA is used to reduce data and obtain as much information of the samples's characteristics as possible by transforming the original multidimensional data-space into a space with lower dimensions. However PCA is no method to discriminate between classes, it allows to separate cellulose-based materials from plastics. For further discrimination an LDA-Algorithm is used. All chemometric methods need training data sets of well defined samples. To classify an unknown spectra, it is necessary to create models for the classes to be distinguished from each other inside the transformed data-space. Training spectra have to be carefully selected to represent the characteristics of a specific class best possible. The classification-tree uses an adapted KNN-algorithm. In order to avoid a serious bottleneck in processing-speed the continuous result space was converted into discrete space representation.

Median Filtering and Convolution operations constitute 60-70% of the preprocessing operations performed on digital images. Software implementations of 3D filters in general-purpose processors do not match the speed requirements for real-time performance. Field Programmable Gate Arrays (FPGAs) support reconfigurable architectures that are sufficiently flexible to implement more than one operation in the existing hardware, yielding higher speed for real-time execution. We present a linear systolic array architecture for median filtering, that implements bit-serial searching and majority voting. The unique arrangement of line delay units endows parallelism to the bit-serial median finding algorithm. Convolution operation, based on the fast embedded multiplier units in the FPGA and an optimized Carry Save Adder array is also presented. The application of the above designs to 3D image preprocessing is described. A voxel rate of 220MHz is achieved for median filtering and 277MHz for convolution operation.

The objective of this paper is to review the anticipated imaging and remote-sensing technology requirements for aerial vehicle survey missions to other planetary bodies in our Solar system that can support in-atmosphere flight. In the not too distant future such planetary aerial vehicle (a.k.a. aerial explorers) exploration missions will become feasible. Imaging and remote-sensing observations will be a key objective for these missions. Accordingly, it is imperative that optimal solutions in terms of imaging acquisition and real-time autonomous analysis of image data sets be developed for such vehicles.

Partners of the CANDELA project are realizing a system for real-time image processing for traffic and video-surveillance applications. This system performs some segmentation, labels the extracted blobs and follows their track into the scene. We also address the problem of evaluating the results of such processes. We are developing a tool to generate and manage the results of the performance evaluation of VCA systems. This evaluation is done by comparison of the results of the global application and its components with a ground truth file generated manually. Both manually and automatically generated description files are formatted in XML. This descriptive markup language is then treated to assemble appropriately parts of the document and process this metadata. For a scientific purpose this tool will provide an objective measure of improvement and a mean to choose between competitive methods. In addition, it is a powerful tool for algorithm designers to measure the progress of their work at the different levels of the processing chain. For an industrial purpose this tool will assess both the accuracy of the VCA with an obvious marketing impact. We present the definition of the evaluation tool, its metrics and specific implementations designed for our applications.

Attentive robots have visual systems with fovea-periphery distinction and saccadic motion capability. Previous work has shown that spatial and temporal redundancy thus present can be exploited in video coding/streaming algorithms and hence considerable bandwidth efficiency can be achieved. In this paper, we present a complete framework for real-time video coding with integrated pre-attentive processing and show that areas of greatest interest can be ensured of being processed in greater detail. The first step is pre-attention where the goal is to fixate on the most interesting parts of the incoming scene using a measure of saliency. The construction of the pre-attention function can vary depending on the set of visual primitives used. Here, we use Cartesian and Non-Cartesian filters and build a pre-attention function for a specific problem -- namely video coding in applications such as robot-human tracking or video-conferencing. Using the most salient and distinguishing filter responses as the input, system parameters of a neural network are trained using resilient back-propagation algorithm with supervised learning. These parameters are then used in the construction of the pre-attentive function. Comparative results indicate that even with a very limited amount of learning, performance robustness can be achieved.

This paper describes a system that estimates the 3D motion of the camera in a cluttered scene containing moving objects from a stereo image sequence. We use KLT[2] trackers to detect and track landmarks in both camera sequences. Range information gives an approximation of the depth for the landmarks and helps us to build a 3D system equation for the scene. By taking a novel method to detect outliers in landmark set from depth discontinuities, the filtered landmarks are then run through an iterated weighted linear square method with a retiring scheme. The estimated ego motion helps warp images of the scene, which enables us to find foreground objects from stabilized images. We describe the overall system as well as the details of the stabilization along with images that show the results of the stabilization results.

An automatic cash-binding machine register system for paper currency numbers is developed in this paper. The paper currency number is recognized and recorded by the system during binding. It can be used as an assistant method for investigation cases of bank highjack and false paper currency. The hardware of the system is composed of money-binding machine, camera, lamp-house, image gather-card and computer. To improve recognition speed, the area of number image is decreased by adjust the camera’s location on the binder. A size of 200×40 pixel is chosen in the system. The software of the system is made up of number image sample, segmentation of gray ridge-vale algorithm, orientation of projection, recognition of structure-analyzing algorithm and record. Factors as follows: the software run-speed, illumination uniformity the fray and smutch in the paper currency are considered. A random experiment with two hundred sheets of RMB 100 Yuan is given. The recognition ratio is 98%, and the average time for one sheet is about 30ms in the experiment. It can content the speed demand of binder binding money. The system has been applied national patent.

Occlusion is always a problem when counting vehicles in congested traffic. This paper tries to present an approach to solve the problem. The proposed approach consists of three main procedures. Firstly, a modified background subtraction is performed. The aim is to segment slow moving objects from an illumination-variant background. Secondly, object tracking is performed, where the CONDENSATION algorithm is used. This can avoid the matching problem. Thirdly, an inspecting procedure is executed. When a bus firstly occludes a car and then the bus moves away a few frames later, the car will appear in the scene. The inspecting procedure should find the “new” car and add it as a tracking object.

We developed blob feature analysis-based real-time marker-free motion capture system. Our system can capture actor’s end-effectors and reconstruct 3-dimensional human motions in real-time without any attaching markers or sensors. To capture robustly various motions of an actor, we proposed blob feature models such as shape model, color model, and spatial model for end-effectors such as a head, hands, and feet. And we introduce weights for each model. According to the clothing conditions of an actor, the proposed method adjusts weights for each model automatically. So, our system can detect and distinguish the actor’s end-effectors although the shapes and the color of end-effectors vary due to various poses and the variation of illumination. And our models are very simple to compute, therefore, the motion capture can be real-time process. Experiments are conducted on a lot of people wearing various clothes under general fluorescent lights. The proposed system can reconstruct actor’s various motions at 30 frames per second with the 99.95% success rate of the detection of an actor’s end-effectors. So, we confirmed that the proposed motion capture system could stably reconstruct motions of a lot of people wearing various clothes in real-time.