KEYWORDS: Cameras, Imaging systems, Sensors, Unmanned aerial vehicles, Control systems, System integration, Data processing, Visualization, Data acquisition, UAV imaging systems
Multispectral imaging technology analyzes for each pixel a wide spectrum of light and provides more spectral
information compared to traditional RGB images. Most current Unmanned Aerial Vehicles (UAV) camera systems are
limited by the number of spectral bands (≤10 bands) and are usually not fully integrated with the ground controller to
provide a live view of the spectral data.
We have developed a compact multispectral camera system which has two CMV2K 4x4 snapshot mosaic sensors
internally, providing 31 bands in total covering the visible and near-infrared spectral range (460-860nm). It is compatible
with (but not limited to) the DJI M600 and can be easily mounted to the drone. Our system is fully integrated with the
drone, providing stable and consistent communication between the flight controller, the drone/UAV, and our camera
payload. With our camera control application on an Android tablet connected to the flight controller, users can easily
control the camera system with a live view of the data and many useful information including histogram, sensor
temperature, etc. The system acquires images at a maximum framerate of 2x20 fps and saves them on an internal storage
of 1T Byte. The GPS data from the drone is logged with our system automatically. After the flight, data can be easily
transferred to an external hard disk. Then the data can be visualized and processed using our software into single
multispectral cubes and one stitched multispectral cube with a data quality report and a stitching report.
SWIR (Short Wave Infrared) imaging can be of great use in precision agriculture, food processing and recycling industry, among other fields. However, hyperspectral SWIR cameras are costly and bulky, preventing their widespread deployment on the field. To answer the market need for compact and cost-efficient hyperspectral cameras covering the SWIR range, imec and SCD have joined efforts to develop a novel integration approach combining imec know-how in pixel level patterned thin film spectral filter technology, with SCD’s InGaAs technology. The here presented line-scan SWIR hyperspectral camera covers the 1.1-1.65 μm range with 100+ bands and a spectral resolution better than 10 nm. This imager uses a set of patterned Fabry-Pérot interferometers processed using semiconductor grade thin-film technology. The optical filters are then integrated directly on top of the sensing side of the InGaAs detector with high accuracy and with a minimum gap between filters and Focal Plane Array to limit cross-talk. The resulting line-scan camera, measuring only 70x62x60 mm and with a weight below 0.5kg, is the lightest and most compact SWIR hyperspectral camera on the market. Full sensor readout can be performed at up to 350 fps. An imecpatented SnapScan system with internal scanning was also developed, capable of acquiring data cubes of 640x512x128 pixels in a second. Maximum cube size is 1200x640x128. By selecting a subset of contiguous spectral bands and a reduced spatial resolution the sensor could be operated @ +1000 fps, for example enabling cube acquisitions of 320x512x64 in less than 300 ms.
Imec has developed a process for the monolithic integration of optical filters on top of CMOS image sensors, leading to compact, cost-efficient and faster hyperspectral cameras. Different prototype sensors are available, most notably a 600- 1000 nm line-scan imager, and two mosaic sensors: a 4x4 VIS (470-620 nm range) and a 5x5 VNIR (600-1000 nm). In response to the users’ demand for a single sensor able to cover both the VIS and NIR ranges, further developments have been made to enable more demanding applications. As a result, this paper presents the latest addition to imec’s family of monolithically-integrated hyperspectral sensors: a line scan sensor covering the range 470-900 nm. This new prototype sensor can acquire hyperspectral image cubes of 2048 pixels over 192 bands (128 bands for the 600- 900 nm range, and 64 bands for the 470-620 nm range) at 340 cubes per second for normal machine vision illumination levels.
In this paper we present a technique to accurately build a 3D hyperspectral image cube from a 2D imager
overlaid with a wedge filter with up to hundreds of spectral bands, providing time-multiplexed data through
scanning. The correctness of the spectral curve of each pixel in the physical scene, being the combination of
its spectral information captured over different time stamps, is directly related to the alignment accuracy and
scanning sensitivity. To overcome the accumulated alignment errors from scanning inaccuracies, frequency-
dependent scaling from lens, spectral band separations and the imager’s spectral filter technology limitations,
we have designed a new image alignment algorithm based on Random Sample Consensus (RANSAC) model
fitting. It estimates many mechanical and optical system model parameters with image feature matching over
the spectral bands, ensuring high immunity against the spectral reflectance variations, noise, motion-blur, blur
etc. The estimated system model parameters are used to align the images captured over different bands in the
3D hypercube, reducing the average alignment error to 0.5 pixels, much below the alignment error obtained
with state-of-the-art techniques. The image feature correspondences between the images in different bands of
the same object are consistently produced, resulting in a hardware-software co-designed hyperspectral imager
system, conciliating high quality and correct spectral curve responses with low-cost.
Colony counting is a procedure used in microbiology laboratories for food quality monitoring, environmental
management, etc. Its purpose is to detect the level of contamination due to the presence and growth of bacteria, yeasts
and molds in a given product. Current automated counters require a tedious training and setup procedure per product and
bacteria type and do not cope well with diversity. This contrasts with the setting at microbiology laboratories, where a
wide variety of food and bacteria types have to be screened on a daily basis. To overcome the limitations of current
systems, we propose the use of hyperspectral imaging technology and examine the spectral variations induced by factors
such as illumination, bacteria type, food source and age and type of the agar. To this end, we perform experiments
making use of two alternative hyperspectral processing pipelines and compare our classification results to those yielded
by color imagery. Our results show that colony counting may be automated through the automatic recovery of the
illuminant power spectrum and reflectance. This is consistent with the notion that the recovery of the illuminant should
minimize the variations in the spectra due to reflections, shadows and other photometric artifacts. We also illustrate how,
with the reflectance at hand, the colonies can be counted making use of classical segmentation and classification
algorithms.
Thanks to its intrinsic scalability features, the wavelet transform has become increasingly popular as decorrelator in image compression applications. Throuhgput, memory requirements and complexity are important parameters when developing hardware image compression modules. An implementation of the classical, global wavelet transform requires large memory sizes and implies a large latency between the availability of the input image and the production of minimal data entities for entropy coding. Image tiling methods, as proposed by JPEG2000, reduce the memory sizes and the latency, but inevitably introduce image artefacts. The Local Wavelet Transform (LWT), presented in this paper, is a low-complexity wavelet transform architecture using a block-based processing that results in the same transformed images as those obtained by the global wavelet transform. The architecture minimizes the processing latency with a limited amount of memory. Moreover, as the LWT is an instruction-based custom processor, it can be programmed for specific tasks, such as push-broom processing of infinite-length satelite images. The features of the LWT makes it appropriate for use in space image compression, where high throughput, low memory sizes, low complexity, low power and push-broom processing are important requirements.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.