Guest editors Deepak Kumar Jain, Johan Debayle, Li Zhang, Vito Di Maio, and Irshad Ahmed Ansari introduce the Special Section on Image and Video Manipulation: Challenges and Solutions.

Image manipulation is transformed into a big issue for data integrity. The use of advanced imaging technology expends the regularity of multimedia forgeries. To detect such forgeries, some effective forgery identification methods are proposed to estimate the 3D lighting fingerprints by making certain suppositions related to the surface and reflection model. Incident lighting dispersal in a scene provides a physics-based key for exposing image exploitations. The proposed technique is more relaxed for multiple light source-based forgery detection. Also, the novelty of the technique is that it considers heterogeneous image surfaces, nearby surface geometry, and texture information to assess the lighting environment. The Phong reflection model is implemented to estimate the structural profiles. The method works in two phases. In the first phase, pre-processing over selected patches is performed followed by angle and error estimations in the second phase. To identify such forgeries, elevation angles ϑ concerning mounted light sources are estimated. The value of the elevation angle is computed for various patches of the image. The proposed method is relevant for a wide range of objects present in the image, i.e., not only limited to head positions, etc. To build up and validate the technique, comprehensive testing on the synthetic image dataset is done. Synthetic images are designed under different intensity light sources. Experimental results demonstrate a better forgery detection accuracy compared with other state-of-the-art methods in this domain. Also, the proposed technique is tested for generalized forged images. A comparative analysis with other related techniques is done to validate the proposed method using a synthetic image dataset. The results are pertinent to all type of images, and an error threshold is limited to 8 degrees with respect to the illuminant position.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

KS, YY, and XZ either did not respond directly or could not be reached.

Image matching is one of the hot issues in recent years. Image matching determines the effect of image mosaic to a large extent. Therefore, a stable, accurate, and fast image matching algorithm is very important for the subsequent processing of image information. Most of the traditional image matching methods have the problems of high-error matching rate and difficulty in to eliminating false matching. To resolve the shortcomings listed here, an improved algorithm was proposed in which a combined measure of Hamming distance. Similarity measurement with Tanimoto similarity measurement was adopted to dispose binary feature vectors. In addition, as an important step in the process of image mosaics, the calculation of the best mosaics can eliminate the ghosting and ghosting that may appear in the mosaics. However, the traditional dynamic planning method is easy to pass through the obstacles and form obvious stitching traces when searching for stitching lines in images with large obstacles. In view of the aforementioned content, we propose a fast and robust method to search the image difference matrix and structure difference matrix of the overlapping area of the input image according to the search strategy to determine the pixel coordinates of the splicing line. Finally, from an optimal splicing line, the experimental results show that our algorithm performs well for eliminating error matching points. Compared with traditional algorithms, matching accuracy is improved by 25%. In terms of calculating the optimal splicing line, the method proposed is faster than the traditional method and has good robustness.

Congenital heart failure (CHF) due to congestion in the blood is a serious cardiac problem correlated with crippling symptoms and leading to a rising death rate, monumental health care spending, and reduced quality of life. Heart disease prevention is among the most crucial functions of any medical system, as many people are prone to heart attacks worldwide. Although several segmentation methods for great vessels and the heart have been proposed in the research, they are not successful when applied to the health records of congenital heart disease. In this proposed work, the thickness and fat accumulation of most arteries are measured and analyzed, and then the measurement is synthesized with the corresponding width of the blood vessels of arteries; this data is used for training purposes in the convolutional neural network with one-off cross-validation and regularization. Using the CNN model, a confusion matrix is created and different statistical parameters such as accuracy sensitivity, specificity, precision, and f-score are generated. The final average accuracy was 97%, precision was 98.13%, and F-score was 98.36%. The results indicate that the CNN-based strategy can distinguish healthy hearts from those with prior cardiovascular disease.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

WL either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

KL, LX, DZ, and JX either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

XL either did not respond directly or could not be reached.

Human activity recognition is a field of video processing that requires restricted temporal analysis of video sequences for estimating the existence of different human actions. Designing an efficient human activity model requires credible implementations of keyframe extraction, preprocessing, feature extraction and selection, classification, and pattern recognition methods. In the real-time video, sequences are untrimmed and do not have any activity endpoints for effective recognition. Thus, we propose a hybrid gated recurrent unit and long short-term memory-based recurrent neural network model for high-efficiency human action recognition in untrimmed video datasets. The proposed model is tested on the TRECVID dataset, along with other online datasets, and is observed to have an accuracy of over 91% for untrimmed video-based activity recognition. This accuracy is compared with various state-of-the-art models and is found to be higher when evaluated on multiple datasets.

Rapid and accurate determination of rice diseases is the key to accurate monitoring of agricultural conditions and timely prevention of agricultural yield reduction. Aiming at the problems of low identification accuracy of rice blast and low efficiency of manual detection, we propose an identification method for the six main diseases of rice in the natural environment. At the same time, the identification algorithm of rice false smut is studied. The purpose is to improve the efficiency of the intelligent collection of rice disease images based on convolutional neural network (CNN) and feature matching, so as to identify rice diseases faster. We propose to use SURF and HOG for feature extraction respectively. Then, 20 models were selected using K-nearest neighbors and support vector machine, respectively, for experiments using cross-validation. At the same time, residual network (ResNet) was used to conduct experimental research on six diseases of rice, and the samples were expanded and enhanced. The image scale is increased, and ResNet-20 is used to train the sample data. The experimental model achieved 96.7% accuracy on the test set. The new network achieved 97.4% accuracy. It is nearly 8% points higher than traditional image feature extraction methods. The experimental results show that the improved CNN has high accuracy and a good effect in the identification of main rice diseases.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

ZX did not agree with the retraction. MMK and JS either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

KF, WS, JK, and KG either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

YL either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

HL and LG either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

ZL, YY, BJ, and YC did not agree with the retraction.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JT and JF agreed with retraction. YW either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JC and HW either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

ST either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JC did not agree with the retraction. HW either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

BDS and SKG do not agree with the retraction.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JF and JT agreed with retraction. XL and YW either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

LX and CY either did not respond directly or could not be reached.

Using deep learning to automatically and quickly extract faults from seismic images is of practical significance. An improved U-Net algorithm is proposed by reducing convolutional layers, designing skip connections, enforcing deep supervision, and improving the loss function and learning rate to build a new model. In the operation, the feature map parameters in the network are further revised, the number of training iterations is increased, a callback function is added, and the parameter adjustment training consumes less time and space and has higher accuracy. Experiments on real public datasets show that the improved network can limit the time required to extract a 128 × 128 × 128 three-dimensional image within 200 ms, which not only requires less time and computing power than existing methods but also has an extraction accuracy as high as 97.6%.

Special section editors Johan Debayle, Wolfgang Osten, and Dmitry Nikolaev introduce the Special Section on Machine Vision: Systems, Methods, and Applications.

Dense vehicle detection in rush hours is important for intelligent transportation systems. Most existing object detection methods can work well in off-peak vehicle detection for surveillance images. However, they may fail in dense vehicle detection in rush hours due to severe overlapping. To address this problem, a dense vehicle detection network is proposed by embedding the deformable channel-wise column transformer (DCCT) into the current you only look once (YOLO)-v5l network with a novel asymmetric focal loss (AF loss). The proposed DCCT fully extracts the column-wise occlusion information of vehicles in the images and guides the network to pay more attention to the visible area of partially occluded vehicles to improve the detection and positioning accuracy of weak feature targets. The proposed AF loss is used to balance the performance between easy and hard targets and address class imbalance. Extensive results demonstrate that the proposed network can accurately detect on-road densely located vehicles, even the minority classes in real time. Compared with the baseline YOLO-v5l, the mean average precision is improved by 3.93%, and it achieves comparable results with the existing state-of-the-art methods on the UA_Detrac dataset.

There is a dire need for vision automation for edible bird’s nest (EBN) hygiene inspection. To date, an effective impurities detection method for EBNs has yet to be realized owing to the inhomogeneous optical properties, various types and sizes of impurities, and limited sample size. The impurities inspection was formulated as an anomaly detection task, and a hybrid autoencoder model that contains an autoencoder and a single layer convolutional network is proposed. The model was trained to reconstruct only nonimpurity regions of the EBN for impurities segmentation and detection as anomalies. The results showed that with only 50 EBN sample images, the hybrid model achieved a recall of 0.9282, a precision of 0.7718, and a 5.63% undetected rate for impurities. Furthermore, a misclassification rate of 21.53% was recorded due to artifacts mostly with sizes <0.20  mm that were detected as false positive. Nonetheless, the applicability of the proposed autoencoder model was confirmed, with >92  %   of successful impurities detected. Therefore, the hybrid autoencoder model is further explored for improvement and practical application.

Deep learning has resulted in a huge advancement in computer vision. However, deep models require an enormous amount of manually annotated data, which is a laborious and time-consuming task. Large amounts of images demand the availability of target objects for acquisition. This is a kind of luxury we usually do not have in the context of automatic inspection of complex mechanical assemblies, such as in the aircraft industry. We focus on using deep convolutional neural networks (CNN) for automatic industrial inspection of mechanical assemblies, where training images are limited and hard to collect. Computer-aided design model (CAD) is a standard way to describe mechanical assemblies; for each assembly part we have a three-dimensional CAD model with the real dimensions and geometrical properties. Therefore, rendering of CAD models to generate synthetic training data is an attractive approach that comes with perfect annotations. Our ultimate goal is to obtain a deep CNN model trained on synthetic renders and deployed to recognize the presence of target objects in never-before-seen real images collected by commercial RGB cameras. Different approaches are adopted to close the domain gap between synthetic and real images. First, the domain randomization technique is applied to generate synthetic data for training. Second, domain invariant features are utilized while training, allowing to use the trained model directly in the target domain. Finally, we propose a way to learn better representative features using augmented autoencoders, getting performance close to our baseline models trained with real images.

Tribology is the science and engineering of interacting surfaces in relative motion. In this context, dry friction between two bodies generates wear particles known as third body particles. We propose to characterize these particles using image acquisition and analysis. The images of wear particles are observed by scanning electron microscopy and further segmented using machine learning at the pixel level. Thereafter, the most relevant geometrical and textural descriptors are selected by a sensitivity study and correlated to tribological characteristics. The proposed tools give first quantitative results to better understand, for industrial purposes, the mechanisms involved in the wear phenomenon, and the morphology of ejected third body particles.

Brain tumors have existed for a long time, but their rate is increasing among all age groups. Early detection is essential for timely prognosis as it is a dangerous and life-threatening disease. Since the advent of artificial intelligence and machine learning, different algorithms have been proposed to classify and segment brain tumors. However, all have their limitations when it comes to local deployment. The main drawback is designing multiple architectures for multiple tasks, which increases the time and computational complexity and affects performance. This paper addresses this drawback by proposing a multi-task learning (MTL) model that can take 2D-magnetic resonance imaging (MRI) images as input and gives predictions for multiple outputs such as detection and segmentation. It gives two outputs from one architecture: the system’s efficiency. Brain Tumor Segmentation (BRaTs) 2019 and BRaTs 2020 dataset has been used to evaluate the proposed architecture. Experimental results show that the best model shows 98% accuracy for detecting MRI images as either normal/abnormal, whereas an overall Dice score of 92% for multi-class segmentation of high-grade glioma gliomas into the whole tumor, enhancing tumor and core tumor. The overall performance of the proposed architecture proves that it can be the best-suited framework for clinical setup.

Phishing is a common cybercrime that tempts a target to reveal personal and financial information by imitating a legitimate webpage. Classical techniques, such as list-based, and heuristic approaches have focused on identifying the uniform resource locator (URL). However, cybercriminals have uncovered several methods to evade URL-based techniques. Recently, researchers have started developing techniques that are based on visual similarity to detect phishing webpages. However, using visual similarity raises several computer vision-related issues such as webpage segmentation and feature extraction. Deep learning provides a suitable solution to these issues as it integrates feature extraction, feature selection, and classification in an end-to-end system. We have made the first attempt to detect phishing webpages based on visual similarity by modifying and retraining deep neural networks. A comprehensive evaluation on two publicly available databases has shown a marked improvement of around 7% classification accuracy.

The purpose of low-light image enhancement is to improve image quality assessment by human visual perception and bolster the performance of subsequent visual tasks. It is necessary to consider not only the complexity of illumination in a real scene but also the issues of color distortion after image enhancement. We propose a two-stage network to solve these problems. In the first stage, we use a two-branch encoder–decoder subnetwork to learn multiscale features of the image, in which the brightness encoder branch is used to improve the network’s attention to a low-light region and improve the problem of uneven illumination. In the second stage, we utilize the detail recovery subnetwork to preserve details. In addition, we introduce an attention mechanism and cross-stage connection between the two stages to utilize the features learned by different subnetworks effectively. Extensive experiment results demonstrate that the proposed method outperforms several state-of-the-art methods in terms of quantitative metrics and the visual effect. Thus, our network enhances low-light images with high quality.

Camouflaged object detection (COD) is a new computer vision challenge for locating and identifying camouflaged objects in complex situations. Camouflaged objects are more similar to their surroundings than conventional objects, and their appearance in terms of size and shape is also considerably different, making accurate identification of the COD tasks difficult. As a result, we propose an enhanced identification network (EINet) to strengthen the COD task’s identification capabilities. First, the pyramid vision transformer is used as an encoder for extracting more robust multiscale features. Second, the multiple texture refinement modules are exploited to refine the multiscale features. Third, an improved neighbor and hop connection decoder is designed to produce a coarse estimation map for guiding the detailed identification of camouflaged objects backward. Finally, numerous new reverse criss-cross block attention modules that gradually recognize fine-grained features at various scales is designed to allow for the accurate recognition of camouflaged objects. Extensive experiments have been conducted on four benchmarked datasets of camouflaged objects. The results of the experiments reveal that our EINet is a powerful COD model that outperforms current state-of-the-art models.

High-quality image registration is the basis of multimodal fusion in the field of industrial welds. Due to the influence of contrast, scale, and illumination, the registration accuracy and robustness of existing algorithms are not high enough for the industrial weld field, and registration performance still needs further improvement. In this case, we propose a hybrid double branch image registration (HDBR) algorithm for multisource images, which has a parallel composite structure. The experimental result demonstrates that compared with the radiation invariant feature transformation and histogram of absolute phase consistency gradients algorithms, HDBR algorithm has higher matching accuracy and lower registration loss, and the registration performance is improved by 1.12% and 15.55%. The composite structure registration algorithm we proposed is mainly applied to the multiview weld image field, and finally high-quality multiview weld data set is obtained.

Versatile Video Coding (VVC) is the latest generation of the video coding standard. In VVC, the advanced quadtree with a nested multitype tree (QTMT) partition structure provides more flexible coding unit (CU) partition sizes compared with the quadtree (QT) decision tree structure applied in the previous High Efficiency Video Coding (HEVC) standard. This flexibility, achieved by the new QTMT partitioning improvement, considerably improves the coding performance while increasing the coding computational complexity caused mainly by the rate distortion optimization processing. To overcome the complexity issue, a fast deep intra QTMT decision tree approach based on a convolution neural network (CNN) is adopted to determine the QTMT depth decision of each 128 × 128 Coding Tree Unit (CTU). The proposed algorithm predicts both the BT depths at 32 × 32 CUs and the QT depths at 64 × 64 using trained CNNs designed for each structure instead of processing the RDcost. Experimental results prove that the suggested deep QTMT approach achieves an important complexity reduction of up to 55.51% compared with the original reference software VTM3.0, with an average of about 35% encoding time reduction accompanied by an insignificant loss in encoding performance.

To solve the problems of low utilization of spatial features and incomplete contour segmentation in building semantic segmentation of remote sensing images, a building semantic segmentation method in remote sensing images based on dual-path with rich-scale features is proposed. In the shallow spatial path of proposed method, Res2Net module and inception module are used to extract shallow rich scale features to avoid improper use of shallow features affecting segmentation results. In the deep semantic path, ResNet50 combined with hybrid dilated convolution is used as the backbone network, and the obtained high-level semantic features are pooled by a spatial pyramid to capture the deeper multi-scale features. Finally, a new feature fusion module is designed to assign weights to feature maps of different levels extracted from two paths. Experimental results on WHU and Massachusetts building datasets show that the proposed method has higher building extraction accuracy and better generalization ability compared with other semantic segmentation networks.

Semantic scene segmentation has become an important application in computer vision and is an essential part of intelligent transportation systems for complete scene understanding of the surrounding environment. Several methods based on convolutional neural networks have emerged, but they have some problems, including small-scale target loss, inaccurate detailed region segmentation, and boundary category confusion. Using shallow features, we exploit the capabilities of global context information according to the theory of pyramids. A weighted pyramid feature fusion module is constructed to fuse the feature maps of different scales generated by the backbone network, and the proportion of feature fusion is dynamically updated by trainable parameters. After that, a self-attention mechanism is introduced to discover information about spatial channel interdependencies. Finally, the atrous spatial pyramid pooling module of the DeepLabv3+ network is improved by connecting the atrous convolution with different dilation rates at the receptive field. The experimental results show 4.1% mean pixel accuracy and 3.92% mean intersection over union improvements in the proposed method compared with the DeepLabv3+, and the result of semantic segmentation is more accurate.

We present a dynamic DNA coding multi-image encryption algorithm based on integer lifting wavelet transform and compound chaos. In the encryption process, the low-frequency sub-bands of four plaintexts are extracted to get a combined image containing the multiple plaintext image information. With the help of a chaotic system, the pixel positions and pixel values of the combined image are modified by the permutation and the diffusion operations. Then we divide the image into equal-sized blocks, and the dynamic DNA coding method is used to complete the final encryption. To enhance security, we employ the RSA algorithm to protect the keys used in our encryption algorithm. Simulation results and security analysis demonstrate the robustness and the security.

The goal of pedestrian trajectory prediction is to predict the future trajectory according to the historical one of pedestrians. Multimodal information in the historical trajectory is conducive to perception and positioning, especially visual information and position coordinates. However, most of the current algorithms ignore the significance of multimodal information in the historical trajectory. We describe pedestrian trajectory prediction as a multimodal problem, in which historical trajectory is divided into an image and coordinate information. Specifically, we apply fully connected long short-term memory (FC-LSTM) and convolutional LSTM (ConvLSTM) to receive and process location coordinates and visual information respectively, and then fuse the information by a multimodal fusion module. Then, the attention pyramid social interaction module is built based on information fusion, to reason complex spatial and social relations between target and neighbors adaptively. The proposed approach is validated on different experimental verification tasks on which it can get better performance in terms of accuracy than other counterparts.

Due to significant performance in the representation of data points, non-negative matrix factorization (NMF) has been widely applied in machine-learning fields, such as dimension reduction, image representation, feature extraction, data mining, and so on. However, classical NMF suffers from a common issue, low efficiency in representing the internal geometric structure of data and sparsity limitation. To circumvent this problem, we innovatively propose a semi-supervised NMF algorithm called semi-supervised dual-graph regularization non-negative matrix factorization (LOSDNMF), into which dual-graph and bi-orthogonal constraints are embedded to reduce the inconsistency between the original matrix and the basic vectors while maintaining the manifold structures of the data and feature spaces. This strategy can fully explore the potential geometry information of the data, which is extremely beneficial to enhance the learning ability of the model. In addition, the local coordinate constraints are introduced to ensure good sparsity of the coefficient matrix and simplify the calculation. Furthermore, an iterative updating scheme for the optimization problem of LOSDNMF and its convergence proofs are also provided in detail. The effectiveness of the proposed method is verified on eight benchmark datasets. Experimental results show that our method can effectively improve clustering performance.

Perceptual quality assessment of digital images is becoming increasingly important due to widespread use of digital multimedia devices. Smartphones and high-speed internet are among the technologies that have increased the amount of multimedia content by several folds. Availability of a representative dataset, required for objective quality assessment training, is therefore an important challenge. We present a blind image quality assessment database (BIQ2021). The dataset addresses the challenge of representative images for no-reference image quality assessment by selecting images with naturally occurring distortions and reliable labeling. The dataset contains three set of images: images captured without intention of their use in image quality assessment, images obtained with intentional introduced natural distortions, and images collected from an open-source image sharing platform. Ensuring that the database contains a mix of images from different devices, containing different type of objects, and having varying degree of foreground and background information has been tried. The subjective scoring of these images is carried out in a laboratory environment through single-stimulus method to obtain reliable scores. The database provides details of subjective scoring, statistics of the human subjects, and the standard deviation of each image. The mean opinion scores (MOSs) provided with the dataset make it useful for assessment of visual quality. Moreover, existing blind image quality assessment approaches are tested on the proposed database, and the scores are analyzed using Pearson and Spearman’s correlation coefficients. The image database and the MOS along with relevant statistics are freely available for use and benchmarking.

In image quality assessment (IQA), the quality of an image is essentially dependent on both semantic contents and image distortions. Conventional nonreference IQA (NR-IQA) methods either encode the semantic contents only or combine them in a straightforward manner without exploring their intrinsic correlation. A unified deep multiview learning network (DMvLNet) is proposed to combine both semantic contents and image distortions for NR-IQA. Using ResNet50 as the backbone, we design IQA-aware semantic features instead of directly using a recognition-specific CNN model for IQA, as in many existing methods. To this end, a high-level feature refining module built on deep layers is developed for generating refined high-level feature maps. Meanwhile, shallow layers are employed for low-level descriptions. Thus, both the refined high-level and low-level features are used for characterizing semantic contents and image distortions, respectively. Furthermore, the refined semantic features and distortion features are simultaneously integrated into DMvLNet by exploring their mutual correlation, leading to the projected multiview embedding passing through the subsequent transformer network for image quality prediction. Extensive experiments conducted on both our assembled large-scale dataset and seven public benchmark datasets demonstrate the superiority of DMvLNet over the state-of-the-art methods.

Though the open-air tracker has achieved an advanced level, its design is still a challenging task for degraded underwater images. Using underwater enhancement technology can improve the performance of underwater trackers. However, most underwater image enhancement methods focus on improving the visual effect rather than serving the tracker better. Therefore, we intended to explore a simple but powerful image domain-adaptive method to improve Stark’s performance by enhancing Stark’s input images. Specifically, it consists of underwater image adaptation network (UIAN) with double heads and adaptation block based on scene estimation (ABSE) that consists of three independent image processing modules without deep learning. UIAN is used to predict the category of image domain and parameters of ABSE. ABSE decodes the parameters and sequentially process underwater images in each module. The training of UIAN is independent of the training of the tracker. After training the class prediction head of UIAN first, freezing its weights, by initializing and tracking in one enhanced image and computing tracker’s loss, the parameter head can be trained to make sure UIANs hyperparameters can match the Stark tracker. The UStark proposed can adaptively process clear and degraded underwater images. Compared with Stark, UStark has improved the accuracy and success rate in typical underwater environments by 3.7% and 1.5% (blue), 5% and 3.4% (yellow), and 5.4% and 3.3% (dark), respectively. In addition, compared with other underwater image enhancement methods, our method can enhance the performance of the tracker in more categories of underwater images.

Solving the problem that features prediction of self-supervised monocular depth estimation is still ambiguous in low-texture regions and boundaries. We proposed an innovative self-supervised monocular depth estimation method, monocular depth self-supervised network, which integrates three effective strategies to construct an innovative self-supervised monocular depth estimation framework: (1) the attention mechanism and feature fusion module are adopted to enhance the semantic and spatial information of feature images, (2) the threshold segmentation mask is utilized to solve object motion and low-texture regions to increase image details, and (3) the residual pose module and deep reconstruction loss are used to enhance the feature extraction capability of the model to improve the accuracy of depth and pose estimation. Comprehensive experiments and visual analysis results demonstrate the effectiveness of each component in isolation. Compared to existing self-supervised methods, our model not only achieves outstanding results on KITTI and NYU Depth V2 datasets but also can be suitable to different environments.

The advent of convolutional neural networks has been driving the rapid development of image super-resolution (SR) tasks. Existing works, however, tend to devise deeper and wider networks to boost accuracy, leading to huge model sizes and computation costs. In addition, they also ignore the effect of frequency domain information on image restoration. To address these challenges, we propose a simple and effective frequency-based attention network, comprising a series of frequency-domain enhancement modules (FDEMs), for accurate image SR. Each FDEM integrates a two-dimensional discrete wavelet transform, progressive frequency enhancement module (PFEM), and frequency aggregation module (FAM). Specifically, DWT first decomposes the features into high-frequency and low-frequency parts, and then the low-frequency branch is processed and gradually added to the high-frequency branch to realize the interaction of frequency information in PFEM. Finally, frequency representations from the PFEM are upsampled to the FAM to further achieve information enhancement in a specific space. Extensive experiments indicate that our method is efficient in reconstruction accuracy with less model capacity, exceeding most existing lightweight SR networks.

After a few seconds of an action, the human eye only needs a few photos to judge, but the action recognition network needs hundreds of frames of input pictures for each action. This results in a large number of floating point operations (ranging from 16 to 100 G FLOPs) to process a single sample, which hampers the implementation of graph convolutional networks (GCN)-based action recognition methods when the computation capabilities are restricted. A common strategy is to retain only the portions of the frames, but this results in the loss of important information in the discarded frames. Furthermore, the selection progress of key frames is too independent and lacks connections with other frames. To solve these two problems, we propose a fusion sampling network to generate fused frames to extract key frames. Temporal aggregation is used to fuse adjacent similar frames, thereby reducing information loss and redundancy. The concept of self-attention is introduced to strengthen the long-term association of key frames. The experimental results on three benchmark datasets show that the proposed method achieves performance levels that are competitive with state-of-the-art methods while using only 16.7% of the number of frames (∼50 and 300 frames in total). On the NTU 60 dataset, the number of FLOPs and Params with a single-channel input are 3.776 G and 3.53 M, respectively. This would greatly reduce the excessive computational power cost in practical applications due to the large amount of data processed by action recognition.

Edge-preserving image smoothing is a fundamental tool in computational photography and graphics. It aims to suppress insignificant details while maintaining salient structures. The classical L₀ filter provides an elegant framework for a variety of applications. However, it inclines to sharpen the salient edges, thus suffering from gradient reversals and color deviations. We propose a solution toward the objective of optimizing summed squared error regularized by the L₀-norm of the gradients. The proposed solution explores an iterative strategy, where each iteration is an optimization problem based on the truncated L₁ regularization, which can be solved efficiently with the combination of alternating direction method of multipliers and Fourier domain optimization. Leveraging L₁-norm limits the aggressive modifications of gradients during the iterative process, which alleviates various artifacts in classical L₀ filter. Experiment results indicate that the proposed method achieves superior performance in various applications, including texture removal, detail enhancement, HDR tone mapping, and compression artifact removal. The proposed filter can be conveniently implemented on GPU. It takes our filter 0.51 s to process a 720P color image on a NVIDIA GTX 1080 GPU.

In order to solve the problem of poor detection accuracy of CenterNet in remote sensing aircraft with dense targets and complex backgrounds, an improved memory CenterNet object detection algorithm is proposed. First, the multiscale receptive field module and residual attention module are introduced to improve ResNet50, which is used as a new backbone feature extraction network. It has a strong feature extraction ability and detection speed at the same time. Different dilated ratios are used in the multiscale receptive field to expand the receptive field and extract multiscale features, and the multiscale target features are highlighted through the attention mechanism. Then the last set of residual networks in ResNet50 is deleted, and the residual attention network is introduced and stacked in the same way. The feature parameters of different channels in the residual attention network are calculated by two-dimensional cosine transform and weighted with the original feature map, and the target features are highlighted to enhance the ability to locate the target at the end of the network. Second, three times deconvolution is replaced by a memory feature fusion module. The shallow detail information and the deep semantic information are fully fused. The key information in the output feature map is further enriched, which is beneficial to the improvement of model detection performance. Finally, the experimental results show that the average accuracy of remote sensing aircraft dataset is 88.46%, which is 18.21% higher than that of the original algorithm and effectively improves the detection accuracy of remote sensing aircraft targets.

Visual simultaneous localization and mapping (VSLAM) is one of the foremost principal technologies for intelligent robots to implement environment perception. Many research works have focused on proposing comprehensive and integrated systems based on the static environment assumption. However, the elements whose motion status changes frequently, namely short-term dynamic elements, can significantly affect the system performance. Therefore, it is extremely momentous to cope with short-term dynamic elements to make the VSLAM system more adaptable to dynamic scenes. This paper proposes a coarse-to-fine elimination strategy for short-term dynamic elements based on motion status check (MSC) and feature points update (FPU). First, an object detection module is designed to obtain semantic information and screen out the potential short-term dynamic elements. And then an MSC module is proposed to judge the true status of these elements and thus ultimately determine whether to eliminate them. In addition, an FPU module is introduced to update the extracted feature points according to calculating the dynamic region factor to improve the robustness of VSLAM system. Quantitative and qualitative experiments on two challenging public datasets are performed. The results demonstrate that our method effectively eliminates the influence of short-term dynamic elements and outperforms other state-of-the-art methods.

Training a robust artificial facial expression recognition system requires diverse samples. In-the-lab datasets are collected under class-balanced distribution. By contrast, samples in the real world are more complex and imbalanced. Traditional data augmentation (DA) methods apply random operation amplitudes and unidirectional expansions in the feature space, which leads to bias in classes with more samples. Our paper proposes a DA model and combines it with a feature enhancement mechanism for unbalanced facial emotion recognition. We introduce a learning-based dynamic sample adjustment mechanism that selects the most valuable samples to guide the training process of each sample’s augmentation parameters. It explores each sample’s maximum augmentation potential and increases the accuracy in a class-balanced way. Compared with state-of-the-art methods on the facial emotion recognition task, it achieves the best average accuracy of 86.08% on the real-world affective faces database, outperforming the state-of-the-art imbalanced learning methods. Primarily, it shows significant accuracy improvement for extreme minority classes, which improve by over 15% at best compared with previous works.

To solve the problems of poor authenticity and lack of clarity for short-time typhoon prediction, we propose a self-attentional spatiotemporal adversarial network (SASTA-Net). First, we introduce a multispatiotemporal feature fusion method to fully extract and fuse the multichannel spatiotemporal feature information to effectively enhance feature expression. Second, we propose an SATA-LSTM prediction model that incorporates spatial memory cell and attention mechanisms in order to capture spatial features and important details in sequences. Finally, a spatiotemporal 3D discriminator is designed to correctly distinguish the generated predicted cloud image from the real cloud image and generate a more accurate and real typhoon cloud image by adversarial training. The evaluation results on the typhoon cloud image data set show that the proposed SASTA-Net achieves 67.3, 0.878, 31.27, and 56.48 in mean square error, structural similarity, peak signal to noise ratio, and sharpness, respectively, which is superior to the most advanced prediction algorithm.

Facial expression recognition plays an increasingly important role in many fields. The richness of facial expressions and the ambiguity of expression recognition increase the difficulty of facial expression recognition. The recognition rate of existing algorithms is either high or complex. Either the complexity is low or the recognition rate is also low. Can a compromise be made between complexity and recognition rate? So, we designed a basic and effective convolutional neural networks (BASE) algorithm. In order to further improve the performance of the BASE algorithm, we propose a facial expression recognition algorithm based on efficient channel attention for deep convolutional neural networks (CECA-NET). We tested it on four commonly used facial expression databases. The efficient channel attention module increases the interactivity between channels and the key information that is not easy to be paid attention to is excavated. Experiments show that our method can obtain richer facial expression information and has stronger generalization ability.

With the remarkable success of the image captioning tasks, visual attention methods have become a vital part of captioning models. However, most attention-based image captioning methods do not consider any relationship among regions, which play a significant role in better image understanding. We proposed an image captioning method based on local relation network using a multilevel attention approach with graph neural network. It not only fully explores the relationship between the object and the image regions but also generates significant and context-based features corresponding to every region in the image. The attention employed in our work enhances the image representation capability of our method by focusing on a given image region and its related image regions. Thus addressing the relevant contextual information, spatial locations, and deep visual features leads to improve caption generation. We verified the effectiveness of the proposed model by conducting extensive experiments on three benchmark datasets: Flickr30k, MSCOCO, and nocaps. The results show the superiority of the proposed method over the existing methods both in quantitative and qualitative manners. Detailed ablation studies are conducted to communicate how each part would contribute to the final performance.

Recently, single object tracking has demonstrated great success. However, due to various problems caused by fast motion, occlusion, and deformation, it is still intractable for traditional trackers to adapt to changes in object appearance. In this work, a dynamic memory network is proposed for visual tracking to handle the template matching process globally. More specifically, we first build a memory model, which consists of a memory feature fusion module and a memory bank. By the memory model, the network not only accepts the first frame as initial information but also memorizes the selective frames in the video sequence to provide rich time-domain information. Second, an innovative sampling strategy is adopted in the tracking process. By updating the template and guiding the selection of memory frames, our model can output higher-quality features. In addition, a spatial-channel fused attention module that effectively improves the representational capability and discriminability of the model is introduced. Our proposed method obtains compelling results on six challenging tracking benchmarks, including the OTB100, VOT2019, UAV123, NFS, GOT-10k, and LaSOT datasets. Extensive experiments demonstrate that our approach shows satisfactory robustness and leading application potential in real-time speed.

As an important part of fire prevention and control, there are high standards for real-time, anti-interference, and accuracy about flame detection. At present, flame target detection methods lack comprehensive research on the above three indicators. To solve this problem, we propose a lightweight anti-interference flame detection method based on improved YOLOv4-tiny. On the ground of dynamic characteristics of flame changing with time, a double-stream structure of flame detection model is designed. First, deeply separable convolution is applied to YOLOv4-tiny, resulting in a lighter backbone network. Second, in the feature extraction stage, the learning ability of the network to shallow features is improved by further integrating multiscale features, and the efficient channel attention module is introduced into the feature pyramid network to further improve the accuracy. Finally, intersection over union (IOU) postprocessing algorithm is used to shield the interference of fire-like targets effectively. Experimental results show that the parameter quantity of our methods are 3.95 and 4.22 MB, respectively, and accordingly their precisions reach 94.10% and 94.27%. Detection times are 35 and 46 ms, separately. In addition, after using IOU postprocessing algorithm, mAP and F1 composite index both increased to varying degrees, the consumption time is only 0.23 ms.

Most of the three-dimensional (3D) object detection methods based on LiDAR point cloud data achieve relatively high performance in general cases. However, when the LiDAR points have noise or some corruptions, the detection performance can be severely affected. We propose a 3D object detection method that combines point cloud information with two-dimensional (2D) semantic segmentation information to enhance the feature representation for difficult cases, such as sparse, noisy, and partially absent data. Motivated by the Pointpainting techniques, we designed an early-stage fusion method based on a Voxel region-based convolutional neural network (R-CNN) architecture. The 2D semantic segmentation scores obtained by the Pointpainting techniques are appended to the raw point cloud data. The voxel-based features and 2D semantic information improve the performance in detecting instances when the point cloud is corrupted. In addition, we also designed a multiscale hierarchical region of interest pooling strategy that reduced the computational cost of Voxel R-CNN by at least 43%. Our method shows competitive results with the state-of-the-art methods on the standard KITTI dataset. In addition, three corrupted KITTI datasets, KITTI sparse (KITTI-S), KITTI jittering (KITTI-J), and KITTI dropout (KITTI-D), were used for robustness testing. With the noisy LiDAR points, our proposed point painted Voxel R-CNN achieved superior detection performance over that of the baseline Voxel R-CNN for the moderate case, with a notable improvement of 11.13% in average precision (AP) on the 3D object detection and 14.3% in AP on the bird’s eye view object detection.

6D object pose estimation is essential for many applications with high demands in accuracy and speed. Compared with end-to-end approaches, pixel-wise voting network (PVNet), a vector-field based two-stage approach, has shown the superiority in accuracy but the inferiority in speed because of the time-consuming RANSAC-based voting strategy. To resolve this problem, we propose an efficient deep architecture that consists of an enhanced vector-field prediction network (VPNet) and a keypoint localization network (KLNet) and call it VP-KLNet. Specifically, the KLNet replaces PVNet’s time-consuming voting scheme by directly regressing 2D keypoints from the vector fields, which significantly improves the inference speed. Furthermore, to capture multiscale contextual information, we embed the pyramid pooling module between the encoder and decoder in VPNet to obtain more accurate object segmentation and vector-field prediction, with negligible speed loss. Experiments demonstrate that our method has more than 50% improved in the running speed to the baseline method PVNet and achieves comparable accuracy with the state-of-the-art methods on the LINEMOD and occlusion LINEMOD datasets.

Person reidentification (Re-ID) aims to match specific pedestrians across nonoverlapping camera views. Due to the dramatic disparities between different datasets, transferring a Re-ID model trained on the source domain to the target domain is challenging. Some outstanding unsupervised domain adaptation (UDA) Re-ID methods use clustering to generate pseudolabels, optimizing the model on the target domain, but the pseudolabels are inevitably noisy. To address the above issues, we propose a framework named mini-transformer with pooling (MTP) to facilitate the generation of superior quality pseudolabels by improving the model’s feature representation capability. First, we introduce an effective mini-transformer (MT) that can be placed directly behind the CNNs as a feature extractor to capture long-range dependency. Then, we design two delicate pooling methods named global hybrid pooling (GHP) and global subvalue pooling (GSVP) to suit mini-transformer’s tremendous capability without increasing computational complexity. Specifically, GHP can keep more global information and GSVP can keep more discriminative information. Finally, experiments on four mainstream UDA Re-ID tasks demonstrate that MTP achieves competitive mAP and rank-1 accuracy to the current state-of-the-art methods, suggesting that our technique is simple but effective. In addition to UDA Re-ID, our MTP can be extended to other supervised retrieval tasks.

As an effective method of information hiding, steganography embeds secret information into images in a way that is not perceived by humans. Recent interest in the combination of image steganography and generative adversarial networks (GANs) has yielded rapid progress. However, existing steganography frameworks still suffer from the low quality of the steganographic images and weak resistance to detection by steganalysis algorithms. To overcome these limitations, we propose an effective GAN-based image steganography framework with multiscale features integration. Specifically, we construct the secret image feature extraction network (SfeNet), which is driven by the spatial attention mechanism to extract multiscale features of secret images. And the encoder combined with the efficient channel attention mechanism is presented to embed multiscale features of secret images into the cover image. Subsequently, a steganalyzer is incorporated as the discriminator with the encoder in GAN to strengthen the ability of the model to resist steganalysis. Besides, a mixed loss function is proposed by combining perceptual loss, MS-SSIM, and L1 loss to preserve the structural similarity of images. Experimental results on ImageNet, Pascal VOC2012, and LFW show that the proposed method achieves better quality steganographic images and better resistance to steganalysis compared to some of the state-of-the-art steganography algorithms.

Image segmentation has been commonly used for segmenting regions of interest in the industrial field. However, segmenting small objects in a reflective background is a complex task that limits the performance of image segmentation in the industry. We take residual cut tobacco in industrial production as an object and propose an automatic detection method. This method overcomes the influence of reflective background and achieves accurate detection of small-area residual cut tobacco. The proposed method combines multi-feature fusion method and Markov random field model to construct a coarse-to-fine segmentation framework. The coarse part is used to eliminate the effect of specular highlights and specular reflection. The fine part is used to detect and clear small regions of residual cut tobacco that are wrongly segmented by the coarse part. We collect 2411 images in the real industrial production line for evaluation. The experimental results show that the proposed method yields an averaged precision and mean intersection over union of 95.85% and 86.71%, which outperforms some detection methods based on deep learning.

Light-field images captured by light-field cameras usually suffer from low spatial resolution due to the inherent limited sensor resolution. Light-field spatial super-resolution thus becomes increasingly desirable for subsequent applications. Although continuous progress has been achieved, the existing methods still failed to thoroughly explore the coherence among light-field views. To address this issue, we propose an efficient neural network for light-field spatial super-resolution, in which the spatial and angular information can be fully exploited by repeatedly alternating spatial and angular domain. Specifically, an enhanced spatial-angular separable convolution block is proposed to efficiently exploit the correlation information between sub-aperture images. Moreover, a multi-scale feature extraction block is introduced to extract feature representations at different scales and capture rich texture and semantic information. Experimental results on both synthetic and real-world light-field datasets demonstrate that the proposed method outperforms other state-of-the-art methods with higher peak signal-to-noise ratio (PSNR)/structural similarity (SSIM) values and fewer parameters.

In recent years, automatic image crack detection has become a critical task for ensuring the safety of various facilities. Many researchers utilized convolutional neural networks (CNNs) for crack detection. However, the existing CNN methods have the limitation of the receptive fields and struggle to establish long dependencies and global background information. Aiming to address this problem, we propose a CNN method with an attention module and recurrent mechanism. In terms of the backbone network, we propose a backbone network named Attention-VGG-19. In our network, we remove the last three fully connected layers of VGG-19 to expect the meaningful side output with different scales and to reduce the number of parameters. The fully connected layers are computationally intensive. The proposed network also provides integrated direct supervision for features of each convolutional stage. Then, to promote important features and suppress unimportant features, an attention module called CrackP attention is proposed. In the recurrent mechanism, the module can be unrolled into R loops to connect the path between one pixel and its neighboring ones. To prove the effectiveness of our proposed method, we evaluate it on four public crack datasets, DeepCrack, crack forest dataset (CFD), Crack500, and CrackTree260, on which it achieves F-score (F₁) values of 0.876, 0.652, 0.692, and 0.593, respectively. After a guided filter is added, our method achieves F₁ values of 0.878, 0.638, 0.681, and 0.526 on these four datasets.

Since the person reidentification (Re-ID) technique was proposed, it has successively produced successful research. The performance of this technique still has room for improvement due to the challenges of cluttered background and spatial misalignment. Recently, researchers have attempted to integrate human parsing or pose estimation results to capture person regions for mitigating these problems. However, many important and recognizable cues remain in the background regions. To mine these useful cues while still noticing the informative human body parts, this work proposes a simple yet effective two-stream model that utilizes one stream from human instance to learn discriminative human part features and enriches the representation with the other stream from the original image. Moreover, to alleviate the spatial misalignment problem, this paper rectifies the feature extraction regions with the assistance of human segmentation results. Experimental results demonstrate that the proposed method significantly improves the performance of person Re-ID.

Smoke is the first visible sign of forest fires and the most commonly used feature for early forest fire detection using data from cameras. However, one of the natural challenges is the dense fog that appears in forests, which decreases the detection accuracy or triggers false alarms. In this study, we propose a system with a deep neural network-based image preprocessing approach that significantly improves the smoke segmentation and classification performance by dehazing the camera view. Our experimental results provide that the classification models reach 99% F1 score for the correct classification of smoke when the image dehazing method is used before the training process. The smoke localization system achieves 60% average precision when the mask region-based convolutional neural network is used with the ResNet101-FPN backbone. The proposed approach can be utilized for all smoke segmentation frameworks to increase fire detection performance.

Graph attention network (GAT) is a promising framework to perform aggregation and massage passing on graphs. GATs attention layer extracts the attention coefficients mainly using node features of the current node. But, we believe that attention can be obtained directly from the structural information of the graph. We argue that local graph structures play a dominant role for calculating good attention coefficients and propose a structure-based graph attention layer called local neighborhood graph attention layer (LNGAL). Different from GAT, LNGAL obtains attention only by local graph structural information and abandons the dependence on the features of nodes, which calculates the attention coefficients directly from theoretical formulas, instead of training with labeled data. In LNGAL, the first-order neighborhood of the current node provides a primary contribution to the calculation of attention coefficients, while the second-order neighborhood plays a fine-tune role. Furthermore, we introduce the proposed LNGAL to a multi-scale architecture and design a novel network called local neighborhood graph attention network (LNGAT). In the experimental section, we show that LNGAT network outperforms several recently proposed graph convolutional network-like models and achieves state-of-the-art performance on six open graph datasets.

The demand for applying the iris segmentation model on mobile devices has been growing rapidly. Most current segmentation networks have an enormous amount of parameters, hence unsuitable for mobile devices, while other small memory footprint models follow the spirit of classification networks and ignore the inherent characteristic of segmentation. To address the challenge, we propose a lightweight segmentation network (LiSeNet) for iris segmentation of noisy images. Unlike previous studies that only focus on improving the accuracy of segmentation masks, LiSeNet can simultaneously obtain segmentation masks, parameterized pupillary and limbic boundaries of the iris, further enabling CNN-based iris segmentation to be applied in any regular iris recognition systems. We first propose a multiscale concatenate (MSC) Block, which connects multiple sizes of convolution kernels in a dense manner, gradually reduces the dimension of feature maps and uses the aggregation of them for image representation. Based on the MSC block, we develop a two-stage refinement encoder to aggregate discriminative features through subnetwork feature reuse and substage feature reassess, thus obtaining a sufficient receptive field and enhancing the model learning ability. To exploit object contextual information more efficiently, we further devise a grouped spatial attention to emphasize the important features and suppress irrelevant noises through a gating mechanism in the decoder. Extensive experiments on three challenging iris datasets show that LiSeNet, without any complicated postprocessing, achieves competitive or state-of-the-art performance with only 2.2M parameters, being 14 × smaller than the previous best method. Code will be publicly available.

Identification of people using different biometric data is becoming more important in network society. Biometrics include voice, ears, palms, fingerprints, faces, iris, retina, and hand shapes. Among these features, iris detection gets more attention because each iris type is unique and does not change throughout life. In this study, an iris recognition framework is proposed using deformable circular hollow kernels and uniform histogram fusion images (UHFIs). This system introduces two different approaches for the iris recognition: one with image matching without machine learning and the other one with machine learning. In the first approach, image matching through Gabor features is employed. After getting the circularly cropped UHFI, Gabor features are extracted and employed in the image matching-based recognition system. In addition, a normalized cross-correlation coefficient is used as a similarity metric to compare the Gabor feature vectors. In the second approach, Gabor feature images are extracted and employed in deep learning (DL). According to the experimental results, the proposed system reaches around 89% accuracy on MMU1, 86% accuracy on MMU2, and about 50% accuracy on the CASIAV3 and CASIAV4 datasets when no machine learning is employed. Extensive benchmarking results also indicate that the proposed system boosts the performance of conventional systems by at least around 40% in terms of accuracy in the absence of machine learning. However, in the presence of machine learning, experimental results show that the proposed method with DL achieves an accuracy of up to 100% for MMU1, MMU2, CASIAV3, and CASIAV4 datasets.

The erratum corrects an error in Table 2.