Unsupervised visible-infrared person re-identification (USVI-ReID) plays a crucial role in computer vision. The key challenge of USVI-ReID is to learn the discriminative features of images and establish cross-modal correspondence without using class labels. We propose a two-stage contrastive learning method for USVI-ReID. The first stage is instance-wise contrastive learning for learning a discriminative model. The learned discriminative model is transferred to the second stage for clustering operation, thus forming category-level supervision and promoting the execution of cluster-wise contrastive learning. Besides, a progressive training strategy is proposed to gradually shift the model’s attention from instances to clusters. Extensive experiments on two public datasets SYSU-MM01 and RegDB demonstrate the effectiveness of the proposed method.

Convolution neural networks (CNNs) and transformers are good at extracting local and global features, respectively, whereas both local and global features are important for the no-reference image quality assessment (NR-IQA) task. Therefore, we innovatively propose a CNN–transformer dual-stream parallel fusion network for NR-IQA that can simultaneously extract local and global hierarchical features related to image quality. In addition, considering the importance of saliency in NR-IQA, a saliency-guided CNN and transformer feature fusion module is proposed to fuse and optimize the hierarchical features extracted by the dual-stream network. Finally, the high-level features of the dual-stream network are fused through the local and global cross-attention module to better model the interaction relationship between local and global information in the image, and the quality prediction module containing evaluation and weight branches is used to obtain the quality score of distorted images. To comprehensively evaluate the performance of our model, we conducted experiments on six standard image quality assessment datasets, and the experimental results showed that our model has better quality prediction performance and generalization ability than previous representative NR-IQA models.

The hippocampus, a crucial structure in the brain, plays a significant role in the early diagnosis of brain disorders such as Alzheimer’s disease through its structural and volumetric changes. To address the medical challenge of accurately segmenting the hippocampus, we propose a lightweight hybrid segmentation network called a parallel cascaded feature reconstruction network (PCFR-Net). This network integrates the advantages of global self-attention and local convolution while utilizing fewer model parameters. Specifically, we introduce a feature reconstruction (FR) module and a multibranch asymmetric residual attention module aimed at accurate segmentation of hippocampus magnetic resonance imaging. The model combines the strengths of the transformer in capturing long-distance relationships and adapting to irregular shapes, as well as the FR block, which can reduce the redundancy in space and channels during feature extraction, and then reconstructs feature maps to enhance the representative feature learning. In addition, the multibranch residual attention module employs the asymmetric residual convolution block, enabling fine-grained feature extraction along the length, width, and depth directions at multiple scales. Remarkably, the proposed PCFR-Net achieves a Dice similarity coefficient (DSC) of 92.74% and an Intersection over Union (IoU) of 86.5% on the Medical Segmentation Decathlon, as well as a DSC of 93.86% and an IoU of 89.29% on the Alzheimer’s Disease Neuroimaging Initiative dataset.

Real-time object detection has consistently confronted the challenges of the trade-off between inference latency and detection accuracy, especially in dense scenarios. The attention mechanism is widely used in improving detection accuracy by obtaining the awareness of networks on the critical region of the raw inputs. Previous works proposed to generate attention along different feature dimensions to strengthen selective attention, but the optimal leverage of attention is still a challenging problem. We argue that the extra trainable parameters and lacking multi-scale capabilities induced by current attention designs make it difficult to detect dense targets accurately in real time. We propose a novel parameter-free attention called guided supervised attention (GSA). Instead of using trainable parameters, GSA innovatively explores the objectness from the detector’s output to generate a reliable spatial attention mask. With the inherent characteristic of label assignment, GSA is also scale-aware, thus improving the multi-scale capability for dense target scenarios. Combined with deep supervision techniques, such a parameter-free design ensures a low training and inference cost, thus realizing a good trade-off between speed and accuracy. Our experiments empirically prove that the one-stage method applied with GSA on the head obtains an average precision (AP) improvement of 0.2% to 1.1% on the CrowdHuman benchmark with negligible additional computational cost. Furthermore, with GSA, the YOLO series achieves an AP of 92.88% CrowdHuman, which is found to have the best AP among all the proposal-based methods, presenting a competitive performance while maintaining real-time inference. The study of the GSA introduces a novel application of objectness information in object detection.

In the realm of video understanding and analysis, solely relying on the appearance features of individuals in video frames significantly falls short of enhancing the accuracy of group activity recognition. The comprehensive utilization of various feature information present is deemed crucial, playing a pivotal role in understanding group activities. Consequently, a three-stream architecture model for feature learning is proposed. This model not only considers the human appearance features and available scene-level context information for group activity recognition within videos but also emphasizes the model’s perception of individual motion, uncovering valuable information about motion features. Integrating appearance, motion, and scene-level context information affords a more comprehensive and rich representation of individual features. Ultimately, these combined features are employed in relation analysis to better predict group activities. The effectiveness of the proposed method is validated on two benchmark datasets, volleyball and collective activities, demonstrating its efficacy for the task.

Non-local attention (NLA) is a powerful technique for capturing long-range feature correlations in deep single-image super-resolution (SR). However, NLA suffers from high computational complexity and memory consumption, as it requires aggregating all non-local feature information for each query response and recalculating the similarity weight distribution for different abstraction levels of features. To address these challenges, we propose a novel learnable collaborative attention (LCoA) that introduces inductive bias into non-local modeling. Our LCoA consists of two components: learnable sparse pattern (LSP) and collaborative attention (CoA). LSP uses the k-means clustering algorithm to dynamically adjust the sparse attention pattern of deep features, which reduces the number of non-local modeling rounds compared with existing sparse solutions. CoA leverages the sparse attention pattern and weights learned by LSP and co-optimizes the similarity matrix across different abstraction levels, which avoids redundant similarity matrix calculations. The experimental results show that our LCoA can reduce the non-local modeling time by about 83% in the inference stage. In addition, we integrate our LCoA into a deep learnable collaborative attention network (LCoAN), which achieves competitive performance in terms of inference time, memory consumption, and reconstruction quality compared with other state-of-the-art SR methods. Our code and pre-trained LCoAN models were uploaded to GitHub ( https://github.com/YigangZhao/LCoAN) for validation.

Foreign object debris (FOD) on airport runways poses a significant threat to airline safety, making the study of FOD detection algorithms crucial for aviation security. Addressing the challenges of FOD detection, such as the small pixel occupancy of FODs in images, their lack of distinctive features, and their susceptibility to misdetection or omission, we propose an enhanced FOD detection method based on YOLOv5s, named VA-YOLO. The VA-YOLO method incorporates several key modifications to improve detection accuracy. First, a small target detection layer is added in the multi-scale fusion part to enhance the identification of small-sized targets, whereas the large target detection layer is removed to reduce computational load. Second, a general but efficient MobileViT block is introduced in the backbone section, replacing convolution with transformers to learn global representations. Third, the SKAttention module is integrated into the neck section to focus more on the target’s feature information. In addition, a hybrid loss function named normalized complete-IoU (NCIoU), is proposed, which combines the advantages of complete-IoU and normalized Gaussian Wasserstein distance to enhance detection accuracy. Comparative and ablation experiments were conducted on our self-constructed FOD-Z dataset. The experimental results, under the assumption of maintaining real-time performance, demonstrate that VA-YOLO achieves 99.2% mAP@0.5, 93.6% mAP@0.75, and 84.3% mAP@0.5:0.95. Compared with the original YOLOv5s method, the detection accuracy is improved by 1.4%, 3.5%, and 3.4%, respectively. Thus, VA-YOLO effectively addresses the issues of misdetection and omission of FODs on airport runways.

Current semantic segmentation algorithms are often burdened by high computational complexity and inadequate boundary localization accuracy in complex scenarios of industrial manufacturing. To achieve lightweight semantic segmentation with high accuracy, we propose a semantic segmentation algorithm named L-DeepLabV3+. First, a CB-MobileNetV3 is proposed as the backbone network, which reduces the computational burden. Second, we propose a DSCA_ASPP module to compensate for the loss of some detailed information in the encoder and enhance the perception of boundary information. Within the DSCA_ASPP module, we propose dilated separable convolution to address the issue of dilated convolutional meshing. Finally, we introduce the Haar Wavelet Transform with Deepwise Separable Convolution (HWTS) module as a replacement for the 3×3 convolution in the decoder. It consists of a deepwise separable convolution and Haar wavelet transform module, which further reduces the number of model parameters and improves the feature representation in the decoder. Experimental results demonstrate that L-DeepLabV3+ achieves an 82.7% mIoU on the PASCAL VOC2012 dataset, utilizing only 2.73 million parameters and a floating-point computation amounting to 43.015 G. The L-DeepLabV3+ model has ∼4% of the number of parameters compared to the traditional DeepLabV3+, resulting in a significant reduction in computational burden. Furthermore, L-DeepLabV3+ outperforms common semantic segmentation algorithms while using fewer model parameters and enhancing segmentation performance. Moreover, L-DeepLabV3+ exhibits excellent performance in detecting defects in industrial ceramic substrates.

In scenarios with strong light sources, images captured by surveillance cameras often exhibit lens flare, which can significantly blur or even completely obscure the details of the surveillance footage, thereby reducing the accuracy of the monitoring data. Existing algorithms for removing lens flare often result in blurred edge details in the damaged areas of the image and suffer from low computational efficiency. To address these issues, this paper proposes an algorithm for removing lens flare from images by integrating semantic information. Based on the principles of optical flow, this algorithm constructs a feature map fusion model to learn and enhance the semantic information between adjacent feature maps, thereby capturing the semantic information in the lens flare areas and restoring the lost edge details of the damaged regions. In addition, an improved channel attention mechanism is introduced, which uses a scaling factor to normalize the features of irrelevant background areas and lens flare, reducing the weight of features from irrelevant areas, and specifically extracting semantic features from the lens flare area to improve the computational efficiency of the algorithm. The feature map fusion model and the improved channel attention mechanism are embedded into the skip connections and decoder pathways of U-Net, focusing on fusing high and low-level semantic information and specifically targeting the lens flare areas, thus effectively removing lens flare from images. Experiments on the Flare7K++ dataset from Nanyang Technological University’s S-Lab show that our proposed algorithm achieves a peak signal-to-noise ratio of 27.7643 dB, a structural similarity index measure of 0.9739, and a reduced learned perceptual image patch similarity of 0.0437, outperforming existing methods and demonstrating the effectiveness of our approach in the field of lens flare removal.

The fuzzy K-nearest neighbors (FKNN), a renowned fuzzy classifier, has attained commendable performance. We introduce adaptive fuzzy contribution classification (AFCC), a novel fuzzy classifier that diverges from FKNN’s reliance on majority voting. Instead, AFCC employs an improved category contribution rate (ICCR) to delineate the hierarchical relationship between samples and categories. ICCRs quantify the contribution of a sample in reconstructing all categories, exhibiting resilience to noise. In the presence of noisy samples, AFCC adjusts all ICCRs concurrently, preserving their differentials and thus the hierarchical relationship between samples and categories. In AFCC, a precise subordination degree for each sample across all categories is determined, bridging the gap between samples and their respective categories. With this subordination degree established, the decision subordination degree for each test sample toward all categories can be derived using relevant samples. The category with the highest decision subordination degree is then selected. Notably, AFCC operates as a parameter-free classifier. Experiments have been extensively conducted on various datasets, including the Yale, Olivetti Research Laboratory (ORL), and Aleix Martinez and Robert Benavente (AR) face databases, as well as the Center for Pattern Recognition and Machine Intelligence (CENPARMI) handwritten numeral database. The outcomes unequivocally demonstrate that the recognition performance of AFCC surpasses that of other methods. Notably, in the AR face database experiments, AFCC achieves remarkable recognition accuracies, reaching 99% for images exhibiting expression variations and an impressive 100% accuracy for images subjected to illumination changes, showcasing its potential for applications in areas requiring robust and efficient pattern recognition, such as facial recognition, handwriting analysis, and other image-based classification tasks.

Because the reconstructed images of existing remote sensing image super-resolution algorithms have problems such as lack of texture information and loss of high-frequency details, we propose a remote sensing image super-resolution reconstruction algorithm based on a multiscale residual network. First, in the shallow feature extraction stage, we design a tandem convolution group containing different scale convolution kernels to improve the model’s ability to extract shallow information. Second, we design an enhanced attention residual block to extract deep features so that the model can capture more texture information. To further improve the performance of the model, during the feature enhancement stage, we design a pyramid-polarized attention convolution module, which uses a more efficient attention mechanism and pyramid convolution to capture long-distance dependencies in the image. To enhance the stability of the model, we use a Charbonnier loss function based on pixel loss. Finally, we use the Adan optimizer to optimize the model to improve the convergence of the model and achieve better reconstruction results. The experimental results show that our algorithm can better reconstruct the texture detail of remote sensing images and, compared with existing super-resolution reconstruction algorithms, the proposed method can achieve better subjective visual effects in multiple remote sensing image datasets.

Detecting small objects in unmanned aerial vehicle (UAV) pictures is a vital but demanding task in the field of computer vision due to factors such as object size, image degradation, and computational resource limitations. Most existing algorithms improve object detection accuracy by improving feature fusion structures and incorporating single-layer attention modules. However, the weight of single-layer attention is computed only based on the original input, which leads to insufficient exploration of correlations among cross-layer features. To remedy the above issues, we design a novel multi-scale UAV image object detection model called CDANet. First, to fully utilize the scattered detail features in the shallow layers, a cross-layer attention module named the enhanced guided attention module is designed. Second, a simplified depth-wise separable dilated spatial pyramid pooling-fast module is built to overcome the shortcomings of max-pooling operation during feature fusion. Third, an efficient coordinate guidance module is designed as the basic module for guiding the inter-channel information interaction. Fourth, a cross-spatial learning encoder module is designed to encode cross-channel information with an efficient multi-scale attention module. Last, to optimize the learning process by dynamically regulating the attention toward diverse objects, an advanced loss function Wise-IoU v3 is introduced. The results show that, on the VisDrone2021-DET dataset, the average precision and mean average precision of the proposed method are superior to YOLOv8, which are respectively improved by 3.9% and 4.9%.

Remote sensing image plays an important role in human activities and environmental monitoring, and it is often necessary to perform super-resolution (SR) reconstruction for clear effects. However, the absence of multi-scale features ignores the correlation among different feature layers, which can affect reconstruction. We propose a micro-pyramid SR generative adversarial network (MSRGAN) method for the SR reconstruction. First, the micro-pyramid block was introduced to extract multi-scale feature information. Then, a layer-by-layer additive method was proposed to fuse the feature information of different scales to improve the network performance. In addition, a parallel attention unit feature selection module is designed to enhance the correlation relationship among image features and improve the utilization of high-frequency features. Finally, the experimental results show that the proposed MSRGAN method has better reconstruction ability than the Bicubic, super-resolution convolutional neural network, efficient sub-pixel convolutional neural network, super-resolution using very deep convolutional network, residual channel attention network, and super-resolution generative adversarial network methods.

Low-light object detection is critical in computer vision, with widespread applications in nighttime surveillance and autonomous driving. Low-light environments pose several challenges, including increased image noise, insufficient contrast, blurriness, and color distortion, all of which affect target discernibility, particularly for small objects. Furthermore, the requirement for real-time processing adds to the complexity of image handling in these conditions. We introduce a novel approach to improving low-light target detection accuracy while maintaining a compact model. We enhance the YOLOv7-tiny network through several key innovations. We replace the simplified efficient layer aggregation network (ELAN-L) with the enhanced ELAN with fused diverse branch block (ELAN-DBB), which better integrates features from various branches and layers to improve information extraction from low-light images. In addition, we incorporate a new neck structure featuring the efficient cross-stage partial network module integrating Convolutional Block Attention Module and Deformable Convolutional Networks (VoVGSCSP-DCN-CBAM) module. This module combines attention mechanisms and depth-separable convolutions to enhance feature fusion and improve detection performance in low-light conditions. To address spatial awareness, we use CoordConv in place of 1×1 convolutions within the feature pyramid module, which enhances the model’s ability to capture spatial location information crucial for accurate detection. Furthermore, we introduce a specialized pruning module, DBB-pru, which, alongside comparing three pruning methods, results in a 35% reduction in model size and a 3.4% increase in accuracy compared with YOLOv7-tiny. Our approach markedly enhances the precision of low-light target detection while offering a more streamlined and efficacious model for practical deployment in detection applications.

Text-to-image (T2I) generation techniques have shown promising results using deep learning models. The recent T2I methods mainly use complex and stacked structures to maintain text-image consistency or focus on the generation of global text information while overlooking the finer details. In addition, brocade images differ from real-world images in various aspects, such as their fine texture and complex pattern elements (totem elements). To solve the problems, we first build a text-image dataset that specifically focuses on Chinese brocade. Moreover, by developing an extra pattern discriminator and the multi-scale feature fusion module, we proposed the brocade dual-discriminator generative adversarial network (BDD-GAN). BDD-GAN addresses the challenges associated with text-image consistency, as well as capturing the intricate textures and unique patterns in Chinese brocade-generated images. To fully verify the performance and effectiveness of the proposed method, we provided ablation studies and comparisons with previous works. Experimental results indicate that our method is capable of synthesizing Chinese brocade images that are higher in quality and totem details than those produced by previous algorithms.

As the application of urban road extraction becomes more widespread, the challenges of segmentation errors and embedding large models into hardware become more complex. To solve these problems, an algorithm called attention mechanism and lightweight network fusion high-resolution network (AMLN-HRNet) is proposed. The network includes a lightweight convolution called deep sparse channel and spatial encoding convolution (DSCEConv) for road extraction, which greatly reduces the number of parameters in the model. To make the extraction of roads more accurate, an attention mechanism called lightweight dynamic weighted is designed. In addition, a parameterless attention mechanism is introduced to make the model properly combine the spatial correlation and topological structure of the road to improve extraction accuracy. Through experimental results, AMLN-HRNet can effectively balance the speed and accuracy of the model.

We propose a new approach to multi-view subspace learning termed multi-view latent space learning via adaptive graph embedding (MvSLGE), which learns a latent representation from multi-view features. In contrast to most existing multi-view latent space learning methods that only encode the complementary information into the latent representation, MvSLGE adaptively learns an adjacent graph that effectively characterizes the similarity among samples to provide additional regularization to the latent representation. To extract neighborhood information from multi-view features, we introduce a novel strategy that constructs one graph for each view. Subsequently, the learned graph is approximately designed as a centroid of these graphs from different views, each assigned with different weights. Therefore, the constructed latent representation not only incorporates complementary information from features across multiple views but also encodes the similarity information of samples. The proposed MvSLGE can be solved by the augmented Lagrangian multiplier with the alternating direction minimization algorithm. Plenty of experiments substantiate the superiority of MvSLGE on a variety of datasets.

Multi-image hiding, which embeds multiple secret images into a cover image and is able to recover these images with high quality, has gradually become a research hotspot in the field of image steganography. However, due to the need to embed a large amount of data in a limited cover image space, issues such as contour shadowing or color distortion often arise, posing significant challenges for multi-image hiding. We propose StegaINR4MIH, a implicit neural representation steganography framework that enables the hiding of multiple images within a single implicit representation function. In contrast to traditional methods that use multiple encoders to achieve multi-image embedding, our approach leverages the redundancy of implicit representation function parameters and employs magnitude-based weight selection and secret weight substitution on pre-trained cover image functions to effectively hide and independently extract multiple secret images. We conduct experiments on images with a resolution from three different datasets: CelebA-HQ, COCO, and DIV2K. When hiding two secret images, the PSNR values of both the secret images and the stego images exceed 42. When hiding five secret images, the PSNR values of both the secret images and the stego images exceed 39. Extensive experiments demonstrate the superior performance of the proposed method in terms of visual quality and undetectability.

Underwater images currently suffer from color tone bias, reduced contrast, and blurring, which significantly hinder their use in advanced visual tasks. To address these visual quality issues, we present an underwater image restoration method guided by a physics model, integrating generative adversarial network and Unified Transformer frameworks. The network is divided into three parts: a physical model parameter estimation network (Par-subnet), an encoder–decoder, and discriminators. Initially, the parameter estimation network is used to compute the parameters required by the physical model, resulting in a preliminarily restored underwater image through inversion. We enhance the encoder–decoder network by incorporating the Unified Transformer and Reinforced Swin-Convs Transformer, thereby improving its capacity to capture both global and local features effectively. Simultaneously, a degradation quantization module is used to assess the degraded regions of the underwater image, further improving the restoration performance of the encoder–decoder. Given the limited availability of real underwater reference images, we employ the reliable bank mechanism to filter the best-quality results from the model output images and the reference images for training the discriminators. This process facilitates the transition from supervised to semi-supervised learning. The comparative experimental findings indicate that our methodology outperforms existing methods across both quantitative and qualitative metrics on multiple benchmark datasets, fully verifying the reliability and efficacy of our method.

Semantic segmentation is a fundamental and important task in the field of remote-sensing images. However, most convolutional neural network (CNN)-based methods often produce undesirable results of blurry object boundaries as convolution operations may inherently fail to retain the edge information while expanding the receptive field. Besides, CNNs struggle to capture subtle relationships between distant parts of an image. This limitation can be particularly detrimental to semantic segmentation as it requires a comprehensive understanding of the global image context. Transformers, nevertheless, excel at processing long-range dependencies. To address the above issues, we propose a novel three-branch network with three new modules, called GDBNet. The Swin-Transformer-based global (G) branch is for capturing global context information. The ResNet-based detail (D) branch and boundary (B) branch for processing local features and edge information. The three modules are feature preservation module (FPM), spatial interaction module (SIM), and balance feature module (BFM). FPM can maximumly retain the context information during the downsampling stage of Swin Transformer in the global branch, whereas SIM and BFM can fuse the information from different branches in a boundary-oriented manner. Experiments show that GDBNet outperforms state-of-the-art methods quantitatively and qualitatively on Vaihingen, Potsdam, and WHDLD datasets, reaching 71.87%, 75.64%, and 62.70% accuracy in MIOU metric, respectively, and generating visually better clearance and coherence of boundaries.

The accurate reconstruction of topology and texture details of three-dimensional (3D) objects from a single two-dimensional image presents a significant challenge in the field of computer vision. Existing methods have achieved varying degrees of success by utilizing different geometric representations, but they all suffer from limitations when accurately reconstructing surfaces with complex topology and texture. Therefore, this study proposes an approach that combines the convolutional block attention module (CBAM), texture detail fusion, and multimodal fusion to address this challenge effectively. To enhance the model’s focus on important areas within images, we integrate the CBAM mechanism with ResNet for feature extraction. Texture detail fusion plays a crucial role as it effectively captures changes in the object’s surface while multimodal fusion improves the accuracy of predicting the signed distance function. We have developed an implicit single-view 3D reconstruction network capable of retrieving topology and surface details of 3D models from a single input image. The integration of global, local, and surface texture features is a significant advancement that improves shape representation and accurately captures surface textures, filling a crucial gap in the field. During the process of reconstruction, we extract features that represent global information, local information, and texture variation information from the input image. By utilizing global information to approximate the shape of the object, refining shape and surface texture details through the utilization of local information, and applying distinct loss terms to constrain various aspects of reconstruction, our method achieves accurate single-image 3D model reconstruction with detailed surface textures. Through qualitative and quantitative analysis, we demonstrate the superiority of our model over state-of-the-art techniques on the ShapeNet dataset. The significance of our work lies in its ability to enhance the quality of single-view implicit 3D reconstruction by effectively integrating these features, leading to a more robust and detailed reconstruction of 3D models from single images. The source code of this work is available online at https://github.com/YangPeppa/ITIR-Net.

Recently, vision transformers started to show impressive results that significantly outperform large convolution-based models. We propose a gating network for salient object detection. The Pyramid Vision Transformer serves as the backbone network of this gating network, learning global and local representations with its self-attention mechanism. Multi-level gating units are used to recover more details of the saliency map by establishing cooperation among different levels of features, thus improving the discriminability of the whole network. With the help of multi-level gating units, the valuable context information from the encoder can be optimally transmitted to the decoder. The pyramid pooling module collects high-level semantic information. Moreover, the semantic information of each level is integrated and decoded by the feature aggregation decoder. The experimental results on five challenging benchmark databases demonstrate that the proposed method achieves more favorable performance than the current state-of-the-art methods in terms of four evaluation criteria.

Recent advancements in 3D object detection using light detection and ranging (LiDAR)-camera fusion have enhanced autonomous driving perception. However, aligning LiDAR and image data during multimodal fusion remains a significant challenge. We propose a novel multi-modal feature alignment and fusion architecture to effectively align and fuse voxel and image data. The proposed architecture comprises four key modules. Z-axis attention aggregates voxel features along the vertical axis using self-attention. Voxel-domain deformable encoder improves context understanding with deformable attention to encode voxel features. Dual-domain deformable feature alignment uses deformable attention to adaptively align voxel and image features, addressing resolution mismatches. Finally, gated fusion utilizes a gating mechanism to dynamically fuse aligned features. The multi-layer design further enhances feature detail retention and improves dual-domain fusion performance. Experimental results show our method increases average precision by 2.41% at the “hard” difficulty level for cars on the KITTI test set. On the KITTI validation set, mean average precision improves by 1.06% for cars, 6.88% for pedestrians, and 1.83% for cyclists.

Underwater image enhancement has attained significant attention due to its applications in different fields such as marine engineering and aquatic robotics. An underwater image suffers from various deterioration issues such as color loss, degraded contrast, and poor visibility due to the attenuation and scattering of light and color. In recent years, many underwater image techniques have been proposed. However, these techniques introduce some undesired color tones with heavily attenuated color channels in deep water images and also do not work well for different types of hazy underwater images. To tackle these issues, we propose a color correction network and an underwater extended dark channel method that handles color cast issues and adaptively controls the different levels of hazy underwater images. The proposed model offers not only high-quality enhanced underwater images but also preserves the surface details. The experimental results are evaluated both qualitatively and quantitatively on three underwater benchmark datasets, namely, underwater image enhancement benchmark, underwater color cast removal and color correction, and enhancing underwater visual perception, and it is found that the proposed method outperforms the state-of-the-art approaches. In addition, the proposed method is generalized to remove the different underwater image degradation issues such as haze, low light, and color degradation. The code for the paper is available at https://github.com/sangeeta-rani/underwater-image-enhacement-using-extented-transmission-map.

Currently, with the development of deep learning techniques and large models, designing efficient network models has become one of the hot topics in research. In the field of image super-resolution reconstruction, although deep convolutional neural networks have made significant progress, the increase in network complexity has led to an increase in computational overhead and excessive consumption of computational resources on high-performance devices (e.g., GPU). To address this issue, a network for image super-resolution reconstruction based on partial convolution (Pconv) and an improved agent attention mechanism is proposed. By reducing redundant computations and memory access, the network can more effectively extract spatial features, significantly reducing computational complexity while maintaining superior performance. Through experiments comparing recent methods on public datasets in terms of performance metrics, the proposed network model demonstrates leading results in objective quantitative measures, promising to provide a more efficient and viable solution for image super-resolution reconstruction tasks.

Unsupervised domain adaptation for object detection leverages a labeled domain to learn an object detector generalizing to a different domain free of annotations. We propose efficient multi-scale attention, confidence mixing, augmentation, and combination (ECAC), an adaptive object detector learning method based on a region-level confidence sample mixing strategy. Compared with the current methods, our approach crops high-confidence detection regions from both the source and target domains, augments them, and combines them to generate composite samples. In addition, consistency loss is utilized to solve the domain adaptation problem. Furthermore, we introduce the efficient multi-scale attention (EMA) into the detector. To retain channel information and reduce computational overhead, EMA attention restructures part of the channels into the batch dimension and groups the channel dimension into multiple sub-features, ensuring spatial semantic features are evenly distributed within each feature group. EMA employs a shared 1×1 convolution branch from the CA attention module, along with a parallel 3×3 convolution kernel to aggregate multi-scale spatial structure information. This approach effectively enhances the model’s focus on region-level features by integrating local and global information with multi-scale parallel sub-networks and cross-spatial learning. For pseudo-label filtering, we progressively transition from a loose to a stricter confidence threshold. Initially, this allows more pseudo-labels, facilitating the detector’s learning of target domain representations. As training progresses, stricter thresholds are applied to select more reliable pseudo-labels, gradually filtering out inaccurate pseudo-detections. Our extensive experiments on three datasets demonstrate that ECAC achieves state-of-the-art performance on two of them. On the third dataset, our method improves the mean average precision by over 2% compared with the latest methods.

Deep learning methods have demonstrated excellent performance in semantic segmentation tasks across various scenarios. However, their performance remains inadequate for real-time applications, such as road scenes, prompting extensive research into real-time semantic segmentation. We proposed a network with a two-branch architecture that is a prevalent framework in this field. We first create an efficient backbone based on the U-like residual block we proposed to enhance multi-scale feature extraction capabilities. In addition, many two-branch networks have designed complex structures for fusing the feature maps of the two branches at the end of the networks. To simplify this process, our network designs a lightweight fusion mechanism that utilizes attention calculations to guide the fusing of features. We name this module as dual-guided attention. This fusion module operates in parallel, with each branch employing a fully connected layer with few neurons to determine the correlation of the flattened feature map, thereby executing the attention and feature fusion simultaneously. Extensive experiments on the Cityscapes and Cambridge-driving Labeled Video datasets show the effectiveness of our method.

Cracks, as a typical type of road defect, are important to be detected and repaired in time for transportation maintenance. However, fine-grained crack detection remains a challenge because of the slender shape, backdrop interference, and noise. We propose an automatic fine-grained crack detection algorithm based on a local enhancement self-attention mechanism and a deep semantic-guided multi-scale fusion strategy. The proposed network is built around an encoder–decoder framework. Inspired by the characteristic of global integrity and local continuity, we employ a Local Enhancement Self-Attention Block (LE Self-AB) in the encoder part. It captures the overall trend of cracks from a global perspective while focusing on the local details in the crack regions. The decoder network adopts a deep semantic-guided multi-scale fusion strategy. This strategy enables the model to benefit from both global and local information through cross-scale feature transfer and fusion. To validate the effectiveness and reliability of our approach, training and evaluation are conducted on the CrackTree260 dataset. The generalization of the model is tested on the CRKWH100 and CrackLS315 datasets. The experimental results demonstrate that the proposed method outperforms other advanced crack detection methods with an Optimal Dataset Scale of 0.919, 0.918, and 0.894 on three datasets, respectively.

Homomorphic encryption (HE) has emerged as a prominent research focus in the development of privacy-preserving biometric authentication systems, offering advantages over conventional methods such as cancelable biometrics and biometric cryptosystems. However, many existing systems relying on HE assume the integrity of the server, which may not always be guaranteed. In scenarios where the server behaves maliciously, there is a risk of false acceptances or rejections due to arbitrary computation results aimed at conserving computational resources. To mitigate this concern, we introduce a novel approach called privacy-preserving correlation-based multi-instance iris verification on malicious servers using Cheon-Kim-Kim-Song (CKKS) fully HE (PCMIA). PCMIA leverages CKKS HE to preserve the privacy of iris templates and one-bit checksum to check the correctness of the computed result. Moreover, we introduce discriminant correlation analysis (DCA), a feature-level fusion approach that improves the pair-wise correlations and reduces the between-class correlations. Extensive experimentation with benchmark iris databases demonstrates that PCMIA fulfills the requirements of the template protection scheme, as well as instilling trust in the comparator results.

Road damage detection plays a vital role in road safety maintenance. Minor road damage cannot often be identified efficiently because of the challenge of extracting damage information, which frequently lacks minor information. In addition, due to the irregular shape of alligator road damage, the performance of existing damage detection algorithms is seriously reduced. To solve the above problems, we propose an improved YOLOv5 model called YOLOv5-MCD for road damage detection. First, a multiscale dilated convolution module combined with an attention mechanism is proposed and used to replace the spatial pyramid pooling fast module in the neck of the original YOLOv5 model. Multiscale dilated convolution with a larger receptive field can be applied to obtain more damage feature information at different scales, and the attention mechanism aggregates the spatial and channel feature information of road damage images, allowing the model to pay more attention to damaged areas. Second, the cross-layer feature fusion method is proposed to merge more road damage features to enhance the original damage feature information. Finally, deformable convolution is used to improve the adaptability and performance of our method for complex alligator damage. Our method was tested and validated by the China-MotorBike RDD2022, and the F1-score and mAP@0.5 of our model were 93.5% and 95.5%, respectively. The experimental results show that our method can efficiently identify minor and alligator road damage and achieve better detection performance.

Infrared-visible image fusion (IVF) proves to be advantageous in conveying scene information comprehensively. Although convolution neural network (CNN)–based methods skillfully extract local information, their global information modeling capabilities remain insufficient. Conversely, attention-based methods excel in capturing global information but often neglect detailed features. Existing CNN-transformer methods have achieved excellent performance, but they still exhibit shortcomings in capturing image details. In response to these limitations, we introduce a Swin Transformer integrated with an invertible neural network (INN) and correlation coefficient (CC) assistance for the IVF network, denoted as SICFuse. SICFuse incorporates an attention enhancement module to enhance the extracted image information. For global information modeling, we employ Swin blocks, capitalizing on their characteristics to deliver multi-scale information and implementing a sliding window attention mechanism, thereby further diminishing computational complexity. Recognizing attention-based methods’ efficacy in emphasizing global information, we address the shortfall in modeling local information by introducing INN blocks. These blocks effectively preserve image information, enabling lossless transmission. To enhance fusion results, CC joints the loss function. Ultimately, the image reconstruction module combines a fused image based on both global information and local features. The whole approach aims to leverage the strengths of attention mechanisms and inversion, bridging the gap between local and global information modeling for superior image fusion outcomes. The results on MSRS, M3FD, and TNO datasets demonstrate our promising performance on IVF tasks.

With the increasing demand for underwater environment monitoring and remote object identification, underwater object detection algorithms based on forward-looking sonar images have become a research hotspot. A kind of algorithm called remote deep feature fusion detection network (RDFFRN) is proposed, which aims to solve the problems of object shadow and background confusion and low detection accuracy of small objects in the underwater object detection algorithm of sonar images. First, the model proposes the deep feature extraction and fusion module (DFEFM). It achieves more multilayered feature fusion and captures more local details. Finally, the RDFFRN proposes a new head detection network, which increases the proportion of small-sized feature maps and improves the model’s discrimination ability for small underwater objects. After extensive experiments on a sonar images dataset, the RDFFRN is verified to be superior to other object detection methods. The RDFFDN improves the mean average accuracy by 3.2% over the baseline model. It can be shown that RDFFRN has a broad application prospect in the underwater operation field.

Object detection in unmanned aerial vehicle (UAV) images has recently become a popular tool with widespread applications, but due to the high altitude of UAV and the subtle inter-class differences, effectively detecting object locations and achieving accurate classification remain a challenge. Existing methods often rely on multi-scale feature fusion and attention mechanisms to address these problems. However, these methods tend to significantly increase the computational overhead and introduce unwanted background noise interference in shallow features. To address these issues, we propose a tiny object detection network for aerial imagery based on a deformable cross-attention module and enhanced classifier module (DCENet). Specifically, a self-designed deformable cross-attention module makes shallow feature maps adaptively focus on regions of interest in deep feature maps, enhancing the detection of object locations without being affected by additional background noise interference. Meanwhile, a crop-images super-resolution module (CSRM) is used to enlarge the cropped images, solving the problem that tiny objects have lower resolution and appear more blurred compared with ground-view objects. In addition, an enhanced classifier module is adopted to enhance the model’s classification capability for some similar categories, thereby improving the overall performance of the network. The module integrates both the CSRM and the ResNet-34 classifier. The experimental results show that DCENet achieves state-of-the-art performance with mean average precision values of 36.0 and 27.5 on the VisDrone-2019 and UAVDT datasets, respectively. This suggests that the proposed DCENet is more suitable for aerial image detection.

Accurate detection of surface defects is crucial for maintaining the quality of strip steel products. We propose a lightweight strip steel surface defect detection model, YOLOv8-VRLG, to address the issues of insufficient detection accuracy and large parameter sizes in existing models. First, the VanillaNet module was integrated into YOLOv8 to preserve efficient feature extraction capabilities while effectively lightening the backbone network. Second, the C2fRL module, which integrates the benefits of RepGhost and large selective kernel modules into the original C2f, was introduced to further reduce the number of parameters and dynamically adjust processing details based on image content, thereby enhancing detection accuracy. Finally, the traditional convolutional layers in the detection head were replaced with the GSConv module. This module optimizes information exchange and feature fusion with its unique mixed convolution strategy, thereby improving the model’s accuracy and computational resource efficiency. Extensive tests on the NEU-DET dataset demonstrated that the mean average precision of the YOLOv8-VRLG model improved by 1.8% compared with that of the YOLOv8n model. In addition, the computational cost and parameter count were reduced by 23.5% and 28.6%, respectively. Tests on the GC10-DET dataset further showcased the model’s exceptional adaptability and robustness in complex environments.

LiDAR-based 3D object detection is crucial for perception systems in autonomous driving. However, current methods perform poorly in detecting occluded and small objects due to the sparse and uneven distribution of point clouds in 3D scenes. To address this problem, we propose a long-range and multi-scale information interaction network (LMIINet) for 3D object detection. First, a feature extraction backbone with a spatial feature pyramid block is designed to effectively capture long-range dependencies between features. Then, multi-scale spatial features from the 3D backbone are adaptively fused at the neck to aggregate local and global contextual information. Finally, by fully utilizing the correlations between bounding box parameters, the proposed rotation-decoupled corrected intersection-over-union (RCIoU) loss is employed for the classification and regression supervision of bounding boxes to improve the one-stage point cloud detectors. LMIINet achieves competitive performance on small objects (pedestrians and cyclists) detection in the KITTI dataset. Compared with the benchmark network SECOND, LMIINet increases mAP3D/mAPBEV by 1.98%/2.09%, 12.25%/11.14%, and 6.44%/6.45% for the car, pedestrian, and cyclist classes, respectively.

Video anomaly detection (VAD) aims to identify events and behaviors in video sequences that deviate from established normal patterns. Traditionally, unsupervised VAD has been viewed as the one-class classification (OCC) task, which is predicated on the use of training data that encompass only normal events and exclude any anomaly samples. This approach can recognize previously unseen anomaly patterns. However, it may misclassify unfamiliar normal patterns as anomalies. In addition, as anomalous samples are not included in OCC training, it leads to unclear classification boundaries, which reduces the generalization ability of the model. To address these challenges, we introduce the self-supervised memory-guided and attention feature fusion method, which models normal events using optimized memory modules and attention feature fusion modules. The method not only generates pseudo-normal and pseudo-anomaly data but also significantly enhances the capability of the model in key feature identification and exploitation, thereby improving the capture and generalization of complex data relationships. Experimental results from three benchmark datasets-UCSD Ped2, CUHK Avenue, and ShanghaiTech show that our method achieves AUROCs of 99.5%, 90.9%, and 81.8%, respectively, demonstrating the efficacy of our approach. Our code is available at https://github.com/jzt-dongli/Self-Supervised-Memory-guided-and-Attention-Feature-Fusion-for-Video-Anomaly-Detection.

Sparse unmixing plays an important role in hyperspectral image analysis due to its ability to sparsely estimate abundances with a potentially large-scale spectral library. However, most of the current unmixing techniques tend to run into bottlenecks due to their tendency to oversimplify the optimization problem by incorporating convex regularizations, which may not adequately address the intricacies present in complex data representations. To address this problem, we propose a nonconvex sparse unmixing method by employing a tensor low-rankness penalty, named NSUTLR. Specifically, the proposed NSUTLR considers ℓ1/2 regularization to enhance the sparsity of abundances, which further couples with the row-sparsity-promoting iterative weights. Besides, the proposed NSUTLR characterizes the low-rank property by applying a nonconvex tensor nuclear norm based on the log-sum of singular values. The proposed NSUTLR can be solved by applying the alternating direction method of the multiplier framework. Experiment results obtained from the proposed NSUTLR and competitors conducted on simulated and real hyperspectral images demonstrate the superior effectiveness of our proposed method.

We propose a image-to-painting translation method that can generate paintings on a stroke-by-stroke basis. Unlike previous pixel-to-pixel or sequential optimization methods, our method generates a set of physically meaningful stroke parameters, which are closer to the way humans draw. These parameters can be further rendered using a renderer. We add an attention mechanism network to the proposed renderer to improve the quality of the painting images and use smooth L1 loss in the training process of the renderer to make the model converge faster. Our method can join neural style transfer, and we used Visual Geometry Group perceptual loss in the neural style transfer stage to get more realistic results. The experimental results show that the renderer used in our method is better than other renderers, and peak signal-to-noise ratio evaluation metrics are improved by 4.9% compared with previous renderers.

Pansharpening aims to enhance the spatial resolution of low-resolution multi-spectral (LRMS) images by leveraging the high-resolution panchromatic images (PANs). Deep learning–based pansharpening technology provides a suite of sophisticated unsupervised algorithms. However, there are several challenges: (1) existing methods introduce more or less some kind of a priori information but lack explicit physical priori. (2) Existing methods often lack generalization capabilities and perform well only on some specific datasets. To address these issues, an unsupervised feature-aligned pansharpening framework based on a generative adversarial network (FaPGAN) is proposed. The proposed FaPGAN employs a single-generator dual-discriminator structure, divisible into the following two steps: (1) the generator adopts a pair of adaptive complementary normalization factors to weight the fusion of LRMS and PAN, which captures the successive transition from LRMS to PAN. The dual discriminator is responsible for preserving the spatial and spectral information. (2) A novel loss is introduced, which constrains the representations in the feature domain for feature alignment, essentially improving prediction accuracy. The experimental results show that the algorithm exhibits excellent spatial and spectral fidelity when fusing images from Gaofen-2 and WorldView-3 satellite datasets, and it can be well generalized to reduced-resolution images.

We proposed a “hybrid fusion multi-ResNet transformer” (HF-MRT) model to improve the diagnosis of atopic dermatitis (AD) by using both hand-crafted and deep learning features. The Vietnam Atopic Dermatitis (VNAD) dataset, consisting of 7861 images of AD and non-AD cases, was utilized. Preprocessing and data augmentation techniques were applied to enhance model performance and generalization. Experiments were conducted across three feature scenarios: the first scenario focused on hand-crafted features, including a histogram of oriented gradients, local binary pattern, and color coherence vector; the second scenario involved deep learning features extracted from ResNet models (ResNet-50, ResNet-101, and ResNet-152); and the third scenario combined both hand-crafted and deep learning features. In all scenarios, high sensitivity rates were achieved by the HF-MRT model, reaching 91.4% in scenario 1, 93% in scenario 2, and 93% in scenario 3, providing reliable identification of most AD cases. The specificity rates, however, differed: 40.2% in scenario 1, 56.2% in scenario 2, and 60.9% in scenario 3. Scenario 3, with a sensitivity of 93% and a specificity of 60.9%, demonstrated diagnostic potential for detecting AD. Nevertheless, improvements in specificity remain necessary to enhance clinical applicability, ensuring accurate discrimination between AD and non-AD cases.

We introduce a fuzzy proportional-integral and derivative (FPI-D) rate control algorithm designed for the variable bitrate (VBR) applications within the H.266/VVC standard. The algorithm operates at the group of picture (GOP) level to minimize unnecessary quantization parameter (QP) fluctuations at the frame level. This approach ensures stable and high visual quality in the encoded video. However, it may lead to increased bitrate variations and buffering delay while the average bitrate converges to the target bitrate in the long term and satisfies the buffer constraints. The proposed algorithm employs an FPI-D controller to compute a base QP for each GOP, considering the buffer status and current bitrate. It then utilizes a new QP cascading model to determine QP offsets for GOP frames. The proposed controller has been implemented in the H.266/VVC reference software (VTM). The rate-distortion (R-D) performance evaluation demonstrates its superiority over the default rate controller employed in the VTM software in terms of peak signal-to-noise ratio (PSNR) and structural similarity index. This achievement includes significant bitrate reductions and quality enhancements while obeying the buffering constraint. Furthermore, its R-D performance is comparable to the constant quantization parameter encoding case, emphasizing its efficiency. Moreover, the buffering delay and mean absolute gradient of PSNR measurements reveal the proposed algorithm’s superiority over the default controller in terms of visual quality. These metrics affirm that compared with the default rate controller, the proposed controller utilizes more efficient buffer space, thus preventing unnecessary QP fluctuations and undesirable PSNR variations.

Perceptual hashing is an effective compression technology that maps the content of an image into a brief summary, which is essential for efficient processing in the Big Data era. However, most existing methods process images from a single view, leading to the omission of partial information. In addition, many methods utilize labels to construct a pairwise similarity matrix, which can result in significant time and space expenses. We propose a perceptual hashing algorithm based on collective matrix factorization. In particular, we embed the specific representations of each view and label information into a unified binary code learning framework. Specifically, a semantic label offset scheme is adopted to control the margins dynamically, which can avoid computational overhead, enhance semantic information, and improve the discrimination of the hashing. Experiments on several widely used datasets verify that when the threshold is T=45, the true positive rate for similar images stands at 100%, whereas the correct rate for different images is 99.9990%. Furthermore, the area under the receiver operating characteristic curve value of the proposed algorithm is 0.997510, which is higher than that of the comparison algorithms, indicating that our hashing method has better performance than state-of-the-art methods.

The advancement of video forgery technologies has made it increasingly easy to create highly realistic yet forged video content, raising significant concerns about their potential misuse. While existing forgery detection methods primarily focus on single-step forgeries, user-friendly video editing tools now enable multiple sequential forgeries. Detecting these sequential alterations is essential for identifying forged videos and understanding the forgery process. We propose a novel approach for detecting sequential video forgery. Unlike traditional binary classification tasks, our method predicts sequences of forgery steps. Our framework employs the Swin Transformer for spatiotemporal relation extraction, enhanced by a spatiotemporal relationships attention (STR) module and hierarchical attention for sequential relation modeling. Experiments show our method outperforms state-of-the-art techniques, achieving significant improvements in fixed-Acc and adaptive-Acc metrics. This research sets a new benchmark for sequential video forgery detection, providing an effective framework and valuable dataset for the community. Our results demonstrate the method’s capability in detecting complex, multi-step forgeries in video content.

In deep steganography, the model size is usually related to the grid resolution of the underlying layer, and a separate neural network needs to be trained as a message extractor. We propose image steganography based on generative implicit neural representation, which breaks through the limitation of image resolution using a continuous function to represent image data and allows various kinds of multimedia data to be used as the cover image for steganography, which theoretically extends the class of carriers. Fixing a neural network as a message extractor, and transferring the training of the network to the training of the image itself, reduces the training cost and avoids the problem of exposing the steganographic behavior caused by the transmission of the message extractor. The experiment proves that the scheme is efficient, and it only takes 3 s to complete the optimization for an image with a resolution of 64×64 and a hiding capacity of 1 bpp, and the accuracy of message extraction reaches 100%.

Capsule networks get achievements in many computer vision tasks. However, in the field of image generation, they have huge room for improvement compared with the mainstream models. This is because capsule networks cannot fully parse useful features and have limited capabilities of modeling the hierarchical and geometrical structure of the object in background noise. To tackle these issues, we propose a capsule autoencoder that can learn the part–object spatial hierarchical features, and we dub it an adversarial capsule autoencoder with style vector (StyleACAE). Specifically, StyleACAE decomposes the object into a set of semantic-consistent part-level descriptions and then assembles them into object-level descriptions to learn the spatial hierarchy information. Furthermore, we effectively apply the modified generator structure that contains novel style modulation and demodulation, which extracts the valid information of the capsules and eliminates the scaled effect of styles. The experimental results show that StyleACAE can generate high-quality images and has competitive performance to the state-of-the-art generative models.

We address the problem of self-supervised depth estimation. Given the significance of long-range correlation in-depth estimation, we propose to use the Segment Anything Model (SAM) encoder, a special Vision Transformer (ViT), to model the global context for accurate depth estimation. We also employ a convolutional neural network encoder to assist the network in gathering local information as ViT lacks spatial inductive bias in modeling local information. However, independent encoders lead to insufficient aggregation among features. To compensate for this deficiency, we design a heterogeneous fusion module that facilitates feature fusion by modeling the affinity among heterogeneous features. Due to the unbearable computational burden imposed by the introduction of SAM, we substitute the original SAM with a lightweight variant of SAM to reduce the complexity of the entire network. Extensive experiments on the KITTI dataset show that our proposed model achieves the most competitive results.

Pituitary disorders affect 20 million people around the world and could be fatal because of the delay in diagnosis. Hence, pre-stage computer-aided diagnosis (CAD) is the essence of modern and remote healthcare systems using the Internet of Medical Things (IoMT). Many existing techniques employ the localization and classification of brain tumors. They had their pros and cons. We address the IoMT-enabled CAD model and propose an IoMT-GWO-RBFGNN classification scheme applying rough-c means segmentation with a finely tuned cluster center by Grey Wolf Optimization (GWO) for the prediction of abnormal tumors using the Radial-Basis Gaussian Function Neural Network (RBFGNN). The remotely acquired MR image underwent histogram equalization with weighted average filters for normalization and further processing of rough c-means cluster-driven segmentation and localization of the tumor region. Prominent feature maps such as HOG, principal component, discrete cosine transform, wavelet transform, kurtosis, skewness, and correlation are extracted from annotated datasets. The calibrated feature vector is fed RBFGNN with a novel GWO-based cluster center for obtaining better accuracy, precision, recall, f1-score, MCC, and Jacard index values. The proposed IoMT-GWO-RBFGNN model outperforms by comparing and validating the existing techniques.

In recent years, attention mechanisms have been widely applied to target detection, significantly enhancing the performance of convolutional neural networks (CNNs). However, attention mechanisms like SE and CBAM either fail to effectively utilize spatial and channel information or introduce channel dimensionality reduction, which results in suboptimal CNNs performance. To address these challenges, we propose an attention mechanism called combination of spatial and channel attention (CSCA). CSCA first aggregates target features along two spatial directions to capture rich feature information. It then encodes the collected features using spatial information embedding (SIE) and channel information embedding (CIE), and finally fuses the encoded information, ensuring comprehensive utilization of spatial and channel information without channel dimension reduction. Building on this, we introduce CSCA-YOLO, a lightweight target detection network designed to improve detection efficiency. Extensive experiments on SMID and VOC (2007+2012) datasets demonstrate that CSCA effectively enhances the network’s ability to focus on both single and multiple target features. Moreover, on SIMD and Seaships7000 datasets, CSCA-YOLO maintains high mAP value with a significantly reduced parameter count compared to other lightweight networks, making it well-suited for practical deployment. That provides an important reference for future research on attention mechanisms and lightweight target detection networks.

Visible-infrared person re-identification, which combines visible light and thermal imaging data, provides more comprehensive and accurate monitoring capabilities under various environmental conditions. However, the differences between image modalities pose significant challenges in this field. Existing methods often focus on learning and analyzing the differences and consistencies between the semantics of different modalities, without considering the variability of background information and the considerable impact of surrounding noise on learning effectiveness. In view of this, we propose a novel multimodal pedestrian re-identification model, namely the feature fusion and spatial information adaptive network (FSIANet). Specifically, we designed an enhanced multi-feature aggregation module, which comprehensively mines semantic features from multiple scales while fusing shallow and deep features to explore feature representations from different channels and spaces. Furthermore, we also designed a spatially informed adaptive module, which simulates people’s attention to regions of interest during filming and can adaptively identify and focus on areas with dense information distribution, effectively reducing the interference of surrounding noise and irrelevant information. The superiority of our proposed FSIANet has been demonstrated through extensive experimentation on the SYSU-MM01 and RegDB datasets, compared with several other state-of-the-art methods.

Face recognition is a difficult task. It has to deal with various challenges such as lighting, orientation, and variability among different faces. Previous work has shown that the 3D face is a robust biometric modality, less affected by variations in pose or light. In addition, with the availability of depth cameras, capturing 3D data has become much easier. We propose a multi-scale framework for 3D face recognition. First, 3D point clouds are preprocessed and presented at different scales; each scale is then converted to depth maps. Next, the statistical measurements are extracted based on fast grid-based models. After that, the histogram of the oriented gradient descriptor is applied to encode the statistical grids. Finally, the features are projected and postprocessed to increase the range of data distinctiveness. Our proposed technique shows encouraging performance on Bosphorus and GavabDB databases. The recognition accuracies of the proposed algorithm achieved an overall rank-1 identification rate of 98.91% on the GavabDB database and 99.89% on the Bosphorus database.

Real-time and accurate detection of flames, smoke, and electric arcs is an important prerequisite for the safe production in factories. When dealing with complex features of flames, smoke, and arc targets, traditional detection algorithms suffer from insufficient accuracy, high missed detection rates for small targets. We proposed a smoke and fire electrical detection model MS-YOLOv5 based on YOLOv5s. First, the model’s detection accuracy for small targets is improved by clustering the anchor frame sizes suitable for small target detection based on the K-mean++ clustering algorithm. Second, the C3-DySnake module was designed, and dynamic snake convolution was utilized to replace the normal convolution module in the C3 module of the backbone network, giving the backbone network the flexibility to adaptively focus on key features. Third, the multidimensional collaborative attention module was added to the end of the neck network, so that the model focused on the local area information in the feature fusion stage while obtaining a stronger feature map. Finally, the alpha-intersection over union (α-IoU) loss function was used to retain the high-quality prediction frames, and effectively improve the model convergence speed. The MS-YOLOv5 detection model established on a self-constructed smoke and fire electrical dataset was high in precision and effective in recall, with better mAP@0.5-value compared with the original YOLOv5.

The convolutional neural network (CNN) method is widely used in the field of stereo matching, and common CNNs regress disparity information by forming cost volumes. However, during the construction of cost volume, various types of information are integrated and combined; these pieces of information are input into an aggregation network to enable full learning when there is no significant feature guidance, which is a hypodynamic approach. To address this problem, we propose a stereo-matching slice method. Our core idea is to improve the salience of cost features through slicing, enhance feature self-guidance and self-interaction, and alleviate the mismatch between network aggregation capability and cost information. First, we use a group-wise correlation stereo network as the baseline; integrate the slice idea into feature extraction, cost volume construction, and feature aggregation; and propose the Gwc-Slice. Specifically, the residual extraction network is divided into seven groups of feature extractors by channel, the outputs of the feature extractors are used to construct slice cost volumes using the adaptive slice method, and different slice cost volumes are aggregated through different scales to enhance feature interactions and guidance between slice cost volumes. Finally, a slice network is designed to further optimize Gwc-Slice and verify the effectiveness of the slice method. The test results based on the SceneFlow, KITTI2015, KITTI2012, and Middlebury datasets show that our approach can significantly improve the prediction accuracy of the baseline and achieve competitive results.

The proposed work for chart question answering (CQA) addresses crucial aspects, particularly those associated with questions requiring complex reasoning and visual references to charts and those related to optical character recognition (OCR) noise. Existing methods focused mainly on image features, neglecting the unique characteristics of the visual representation, which consequently not able to answer complex reasoning questions. Introducing structural embedding, question embedding, and a transformer-based relational understanding of the visualizations enable the proposed model to answer complex reasoning questions. The incorporation of the reading component and visual component using BERT encoding leverages both textual and visual information, which overcomes the OCR noise-related issue. The comprehensive ablation study systematically evaluates the different components of our model. Moreover, experiments are conducted on five datasets: DVQA, FigureQA, PlotQA, LEAFQA++, and ChartQA, which makes the assessment robust and generalized. The overall accuracy of our model is increased by 1.15% compared with STL-CQA on the test familiar of DVQA, 0.55% compared with PReFIL on FigureQA, 8.03% compared with Plot-QA, 2.28% compared with STL-CQA test-familiar split of LEAFQA++, and 0.40% compared with VisionTaPas on the test-set of ChartQA, which shows the effectiveness and superiority of the proposed model over existing methods on these datasets.

Most of the existing infrared and visible image fusion methods focus only on the extraction of detail and basic features of both images. However, these methods ignore the extraction of shallow common features, which causes the problem of weakening the ability of detail and basic feature extraction. Consequently, we propose an infrared and visible image fusion network based on multi-feature decomposition and transformer (MFDT-Fusion). The multi-feature decomposition is processed by three encoders. First, to fully extract the shallow common features of the images, multi-head gated block based on transformer improvement is proposed as the common feature encoder (CFE). Multi-head dilated attention is designed in CFE to aggregate information from different receptive fields of infrared and visible images. Then, the long-range information-capturing ability of the transformer is used to compensate for the limited receptive field problem of the convolutional neural network. The detail feature encoder and basic feature encoder are designed to decompose the two features of the images. Finally, the features are separately fused by detail and basic fusion layers. Experimental results on three public datasets show that MFDT-Fusion achieves better performance than state-of-the-art methods.

Remote sensing images are often characterized by complex background environments, significant scale variations, and high visual similarity. Most remote sensing object detection algorithms have adopted the structure of a feature pyramid with coupled heads. However, this design suffers from weak feature fusion capabilities and limited algorithm flexibility. To tackle these issues, we propose a remote sensing object detection algorithm based on the improved YOLOv7-tiny. First, we introduce the spatial information integration module to address the imbalance between semantic and positional information during feature extraction. Second, the GD-Neck, based on a gather-and-distribute mechanism, is adopted to mitigate the issue of information loss during feature fusion in the neck network. Finally, the original coupled detection head is replaced with a re-designed multi-channel decoupled head to solve the problem of performance degradation due to the shared network layer of the coupled detector head limiting the flexibility of the algorithm. Validated on the public datasets RSOD and NWPU VHR-10, the experimental results show that the algorithm in this paper obtains an accuracy improvement of 4.8% and 3.7%, respectively, and realizes high-performance remote sensing object detection.

The increase in transmission errors poses a serious challenge to underwater acoustic (UWA) communication methods, especially with the real-time error-correcting transmission requirements for underwater videos. We propose a video transmission method that enhances efficiency by using target segmentation. By integrating the reliability of UWA channels with the structure of compressed video, this approach effectively combats the loss of crucial information. An enhanced YOLOv5 algorithm is proposed to partition video information flow into different importance levels, and orthogonal frequency-division multiplexing (OFDM) is employed for transmission. This allows video data assigned to higher importance levels to be transmitted over the low-frequency bands with lower attenuation, whereas less important data are transmitted over the high-frequency bands with higher attenuation. Simulation results demonstrate that our proposed underwater video transmission method significantly improves the peak signal-to-noise ratio compared with three baseline techniques in the signal-to-noise ratio range of 25 to 50.

Sequential and causal relationships among actions are critical for accurate video interpretation. Therefore, capturing both short-term and long-term temporal information is essential for effective action recognition. Current research, however, primarily focuses on fusing spatial features from diverse modalities for short-term action recognition, inadequately modeling the complex temporal dependencies in videos, leading to suboptimal performance. To address this limitation, we propose a skeleton-weighted and multi-scale temporal-driven action recognition network that integrates RGB and skeleton modalities to effectively capture both short-term and long-term temporal information. First, we propose a temporal-enhanced adaptive graph convolutional network. This network derives motion attention masks from the skeletal joints and transfers them to RGB videos to generate visually salient regions, thereby achieving a concise and effective input representation. Subsequently, we develop a multi-scale local–global temporal modeling network driven by a self-attention mechanism, which effectively captures fine-grained local details of individual actions along with global temporal relationships among actions across multiple temporal resolutions. Moreover, we design a multi-level adaptive temporal scale mixer module that efficiently integrates multi-scale features, creating a unified temporal feature representation to ensure temporal consistency. Finally, we conducted extensive experiments on the NTU-RGBD-60, NTU-RGBD-120, NW-UCLA, and Kinetics datasets to validate the effectiveness of the proposed method.

Due to the easy accessibility of various image editing tools, image authentication has become a demanding area of research in multimedia technology. An image hash can be used for the authentication of an image that could be sensitive to content changes and invariant to perceptually similar images. The main problem in image hashing is when a system is designed to address the issues of perceptual robustness and discrimination simultaneously, as they are both inverse to each other. To address this problem, we exploited the phase spectrum of the Fourier transform (PFT), a visual attention model, to reveal the saliency map of the pre-processed test image. After that, the first-level discrete wavelet transform (DWT) is applied to obtain the four subbands, out of which the low low (LL) subband, which is located at the top left corner, comprises the lower-frequency components of the DWT and has a quite small variation because of suitable manipulations, similar to JPEG compression, and any malicious tampering in a particular portion of the image results in clear variations in the resultant coefficients of the LL subband. These LL subband coefficients are further exploited using the discrete cosine transform (DCT), as it has an energy compaction property. The energy of the image is gathered into a few DCT coefficients. Among these coefficients, the first one is the DC, which has the main information of the image, and the other ones are AC coefficients, which have the detailed information of the data. The hamming distance is used as a similarity metric among different hashes. Experimental validation is performed, focusing on malicious tampering detection, perceptual robustness, discrimination, hit rate, error rate, and computational complexity, confirming the effectiveness and efficiency of the proposed algorithm. Compared with other eminent algorithms, our proposed algorithm achieves good performance in terms of both hash length and execution time.

This study investigates the enhancement of remote photoplethysmography (rPPG) using a hybrid system that integrates RGB and near-infrared (NIR) cameras. rPPG, a non-contact method for heart rate (HR) measurement, relies on analyzing subtle color changes in facial videos. We compare traditional signal processing methods, such as independent component analysis and principal component analysis, with advanced deep learning models such as DeepPhys and PhysFormer. Traditional methods often struggle with lighting variations and motion artifacts, whereas deep learning approaches demonstrate improved robustness and accuracy. Our hybrid approach leverages the strengths of both RGB and NIR cameras, addressing their individual limitations. We propose two methods, the first is a four-channel method, which overlays RGB and NIR frames to create a four-channel image, and the second is an ensemble method, which applies weights to the tokens extracted from each video and then sums them. These hybrid approaches significantly enhance performance, achieving higher percentage of time with error within 6 BPM (PTE6) scores across various scenarios, including still conditions and motion environments. PTE6 represents the proportion of the entire timeline where the predicted HR is within six beats per minute (BPM) of the actual HR and is a commonly used metric in HR prediction studies. For example, the four-channel method achieved a PTE6 score of 72.89% in still conditions and 56.73% in motion scenarios, outperforming the single-camera setups. Specifically, the RGB camera achieved 70.97% and the NIR camera 67.89% in still conditions, whereas in motion scenarios, the RGB camera scored 53.24% and the NIR camera 37.99%. These findings highlight the potential of combining RGB and NIR cameras to improve the accuracy and reliability of rPPG. This approach opens new avenues for advanced telemedicine applications, providing a robust solution for continuous health monitoring in diverse environments.