High-level Information Research Articles

Infrared Small Target Detection Algorithm Based on Improved Dense Nested U-Net Network

Infrared weak and small target detection technology has attracted much attention in recent years and is crucial in the application fields of early warning, monitoring, medical diagnostics, and anti-UAV detection.With the advancement of deep learning, CNN-based methods have achieved promising results in general-purpose target detection due to their powerful modeling capabilities; however, CNN-based methods cannot be directly applied to infrared small targets due to the disappearance of deep targets caused by multiple downsampling operations. Aiming at these problems, we proposed an improved dense nesting and attention infrared small target detection method based on U-Net called IDNA-UNet. A dense nested interaction module (DNIM) is designed as a feature extraction module to achieve level-by-level feature fusion and retain small targets’ features and detailed positioning information. To integrate low-level features into deeper high-level features, we designed a bottom-up feature pyramid fusion module, which can further retain high-level semantic information and target detail information. In addition, a more suitable scale and position sensitive (SLS) loss is applied to each prediction scale to help the detector locate the target more accurately and distinguish different scales of the target. With our IDNA-UNet, the contextual information of small targets can be well incorporated and fully exploited by repetitive fusion and enhancement. Compared with existing methods, IDNA-UNet has achieved significant advantages in the intersection over union (IoU), detection probability (Pd), and false alarm rate (Fa) of infrared small target detection.

The Role of Deep Learning in Medical Image Inpainting: A Systematic Review

Image inpainting is a crucial technique in computer vision, particularly for reconstructing corrupted images. In medical imaging, it addresses issues from instrumental errors, artifacts, or human factors. The development of deep learning techniques has revolutionized image inpainting, allowing for the generation of high-level semantic information to ensure structural and textural consistency in restored images. This paper presents a comprehensive review of 53 studies on deep image inpainting in medical imaging, analyzing its evolution, impact, and limitations. The findings highlight the significance of deep image inpainting in artifact removal and enhancing the performance of multi-task approaches by localizing and inpainting regions of interest. Furthermore, the study identifies magnetic resonance imaging and computed tomography as the predominant modalities and highlights generative adversarial networks and U-Net as preferred architectures. Future research directions include the development of blind inpainting techniques, the exploration of techniques suitable for 3D/4D images, multiple artifacts, and multi-task applications, and the improvement of architectures.

Concrete sleeper crack detection based on image saliency

Abstract Concrete sleepers are crucial infrastructure in railway construction and operation. Sleeper cracks may cause structural damage or functional failure of sleepers, thereby endangering railway operation safety. A third-order residual crack saliency detection network (R3Net CSD) model was constructed based on ConvNeXt for detecting concrete sleeper cracks in track inspection images, which is used to accurately extract the saliency features of rail sleeper crack images. The model first uses ConNeXt to form the backbone network and Block module to form the first residual, followed by a spatial pyramid (DCSP) module based on the anti residual structure and dilated convolution group to form the second residual, and finally uses the saliency residual (Att-R) module to form the third residual. At the same time, combined with a multi-level joint loss function supervision strategy for supervised learning, the model fully utilizes the low-level edge information and high-level semantic information of cracks. Using the images collected by the hand pushed rail inspection vehicle, a dataset of rail sleeper cracks was manually constructed and the model was experimentally validated. The experimental results show that this model achieved the best results in multiple indicators, such as an average absolute error MAE of 0.0082, an average accuracy AP of 0.9764, and an AUC value of 0.995. By combining the calculation method of crack geometric parameters, quantitative indicators of crack length and width were obtained, with a detection accuracy of 0.001mm. The research provides a new solution for the detection of cracks in concrete sleepers and the measurement of geometric parameters.

Crowd counting in domain generalization based on multi-scale attention and hierarchy level enhancement

In order to solve the problem of weak single domain generalization ability in existing crowd counting methods, this study proposes a new crowd counting framework called Multi-scale Attention and Hierarchy level Enhancement (MAHE). Firstly, the model can focus on both the detailed features and the macro information of structural position changes through the fusion of channel attention and spatial attention. Secondly, the addition of multi-head attention feature module facilitates the model’s capacity to effectively capture complex dependency relationships between sequence elements. In addition, the three-stage encoding and decoding processing mode enables the model to effectively represent crowd density information. Finally, the fusion of multi-scale features derived from different receptive fields is further enhanced through multi-scale hierarchy level feature fusion, thereby enabling the model to learn high-level semantic information and low-level multi-scale visual field feature information. This method enhances the model’s capacity to capture key feature information, even in highly differentiated datasets, thereby improving the model’s generalization ability on a single domain. The model has demonstrated strong generalization capabilities through extensive experiments on different datasets. This study not only improves the accuracy of crowd counting, but also introduces a new research approach for single domain generalization of crowd counting.

Multi-Scale Contrastive Learning with Hierarchical Knowledge Synergy for Visible-Infrared Person Re-Identification.

Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality retrieval task to match a person across different spectral camera views. Most existing works focus on learning shared feature representations from the final embedding space of advanced networks to alleviate modality differences between visible and infrared images. However, exclusively relying on high-level semantic information from the network's final layers can restrict shared feature representations and overlook the benefits of low-level details. Different from these methods, we propose a multi-scale contrastive learning network (MCLNet) with hierarchical knowledge synergy for VI-ReID. MCLNet is a novel two-stream contrastive deep supervision framework designed to train low-level details and high-level semantic representations simultaneously. MCLNet utilizes supervised contrastive learning (SCL) at each intermediate layer to strengthen visual representations and enhance cross-modality feature learning. Furthermore, a hierarchical knowledge synergy (HKS) strategy for pairwise knowledge matching promotes explicit information interaction across multi-scale features and improves information consistency. Extensive experiments on three benchmarks demonstrate the effectiveness of MCLNet.

Physics-informed geometric operators to support surrogate, dimension reduction and generative models for engineering design

In this work, we propose a set of physics-informed geometric operators (GOs) to enrich the geometric data provided for training surrogate/discriminative models, dimension reduction, and generative models, typically employed for performance prediction, dimension reduction, and creating data-driven parameterisations, respectively. However, as both the input and output streams of these models consist of low-level shape representations, they often fail to capture shape characteristics essential for performance analyses. Therefore, the proposed GOs exploit the differential and integral properties of shapes — accessed through Fourier descriptors, curvature integrals, geometric moments, and their invariants — to infuse high-level intrinsic geometric information and physics into the feature vector used for training, even when employing simple model architectures or low-level parametric descriptions. We showed that for surrogate modelling, along with the inclusion of the notion of physics, GOs enact regularisation to reduce over-fitting and enhance generalisation to new, unseen designs. Furthermore, through extensive experimentation, we demonstrate that for dimension reduction and generative models, incorporating the proposed GOs enriches the training data with compact global and local geometric features. This significantly enhances the quality of the resulting latent space, thereby facilitating the generation of valid and diverse designs. Lastly, we also show that GOs can enable learning parametric sensitivities to a great extent. Consequently, these enhancements accelerate the convergence rate of shape optimisers towards optimal solutions.

Development of Women’s Shirt Style Recognition Model (WSSRM) based on a ResNet and Transformer integrated method

The rise of e-commerce has brought women’s shirt images to the forefront, showcasing complex styles and diverse features. This presents a challenge for consumers in finding their preferred styles. Deep learning, known for its speed and superior retrieval performance, offers a solution by learning image features through multi-layer neural networks and extracting high-level semantic information, making it highly effective for recognizing and classifying clothing styles. The ResNet model excels in extracting detailed pixel information, but has limitations: (1) Jump connections in ResNet, which add input directly to the output in the residual block, can cause feature distortion, especially when input and output sizes do not match; (2) Despite many network layers, the effective receptive field of ResNet is smaller than theoretically expected. To address these limitations, this paper introduces the Transformer model into ResNet, proposing a new method for recognizing and classifying women’s shirt styles based on local detail feature extraction. The Transformer’s attention mechanism enhances the model’s ability to focus on important features and suppress less relevant ones, improving the accuracy of local detail feature extraction. This study examines six typical women’s shirt style features, applying the improved ResNet to their recognition and classification, resulting in a highly accurate and reliable model. This theoretical and practical advancement enhances the recognition of detailed features in women’s shirts, significantly contributing to the development of intelligent clothing design.

A spatiotemporal style transfer algorithm for dynamic visual stimulus generation.

Understanding how visual information is encoded in biological and artificial systems often requires the generation of appropriate stimuli to test specific hypotheses, but available methods for video generation are scarce. Here we introduce the spatiotemporal style transfer (STST) algorithm, a dynamic visual stimulus generation framework that allows the manipulation and synthesis of video stimuli for vision research. We show how stimuli can be generated that match the low-level spatiotemporal features of their natural counterparts, but lack their high-level semantic features, providing a useful tool to study object recognition. We used these stimuli to probe PredNet, a predictive coding deep network, and found that its next-frame predictions were not disrupted by the omission of high-level information, with human observers also confirming the preservation of low-level features and lack of high-level information in the generated stimuli. We also introduce a procedure for the independent spatiotemporal factorization of dynamic stimuli. Testing such factorized stimuli on humans and deep vision models suggests a spatial bias in how humans and deep vision models encode dynamic visual information. These results showcase potential applications of the STST algorithm as a versatile tool for dynamic stimulus generation in vision science.

CMFNet: a cross-dimensional modal fusion network for accurate vessel segmentation based on OCTA data.

Optical coherence tomography angiography (OCTA) is a novel non-invasive retinal vessel imaging technique that can display high-resolution 3D vessel structures. The quantitative analysis of retinal vessel morphology plays an important role in the automatic screening and diagnosis of fundus diseases. The existing segmentation methods struggle to effectively use the 3D volume data and 2D projection maps of OCTA images simultaneously, which leads to problems such as discontinuous microvessel segmentation results and deviation of morphological estimation. To enhance diagnostic support for fundus diseases, we propose a cross-dimensional modal fusion network (CMFNet) using both 3D volume data and 2D projection maps for accurate OCTA vessel segmentation. Firstly, we use different encoders to generate 2D projection features and 3D volume data features from projection maps and volume data, respectively. Secondly, we design an attentional cross-feature projection learning module to purify 3D volume data features and learn its projection features along the depth direction. Then, we develop a cross-dimensional hierarchical fusion module to effectively fuse coded features learned from the volume data and projection maps. In addition, we extract high-level semantic weight information and map it to the cross-dimensional hierarchical fusion process to enhance fusion performance. To validate the efficacy of our proposed method, we conducted experimental evaluations using the publicly available dataset: OCTA-500. The experimental results show that our method achieves state-of-the-art performance.

High Defect Tolerance Breaking the Design Limitation of Full-Spectrum Multimodal Luminescence Materials.

With the development of optical anti-counterfeiting and the increasing demand for high-level information encryption, multimodal luminescence (MML) materials attract much attention. However, the discovery of these multifunctional materials is very accidental, and the versatile host suitable for developing such materials remains unclear. Here, a grossite-type fast ionic conductor CaGa4O7, characterized by layered and tunnel structure with excellent defect tolerance, is found to meet the needs of various luminescent processes. Almost all luminescent modes, including down/up-conversion luminescence (DCL/UCL), long persistent luminescence (LPL), mechanoluminescence (ML), and X-ray excited optical luminescence (XEOL), are realized in this single host. Full-spectrum (from violet to near-infrared) photoluminescence and ML as well as multicolor XEOL are achieved by simply changing the doped luminescent center. A series of anti-counterfeiting devices, including the quasi-dynamic display of famous paintings, digital information encryption, and multi-color handwritten signatures, are designed to show the encryption of information in temporal and spatial dimensions. This study clarifies the importance of defect tolerance of the host for the development of MML materials, and provides a unique insight into the cross-field applications of special functional materials, which is a new strategy to accelerate the development of novel MML materials.

RT-DETR-Tea: A Multi-Species Tea Bud Detection Model for Unstructured Environments

Accurate bud detection is a prerequisite for automatic tea picking and yield statistics; however, current research suffers from missed detection due to the variety of singleness and false detection under complex backgrounds. Traditional target detection models are mainly based on CNN, but CNN can only achieve the extraction of local feature information, which is a lack of advantages for the accurate identification of targets in complex environments, and Transformer can be a good solution to the problem. Therefore, based on a multi-variety tea bud dataset, this study proposes RT-DETR-Tea, an improved object detection model under the real-time detection Transformer (RT-DETR) framework. This model uses cascaded group attention to replace the multi-head self-attention (MHSA) mechanism in the attention-based intra-scale feature interaction (AIFI) module, effectively optimizing deep features and enriching the semantic information of features. The original cross-scale feature-fusion module (CCFM) mechanism is improved to establish the gather-and-distribute-Tea (GD-Tea) mechanism for multi-level feature fusion, which can effectively fuse low-level and high-level semantic information and large and small tea bud features in natural environments. The submodule of DilatedReparamBlock in UniRepLKNet was employed to improve RepC3 to achieve an efficient fusion of tea bud feature information and ensure the accuracy of the detection head. Ablation experiments show that the precision and mean average precision of the proposed RT-DETR-Tea model are 96.1% and 79.7%, respectively, which are increased by 5.2% and 2.4% compared to those of the original model, indicating the model’s effectiveness. The model also shows good detection performance on the newly constructed tea bud dataset. Compared with other detection algorithms, the improved RT-DETR-Tea model demonstrates superior tea bud detection performance, providing effective technical support for smart tea garden management and production.

A Novel Temporal Fusion Channel Network with Multi-Channel Hybrid Attention for the Remaining Useful Life Prediction of Rolling Bearings

The remaining useful life (RUL) prediction of rolling bearings is crucial for optimizing maintenance schedules, reducing downtime, and extending machinery lifespan. However, existing multi-channel feature fusion methods do not fully capture the correlations between channels and time points in multi-dimensional sensor data. To address the above problems, this paper proposes a multi-channel feature fusion algorithm based on a hybrid attention mechanism and temporal convolutional networks (TCNs), called MCHA-TFCN. The model employs a dual-channel hybrid attention mechanism, integrating self-attention and channel attention to extract spatiotemporal features from multi-channel inputs. It uses causal dilated convolutions in TCNs to capture long-term dependencies and incorporates enhanced residual structures for global feature fusion, effectively extracting high-level spatiotemporal degradation information. The experimental results on the PHM2012 dataset show that MCHA-TFCN achieves excellent performance, with an average Root-Mean-Square Error (RMSE) of 0.091, significantly outperforming existing methods like the DANN and CNN-LSTM.

Ensemble representation of animacy could be based on mid-level visual features.

Studies suggest that mid-level features could underlie object animacy perception. In the current research, we tested whether ensemble animacy perception is based on high- or mid-level features. We used five types of images of animals and inanimate objects: color, grayscale, silhouettes, texforms - unrecognizable images that preserve mid-level texture and shape information - and scrambled images. In the series of Experiments 1, we asked participants to evaluate the animacy of single images and sets of eight images using a 10-point scale. In the series of Experiments 2, participants were shown two sets of eight images and had to choose a more animate one in the two-alternative forced-choice (2AFC) task. We found that in both paradigms, observers could report the mean animacy of the set of texform images without direct access to information about high-level features. Thus, ensemble animacy could be extracted only based on mid-level features such as shape and texture without access to more high-level information.

Optimizing Urban Traffic Flow through Advanced Tensor Analysis and Multilinear Algebra: A Computational Approach to Enhancing Smart City Dynamics

Growing traffic congestion is a worldwide problem that collides against the aims of environmental sustainability, economic productivity, and the quality of life in cities. This research proposes a new computational framework for traffic management that integrates advanced tensor analysis and methods from multilinear algebra. We have developed and validated a new predictive model that greatly improves the optimization of traffic flows by synthesizing the naturally complex multi-dimensional traffic data analysis. Our results demonstrate that, compared with existing systems, the proposed approach results in higher accuracy of prediction, much improved computational efficiency, and provides scalable and adaptable solutions for application in a wide range of urban habitats. Such research may push the boundaries further on the smart city infrastructures to provide a very well-founded mathematical framework for the dynamics of improved urban mobility through high-level data-oriented information.

A temporal-spectral graph convolutional neural network model for EEG emotion recognition within and across subjects

EEG-based emotion recognition uses high-level information from neural activities to predict emotional responses in subjects. However, this information is sparsely distributed in frequency, time, and spatial domains and varied across subjects. To address these challenges in emotion recognition, we propose a novel neural network model named Temporal-Spectral Graph Convolutional Network (TSGCN). To capture high-level information distributed in time, spatial, and frequency domains, TSGCN considers both neural oscillation changes in different time windows and topological structures between different brain regions. Specifically, a Minimum Category Confusion (MCC) loss is used in TSGCN to reduce the inconsistencies between subjective ratings and predefined labels. In addition, to improve the generalization of TSGCN on cross-subject variation, we propose Deep and Shallow feature Dynamic Adversarial Learning (DSDAL) to calculate the distance between the source domain and the target domain. Extensive experiments were conducted on public datasets to demonstrate that TSGCN outperforms state-of-the-art methods in EEG-based emotion recognition. Ablation studies show that the mixed neural networks and our proposed methods in TSGCN significantly contribute to its high performance and robustness. Detailed investigations further provide the effectiveness of TSGCN in addressing the challenges in emotion recognition.

Pixel-Centric Context Perception Network for Camouflaged Object Detection.

Camouflaged object detection (COD) aims to identify object pixels visually embedded in the background environment. Existing deep learning methods fail to utilize the context information around different pixels adequately and efficiently. In order to solve this problem, a novel pixel-centric context perception network (PCPNet) is proposed, the core of which is to customize the personalized context of each pixel based on the automatic estimation of its surroundings. Specifically, PCPNet first employs an elegant encoder equipped with the designed vital component generation (VCG) module to obtain a set of compact features rich in low-level spatial and high-level semantic information across multiple subspaces. Then, we present a parameter-free pixel importance estimation (PIE) function based on multiwindow information fusion. Object pixels with complex backgrounds will be assigned with higher PIE values. Subsequently, PIE is utilized to regularize the optimization loss. In this way, the network can pay more attention to those pixels with higher PIE values in the decoding stage. Finally, a local continuity refinement module (LCRM) is used to refine the detection results. Extensive experiments on four COD benchmarks, five salient object detection (SOD) benchmarks, and five polyp segmentation benchmarks demonstrate the superiority of PCPNet with respect to other state-of-the-art methods.

Hierarchically Recognizing Vector Graphics and A New Chart-Based Vector Graphics Dataset.

The conventional approach to image recognition has been based on raster graphics, which can suffer from aliasing and information loss when scaled up or down. In this paper, we propose a novel approach that leverages the benefits of vector graphics for object localization and classification. Our method, called YOLaT (You Only Look at Text), takes the textual document of vector graphics as input, rather than rendering it into pixels. YOLaT builds multi-graphs to model the structural and spatial information in vector graphics and utilizes a dual-stream graph neural network (GNN) to detect objects from the graph. However, for real-world vector graphics, YOLaT only models in flat GNN with vertexes as nodes ignore higher-level information of vector data. Therefore, we propose YOLaT++ to learn Multi-level Abstraction Feature Learning from a new perspective: Primitive Shapes to Curves and Points. On the other hand, given few public datasets focus on vector graphics, data-driven learning cannot exert its full power on this format. We provide a large-scale and challenging dataset for Chart-based Vector Graphics Detection and Chart Understanding, termed VG-DCU, with vector graphics, raster graphics, annotations, and raw data drawn for creating these vector charts. Experiments show that the YOLaT series outperforms both vector graphics and raster graphics-based object detection methods on both subsets of VG-DCU in terms of both accuracy and efficiency, showcasing the potential of vector graphics for image recognition tasks.

Identifying the onset and decay of quasi-stationary families of almost-invariant sets with an application to atmospheric blocking events.

Macroscopic features of dynamical systems such as almost-invariant sets and coherent sets provide crucial high-level information on how the dynamics organizes phase space. We introduce a method to identify time-parameterized families of almost-invariant sets in time-dependent dynamical systems, as well as the families' emergence and disappearance. In contrast to coherent sets, which may freely move about in phase space over time, our technique focuses on families of metastable sets that are quasi-stationary in space. Our straightforward approach extends successful transfer operator methods for almost-invariant sets to time-dependent dynamics and utilizes the Ulam scheme for the generator of the transfer operator on a time-expanded domain. The new methodology is illustrated with an idealized fluid flow and with atmospheric velocity data. We identify atmospheric blocking events in the 2003 European heatwave and compare our technique to existing geophysical methods of blocking diagnosis.

A Lightweight Deep Learning Network with an Optimized Attention Module for Aluminum Surface Defect Detection.

Aluminum is extensively utilized in the aerospace, aviation, automotive, and other industries. The presence of surface defects on aluminum has a significant impact on product quality. However, traditional detection methods fail to meet the efficiency and accuracy requirements of industrial practices. In this study, we propose an innovative aluminum surface defect detection method based on an optimized two-stage Faster R-CNN network. A 2D camera serves as the image sensor, capturing high-resolution images in real time. Optimized lighting and focus ensure that defect features are clearly visible. After preprocessing, the images are fed into a deep learning network incorporated with a multi-scale feature pyramid structure, which effectively enhances defect recognition accuracy by integrating high-level semantic information with location details. Additionally, we introduced an optimized Convolutional Block Attention Module (CBAM) to further enhance network efficiency. Furthermore, we employed the genetic K-means algorithm to optimize prior region selection, and a lightweight Ghost model to reduce network complexity by 14.3%, demonstrating the superior performance of the Ghost model in terms of loss function optimization during training and validation as well as in terms of detection accuracy, speed, and stability. The network was trained on a dataset of 3200 images captured by the image sensor, split in an 8:1:1 ratio for training, validation, and testing, respectively. The experimental results show a mean Average Precision (mAP) of 94.25%, with individual Average Precision (AP) values exceeding 80%, meeting industrial standards for defect detection.

Melodic contour supersedes short-term statistical learning in expressive accentuation.

Sensory systems are permanently bombarded with complex stimuli. Cognitive processing of such complex stimuli may be facilitated by accentuation of important elements. In the case of music listening, alteration of some surface features -such as volume and duration- may facilitate the cognitive processing of otherwise high-level information, such as melody and harmony. Hence, musical accents are often aligned with intrinsically salient elements in the stimuli, such as highly unexpected notes. We developed a novel listening paradigm based on an artificial Markov-chain melodic grammar to probe the hypothesis that listeners prefer structurally salient events to be consistent with salient surface properties such as musical accents. We manipulated two types of structural saliency: one driven by Gestalt principles (a note at the peak of a melodic contour) and one driven by statistical learning (a note with high surprisal, or information content [IC], as defined by the artificial melodic grammar). Results suggest that for all listeners, the aesthetic preferences in terms of surface properties are well predicted by Gestalt principles of melodic shape. In contrast, despite demonstrating good knowledge of novel statistical properties of the melodies, participants did not demonstrate a preference for accentuation of high-IC notes. This work is a first step in elucidating the interplay between intrinsic, Gestalt-like and acquired, statistical properties of melodies in the development of expressive musical properties, with a focus on the appreciation of dynamic accents (i.e. a transient increase in volume). Our results shed light on the implementation of domain-general and domain-specific principles of information processing during music listening.