Multiscale Representation Research Articles

Batch process soft sensing based on data-stacking multiscale adaptive graph neural network

Abstract Soft sensing technology has found extensive application in predicting key quality variables in batch processes. However, its application in batch process is limited by the uneven batch length, the correlation of data and the difficulty in extracting the dependencies between variables and within variables. To address these issues, we propose a data-stacking multiscale adaptive graph neural network (DSMAGNN) soft sensor model. Firstly, Mutual information (MI) is used to selected quality-related variables, the 3D batch data is converted into a time-delay sequence suitable for input to the soft sensor model through the data stacking strategy, and the underlying time correlation at different time scales is preserved by incorporating the multi-scale pyramid network. Secondly, the dependencies between variables are inferred by the adaptive graph learning module, while the dependencies both within and between variables are modeled by the multi-scale temporal graph neural network. Thirdly, collaborative work across different time scales is further facilitated by the scale fusion module. Finally, the feasibility and effectiveness of the model are verified through experiments in the industrial-scale penicillin fermentation process and hot rolling process. (DSMAGNN) soft sensor model. Firstly, MI is used to selected quality-related variables, and the data stacking strategy is employed to convert 3D batch data into a time-delay sequence suitable for input to the soft sensor model. A multiscale pyramid network is utilized to preserve the underlying time dependence across different time scales. Secondly, since the dependence relationship between variables may vary across different time scales, the adaptive graph learning modules deduce scale-specific variable dependencies without relying on predefined priors. Thirdly, given the multiscale feature representation and the inferred dependency relationships between scale-specific variables, we introduce a multiscale temporal graph neural network to jointly model dependencies within and between variables. Building upon this foundation, the scale fusion module is deployed to facilitate collaboration across different temporal scales and automatically capture the significance of contributing temporal patterns. We validate the feasibility and effectiveness of the proposed model through experiments conducted in industrial-scale penicillin fermentation and hot continuous rolling processes, the results show that the proposed model is superior to several compared models, thus confirming its validity.

A Multi-Hierarchical Complementary Feature Interaction Network for Accelerated Multi-Modal MR Imaging

Magnetic resonance (MR) imaging is widely used in the clinical field due to its non-invasiveness, but the long scanning time is still a bottleneck for its popularization. Using the complementary information between multi-modal imaging to accelerate imaging provides a novel and effective MR fast imaging solution. However, previous technologies mostly use simple fusion methods and fail to fully utilize their potential sharable knowledge. In this study, we introduced a novel multi-hierarchical complementary feature interaction network (MHCFIN) to realize joint reconstruction of multi-modal MR images with undersampled data and thus accelerate multi-modal imaging. Firstly, multiple attention mechanisms are integrated with a dual-branch encoder–decoder network to represent shared features and complementary features of different modalities. In the decoding stage, the multi-modal feature interaction module (MMFIM) acts as a bridge between the two branches, realizing complementary knowledge transfer between different modalities through cross-level fusion. The single-modal feature fusion module (SMFFM) carries out multi-scale feature representation and optimization of the single modality, preserving better anatomical details. Extensive experiments are conducted under different sampling patterns and acceleration factors. The results show that this proposed method achieves obvious improvement compared with existing state-of-the-art reconstruction methods in both visual quality and quantity.

Adaptive multiscale wavelet-guided periodic sparse representation for bearing incipient fault feature extraction

Adaptive multiscale wavelet-guided periodic sparse representation for bearing incipient fault feature extraction

Multilevel identity fine authentication method based on time-frequency domain feature extraction in industrial internet of things system

The cyber physical system is widely applied to industrial Internet of Things system and its security determines the safety of the industrial IoT system. This paper proposes a multilevel fine fingerprint authentication method for the security protection of key operating equipment. Firstly, Because of the acquired signals, the mathematical representation models are established separately, including the self-encoding model on the basis of the multiorder moment features and the multi-scale representation model based on the wavelet. Secondly, the multigranularity features of the time domain and frequency domain are extracted respectively based on the real-time data's self-encoding representation and multiscale representation. Then, the extracted features of different granularities can be built into the image representation based on the time-frequency fusion portrait. Based on the images of different granularities and the legitimate image library, a multiscale matching authentication method is established. Finally, simulation experiments show that the proposed method is effective.

A novel enhancing method for terahertz imaging of integrated circuits flaw detection

This research paper addresses the current gap in understanding the intricate damage types within packaged integrated circuits (ICs). We introduce a novel and multiscale latent low-rank representation (M-LatLRR) methodology, meticulously tailored to enhance the quality of terahertz (THz) IC images and reveal latent features. By integrating THz imaging with our proposed M-LatLRR method, we aim to facilitate the precise identification of damage types within inside the packaged ICs. Firstly, the multiscale Gaussian functions are used to remove the blur and the components are obtained via averaging. Secondly, the image matrices are utilized to extract base matrices and detail matrices. With LatLRR, the multiscale detail matrices are extracted at several representation levels. The final enhanced image is reconstructed by average strategy for dealing with the detail and pre-enhancement parts. The M-LatLRR framework is universal and can be effectively applied to extract multi-level features of packaged IC images. In a comparative analysis, our method demonstrates superior capabilities in determining the failure types of fractured wire bonds, cracks, and delamination of the dielectric layer, outperforming alternative methodologies.

Category-Based Interactive Attention and Perception Fusion Network for Semantic Segmentation of Remote Sensing Images

With the development of CNNs and the application of transformers, the segmentation performance of high-resolution remote sensing image semantic segmentation models has been significantly improved. However, the issue of category imbalance in remote sensing images often leads to the model’s segmentation ability being biased towards categories with more samples, resulting in suboptimal performance for categories with fewer samples. To make the network’s learning and representation capabilities more balanced across different classes, in this paper we propose a category-based interactive attention and perception fusion network (CIAPNet), where the network divides the feature space by category to ensure the fairness of learning and representation for each category. Specifically, the category grouping attention (CGA) module utilizes self-attention to reconstruct the features of each category in a grouped manner, and optimize the foreground–background relationship and its feature representation for each category through the interactive foreground–background relationship optimization (IFBRO) module therein. Additionally, we introduce a detail-aware fusion (DAF) module, which uses shallow detail features to complete the semantic information of deep features. Finally, a multi-scale representation (MSR) module is deployed for each class in the CGA and DAF modules to enhance the description capability of different scale information for each category. Our proposed CIAPNet achieves mIoUs of 54.44%, 85.71%, and 87.88% on the LoveDA urban–rural dataset, and the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam urban datasets, respectively. Compared with current popular methods, our network not only achieves excellent performance but also demonstrates outstanding class balance.

Multiscale implicit frequency selective network for single‐image dehazing

AbstractImage dehazing is aimed to reconstruct a clear latent image from a degraded image affected by haze. Although vision transformers have achieved impressive success in various computer vision tasks, the limitations in scale and quality of available datasets have hindered the transformer effectiveness for image dehazing. Thus, convolutional neural networks (CNNs) remain the mainstream approach for image dehazing, offering robust performance and adaptability. We further explore the potential of CNNs in image dehazing by proposing a multiscale implicit frequency selection network (MIFSN). The proposed MIFSN enhances multiscale representation learning based on U‐shaped networks. As hazy and clear images considerably differ in high‐frequency components, we introduce an implicit frequency selection module to amplify high‐frequency components of features and generate candidate feature maps. Implicit frequency selection attention is then used to emphasize and merge beneficial frequency components. Results from extensive experiments on synthetic and real‐world datasets demonstrate the superior performance of MIFSN for image dehazing.

Rapid identification and early warning of axial compressor stall based on multiscale CNN-SVM-FC model

Early prewarning of compressor stall and surge is crucial to avoid aircraft engine instability, yet it is challenging due to the complex and unstable flow field characterized by multiple modes and multiscale features. To enhance the multi-scale feature representation capability of Convolutional Neural Network-Support Vector Machine (CNN-SVM) algorithm, a novel classifier modelling method combined multiscale windows with CNN-SVM is introduced for stall prewarning in this paper, named Multiscale CNN-SVM-FC. Multiscale detection windows are utilized to adaptively identify various pressure features during the stall process. Additionally, to reduce the false alarm rate, a fuzzy control algorithm is integrated with the temporal accumulation of prediction results from the multi-branch network for joint analysis. A series of test data from a five-stage axial compressor at different operating speeds is used to verify this method. The results indicate that the proposed Multiscale CNN-SVM-FC method enhances the accuracy of classification and reduces the false alarm rate compared to the standard CNN-SVM model, achieving over 99% accuracy in identifying unstable states under various speeds. Compared to three traditional stall prewarning methods, the Multiscale CNN-SVM-FC model provides an average warning signal 164 milliseconds ahead of stall, and reduces the uncertainty associated with threshold selection, which typically relies on engineering experience.

Decoupling visual and identity features for adversarial palm-vein image attack

Palm-vein has been widely used for biometric recognition due to its resistance to theft and forgery. However, with the emergence of adversarial attacks, most existing palm-vein recognition methods are vulnerable to adversarial image attacks, and to the best of our knowledge, there is still no study specifically focusing on palm-vein image attacks. In this paper, we propose an adversarial palm-vein image attack network that generates highly similar adversarial palm-vein images to the original samples, but with altered palm-identities. Unlike most existing generator-oriented methods that directly learn image features via concatenated convolutional layers, our proposed network first maps palm-vein images into multi-scale high-dimensional shallow representation, and then develops attention-based dual-path feature learning modules to extensively exploit diverse palm-vein-specific features. After that, we design visual-consistency and identity-aware loss functions to specially decouple the visual and identity features to reconstruct the adversarial palm-vein images. By doing this, the visual characteristics of palm-vein images can be largely preserved while the identity information is removed in the adversarial palm-vein images, such that high-aggressive adversarial palm-vein samples can be obtained. Extensive white-box and black-box attack experiments conducted on three widely used databases clearly show the effectiveness of the proposed network.

SiGNN: A spike-induced graph neural network for dynamic graph representation learning

In the domain of dynamic graph representation learning (DGRL), capturing the temporal evolution within real-world networks is of paramount importance. Spiking Neural Networks (SNNs), renowned for their temporal dynamics and low-power characteristics, provide an efficient solution for temporal processing in DGRL tasks. However, due to the spike-based information encoding mechanism of SNNs, existing DGRL methods that employ SNNs face significant limitations in their representational capacity. To address this issue, we propose an innovative framework named Spike-induced Graph Neural Network (SiGNN) for learning enhanced spatiotemporal representations on dynamic graphs. Specifically, we achieve a harmonious integration of SNNs and GNNs through a novel Temporal Activation (TA) mechanism. This mechanism enables SiGNN to effectively harness the temporal dynamics of SNNs while circumventing the representational constraints imposed by the binary nature of spikes. Furthermore, leveraging the inherent adaptability of SNNs, SiGNN incorporates an in-depth analysis of evolutionary patterns within graphs across multiple time granularities, thereby facilitating the acquisition of multiscale temporal node representations. Extensive experiments on various real-world dynamic graph datasets demonstrate the superior performance of SiGNN in the node classification task. Our code is publicly available at https://github.com/CharliedoD/SiGNN.

Improved Detection of Multi-Class Bad Traffic Signs Using Ensemble and Test Time Augmentation Based on Yolov5 Models

Various factors such as natural disasters, vandalism, weather, and environmental conditions can affect the physical state of traffic signs. The proposed model aims to improve detection of traffic signs affected by partial occlusion as a result of overgrown vegetation, displaced signs (those knocked down, bent), perforated signs (those damaged with holes), faded signs (color degradation), rusted signs (corroded surface), and de-faced signs (placing graffiti, etc., by vandals). This research aims to improve the detection of bad traffic signs using three approaches. In the first approach, Spiral Pooling Pyramid-Fast (SPPF) and C3TR modules are introduced to the architecture of Yolov5 models. SPPF helps provide a multi-scale representation of the input feature map by pooling at different scales, which is useful in improving the quality of feature maps and detecting bad traffic signs of various sizes and perspectives. The C3TR module uses convolutional layers to enhance local feature extraction and transformers to boost understanding of the global context. Secondly, we use predictions of Yolov5 as base models to implement a mean ensemble to improve performance. Thirdly, test time augmentation (TTA) is applied at test time by using scaling and flipping to improve accuracy. Some signs are generated using stable diffusion techniques to augment certain classes. We test the proposed models on the CCTSDB2021, TT100K, GTSDB, and GTSRD datasets to ensure generalization and use k-fold cross-validation to further evaluate the performance of the models. The proposed models outperform other state-of-the-art models in comparison.

MuSic-UDF: Learning Multi-Scale dynamic grid representation for high-fidelity surface reconstruction from point clouds

Surface reconstruction for point clouds is a central task in 3D modeling. Recently, the attractive approaches solve this problem by learning neural implicit representations, e.g., unsigned distance functions (UDFs), from point clouds, which have achieved good performance. However, the existing UDF-based methods still struggle to recover the local geometrical details. One of the difficulties arises from the used inflexible representations, which is hard to capture the local high-fidelity geometry details. In this paper, we propose a novel neural implicit representation, named MuSic-UDF, which leverages Multi-Scale dynamic grids for high-fidelity and flexible surface reconstruction from raw point clouds with arbitrary typologies. Specifically, we initialize a hierarchical voxel grid where each grid point stores a learnable 3D coordinate. Then, we optimize these grids such that different levels of geometry structures can be captured adaptively. To further explore the geometry details, we introduce a frequency encoding strategy to hierarchically encode these coordinates. MuSic-UDF does not require any supervisions like ground truth distance values or point normals. We conduct comprehensive experiments under widely-used benchmarks, where the results demonstrate the superior performance of our proposed method compared to the state-of-the-art methods.

BreasTDLUSeg: A coarse-to-fine framework for segmentation of breast terminal duct lobular units on histopathological whole-slide images

Automatic segmentation of breast terminal duct lobular units (TDLUs) on histopathological whole-slide images (WSIs) is crucial for the quantitative evaluation of TDLUs in the diagnostic and prognostic analysis of breast cancer. However, TDLU segmentation remains a great challenge due to its highly heterogeneous sizes, structures, and morphologies as well as the small areas on WSIs. In this study, we propose BreasTDLUSeg, an efficient coarse-to-fine two-stage framework based on multi-scale attention to achieve localization and precise segmentation of TDLUs on hematoxylin and eosin (H&E)-stained WSIs. BreasTDLUSeg consists of two networks: a superpatch-based patch-level classification network (SPPC-Net) and a patch-based pixel-level segmentation network (PPS-Net). SPPC-Net takes a superpatch as input and adopts a sub-region classification head to classify each patch within the superpatch as TDLU positive or negative. PPS-Net takes the TDLU positive patches derived from SPPC-Net as input. PPS-Net deploys a multi-scale CNN-Transformer as an encoder to learn enhanced multi-scale morphological representations and an upsampler to generate pixel-wise segmentation masks for the TDLU positive patches. We also constructed two breast cancer TDLU datasets containing a total of 530 superpatch images with patch-level annotations and 2,322 patch images with pixel-level annotations to enable the development of TDLU segmentation methods. Experiments on the two datasets demonstrate that BreasTDLUSeg outperforms other state-of-the-art methods with the highest Dice similarity coefficients of 79.97% and 92.93%, respectively. The proposed method shows great potential to assist pathologists in the pathological analysis of breast cancer. An open-source implementation of our approach can be found at https://github.com/Dian-kai/BreasTDLUSeg.

PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks.

Multiscale features are of great importance in modern convolutional neural networks, showing consistent performance gains on numerous vision tasks. Therefore, many plug-and-play blocks are introduced to upgrade existing convolutional neural networks for stronger multiscale representation ability. However, the design of plug-and-play blocks is getting more and more complex, and these manually designed blocks are not optimal. In this work, we propose PP-NAS to develop plug-and-play blocks based on neural architecture search (NAS). Specifically, we design a new search space PPConv and develop a search algorithm consisting of one-level optimization, zero-one loss, and connection existence loss. PP-NAS minimizes the optimization gap between super-net and subarchitectures and can achieve good performance even without retraining. Extensive experiments on image classification, object detection, and semantic segmentation verify the superiority of PP-NAS over state-of-the-art CNNs (e.g., ResNet, ResNeXt, and Res2Net). Our code is available at https://github.com/ainieli/PP-NAS.

Neural populations in the language network differ in the size of their temporal receptive windows.

Despite long knowing what brain areas support language comprehension, our knowledge of the neural computations that these frontal and temporal regions implement remains limited. One important unresolved question concerns functional differences among the neural populations that comprise the language network. Here we leveraged the high spatiotemporal resolution of human intracranial recordings (n = 22) to examine responses to sentences and linguistically degraded conditions. We discovered three response profiles that differ in their temporal dynamics. These profiles appear to reflect different temporal receptive windows, with average windows of about 1, 4 and 6 words, respectively. Neural populations exhibiting these profiles are interleaved across the language network, which suggests that all language regions have direct access to distinct, multiscale representations of linguistic input-a property that may be critical for the efficiency and robustness of language processing.

Siamese Tracking Network with Multi-attention Mechanism

Object trackers based on Siamese networks view tracking as a similarity-matching process. However, the correlation operation operates as a local linear matching process, limiting the tracker’s ability to capture the intricate nonlinear relationship between the template and search region branches. Moreover, most trackers don’t update the template and often use the first frame of an image as the initial template, which will easily lead to poor tracking performance of the algorithm when facing instances of deformation, scale variation, and occlusion of the tracking target. To this end, we propose a Simases tracking network with a multi-attention mechanism, including a template branch and a search branch. To adapt to changes in target appearance, we integrate dynamic templates and multi-attention mechanisms in the template branch to obtain more effective feature representation by fusing the features of initial templates and dynamic templates. To enhance the robustness of the tracking model, we utilize a multi-attention mechanism in the search branch that shares weights with the template branch to obtain multi-scale feature representation by fusing search region features at different scales. In addition, we design a lightweight and simple feature fusion mechanism, in which the Transformer encoder structure is utilized to fuse the information of the template area and search area, and the dynamic template is updated online based on confidence. Experimental results on publicly tracking datasets show that the proposed method achieves competitive results compared to several state-of-the-art trackers.

Cartographic Generalization of Islands Using Remote Sensing Images for Multiscale Representation

The multi-scale representation of remote sensing images provides various levels of image information crucial for decision-making in GIS applications and plays a significant role in information processing, data analysis, and geographic modeling. Traditional methods for multi-scale representation of remote sensing images often struggle to simplify local details of individual targets while preserving the overall characteristics of target groups. These methods also encounter issues such as transitional texture distortion and rough final boundaries. This paper proposes a novel multi-scale representation method for remote sensing images based on computer vision techniques, which effectively maintains the overall characteristics of target groups. Initially, the K-means algorithm is employed to distinguish between islands and oceans. Subsequently, a superpixel segmentation algorithm is used to aggregate island groups and simplify the generated boundaries. Finally, texture synthesis and transfer are applied based on the original image to produce the aggregated island images. Evaluation metrics demonstrate that this method can generate multi-scale aggregated images of islands, effectively eliminate redundant information, and produce smooth boundaries.

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

MH2AFormer: An Efficient Multiscale Hierarchical Hybrid Attention With a Transformer for Bladder Wall and Tumor Segmentation.

Achieving accurate bladder wall and tumor segmentation from MRI is critical for diagnosing and treating bladder cancer. However, automated segmentation remains challenging due to factors such as comparable density distributions, intricate tumor morphologies, and unclear boundaries. Considering the attributes of bladder MRI images, we propose an efficient multiscale hierarchical hybrid attention with a transformer (MH2AFormer) for bladder cancer and wall segmentation. Specifically, a multiscale hybrid attention and transformer (MHAT) module in the encoder is designed to adaptively extract and aggregate multiscale hybrid feature representations from the input image. In the decoder stage, we devise a multiscale hybrid attention (MHA) module to generate high-quality segmentation results from multiscale hybrid features. Combining these modules enhances the feature representation and guides the model to focus on tumor and wall regions, which helps to solve bladder image segmentation challenges. Moreover, MHAT utilizes the Fast Fourier Transformer with a large kernel (e.g., 224 × 224) to model global feature relationships while reducing computational complexity in the encoding stage. The model performance was evaluated on two datasets. As a result, the model achieves relatively best results regarding the intersection over union (IoU) and dice similarity coefficient (DSC) on both datasets (Dataset A: IoU = 80.26%, DSC = 88.20%; Dataset B: IoU = 89.74%, DSC = 94.48%). These advantageous outcomes substantiate the practical utility of our approach, highlighting its potential to alleviate the workload of radiologists when applied in clinical settings.

Snowdrift‐Permitting Simulations of Seasonal Snowpack Processes Over Large Mountain Extents

AbstractThe melt of seasonal snowpack in mountain regions provides downstream river basins with a critical supply of freshwater. Snowdrift‐permitting models have been proposed as a way to accurately simulate snowpack heterogeneity that stems from differences in energy inputs, over winter redistribution, sublimation, melt, and variations in precipitation. However, these spatial scales can be computationally intractable for large extents. In this work, the multiscale Canadian Hydrological Model (CHM) was applied to simulate snowpacks at snowdrift‐permitting scales (≈50 m) across the Canadian Cordillera and adjacent regions (1.37 million km2) forced by downscaled atmospheric data. The use of a multiscale land surface representation resulted in a reduction of computational elements of 98% while preserving land‐surface heterogeneity. CHM includes complex terrain windflow and radiative transfer calculations, lapses temperature, humidity, and precipitation with elevation, redistributes snow by avalanching, wind transport and forest canopy interception and calculates the energetics of canopy and surface snowpacks. Model outputs were compared to a set of multiscale observations including snow‐covered area (SCA) from Sentinel and Landsat imagery, snow depth from uncrewed aerial system lidar, and point surface observations of depth and density. Including snow redistribution and sublimation processes improved the summer SCA r2 from 0.7 to 0.9. At larger scales, inclusion of snow redistribution processes delayed full snowpack ablation by an average of 33 days, demonstrating process emergence with scale. These simulations show how multiscale modeling can improve snowpack predictions to support prediction of water supply, droughts, and floods.