Weight Map Research Articles

Accuracy Improvement With Weight Mapping Strategy and Output Transformation for STT-MRAM-Based Computing-in-Memory

Accuracy Improvement With Weight Mapping Strategy and Output Transformation for STT-MRAM-Based Computing-in-Memory

AI-enhanced chemical paradigm: From molecular graphs to accurate prediction and mechanism

The development of accurate and interpretable models for predicting reaction constants of organic compounds with hydroxyl radicals is vital for advancing quantitative structure-activity relationships (QSAR) in pollutant degradation. Methods like molecular descriptors, molecular fingerprinting, and group contribution methods have limitations, as traditional machine learning struggles to capture all intramolecular information simultaneously. To address this, we established an integrated graph neural network (GNN) with approximately 12 million learnable parameters. GNN represents atoms as nodes and chemical bonds as edges, thus transforming molecules into a graph structures, effectively capturing microscopic properties while depicting atom connectivity in non-Euclidean space. Our datasets comprise 1401 pollutants to develop an integrated GNN model with Bayesian optimization, the model achieves root mean square errors of 0.165, 0.172, and 0.189 on the training, validation, and test datasets, respectively. Furthermore, we assess molecular structure similarity using molecular fingerprint to enhance the model's applicability. Afterwards, we propose a gradient weight mapping method for model explainability, uncovering the key functional groups in chemical reactions in artificial intelligence perspective, which would boost chemistry through artificial intelligence extreme arithmetic power.

TransCNNLoc: End-to-end pixel-level learning for 2D-to-3D pose estimation in dynamic indoor scenes

Accurate localization in GPS-denied environments has always been a core issue in computer vision and robotics research. In indoor environments, vision-based localization methods are susceptible to changes in lighting conditions, viewing angles, and environmental factors, resulting in localization failures or limited generalization capabilities. In this paper, we propose the TransCNNLoc framework, which consists of an encoding–decoding network designed to learn more robust image features for camera pose estimation. In the image feature encoding stage, CNN and Swin Transformer are integrated to construct the image feature encoding module, enabling the network to fully extract global context and local features from images. In the decoding stage, multi-level image features are decoded through cross-layer connections while computing per-pixel feature weight maps. To enhance the framework’s robustness to dynamic objects, a dynamic object recognition network is introduced to optimize the feature weights. Finally, a multi-level iterative optimization from coarse to fine levels is performed to recover six degrees of freedom camera pose. Experiments were conducted on the publicly available 7scenes dataset as well as a dataset collected under changing lighting conditions and dynamic scenes for accuracy validation and analysis. The experimental results demonstrate that the proposed TransCNNLoc framework exhibits superior adaptability to dynamic scenes and lighting changes. In the context of static environments within publicly available datasets, the localization technique introduced in this study attains a maximal precision of up to 5 centimeters, consistently achieving superior outcomes across a majority of the scenarios. Under the conditions of dynamic scenes and fluctuating illumination, this approach demonstrates an enhanced precision capability, reaching up to 3 centimeters. This represents a substantial refinement from the decimeter scale to a centimeter scale in precision, marking a significant advancement over the existing state-of-the-art (SOTA) algorithms. The open-source repository for the method proposed in this paper can be found at the following URL: github.com/Geelooo/TransCNNloc.

GIS based Site Suitability Analysis for Wastewater Treatment Plant in Upper Ponnaiyar Watershed, South India

The management of water resources is a major challenge for most of the dry land areas, as the demand for water are increasing and the quality are being compromised due to a number of natural and economic factors. If the proper location for the treatment plant is not selected, then it may lead to soil degradation and groundwater pollution. This study was conducted by using GIS techniques for selecting suitable wastewater treatment plant sites. There are seven parameters considered in the analysis consists of land use/land cover, elevation, road proximity, a slope of the ground, drainage density, geology, and soil. The weighted index overlay analyses of the final map with final weighted factor map were integrated and produced the final suitable wastewater treatment plant site map using ArcGIS Spatial Analyst tools. As a result, 73.88 km2 (6.90%), 359.55km2 (33.59%), 441.08km2(41.21%), 180.71km2 (16.88%), 15.05 km2 (1.41%) of the total study area was found to be unsuitable, low suitable, moderate suitable, high suitable and very high suitable respectively. The area of very high suitable is preferable for wastewater treatment plant sites, because of their minimum effect on the environment, public health and cost effective than other parts of the study area. Therefore, the study will help the concerned authorities to formulate their development strategies according to the selected suitable wastewater treatment plant site available to the area.

Robust multimodal federated learning for incomplete modalities

Consumer electronics are continuously collecting multimodal data, such as audio, video, and so on. A multimodal learning mechanism can be adopted to deal with these data. Due to the consideration of privacy protection, some successful attempts at multimodal federated learning (MMFed) have been conducted. However, real-world multimodal data is usually missing modalities, which can significantly affect the accuracy of the global model in MMFed. Effectively fusing and analyzing multimodal data with incompleteness remains a challenging problem. To tackle this problem, we propose a robust Multimodal Federated Learning Framework for Incomplete Modalities (FedInMM). More specifically, we design a Long Short-Term Memory (LSTM)-based module to extract the information in the temporal sequence. We dynamically learn a weight map to rescale the feature in each modality and formulate the different contributions of features. And then the content of each modality is further fused to form a uniform representation of all modalities of data. By considering the temporal dependency and intra-relation of multi-modalities automatically through the learning stage, this MMFed framework can efficiently mitigate the effects of missing modalities. By using two multimodal datasets, DEAP and AReM, we have conducted comprehensive experiments by simulating different levels of incompleteness. Experimental results demonstrate that FedInMM outperforms other approaches and can train highly accurate models on datasets comprising different incompleteness patterns, which is more appropriate for integration into a practical multimodal application.

Multi-scale convolutional neural networks and saliency weight maps for infrared and visible image fusion

Image fusion is the fusion of multiple images from the same scene to produce a more informative image, and infrared and visible image fusion is an important branch of image fusion. To tackle the issues of diminished luminosity in the infrared target, inconspicuous target features, and blurred texture of the fused image after the fusion of infrared and visible images. This paper introduces a novel effective fusion framework that merges multi-scale Convolutional Neural Networks (CNN) with saliency weight maps. First, the method measures the source image features to estimate the initial saliency weight map. Then, the initial weight map is segmented and optimized using a guided filter before being further processed by CNN. Next, a trained Siamese convolutional network is used to solve the two key problems of activity measure and weight assignment. Meanwhile, a multi-layer fusion strategy is designed to effectively retain the luminance of the infrared target and the texture information in the visible background. Finally, adaptive adjustment of the fusion coefficients is achieved by employing saliency. The experimental results show that the method outperforms the state-of-the-art algorithms in terms of both subjective visual quality and objective evaluation effects.

Cortical structural changes of morphometric similarity network in early-onset schizophrenia correlate with specific transcriptional expression patterns

BackgroundThis study aimed to investigate the neuroanatomical subtypes among early-onset schizophrenia (EOS) patients by exploring the association between structural alterations and molecular mechanisms using a combined analysis of morphometric similarity network (MSN) changes and specific transcriptional expression patterns.MethodsWe recruited 206 subjects aged 7 to 17 years, including 100 EOS patients and 106 healthy controls (HC). Heterogeneity through discriminant analysis (HYDRA) was used to identify the EOS subtypes within the MSN strength. The differences in morphometric similarity between each EOS subtype and HC were compared. Furthermore, we examined the link between morphometric changes and brain-wide gene expression in different EOS subtypes using partial least squares regression (PLS) weight mapping, evaluated genetic commonalities with psychiatric disorders, identified functional enrichments of PLS-weighted genes, and assessed cellular transcriptional signatures.ResultsTwo distinct MSN-based EOS subtypes were identified, each exhibiting different abnormal MSN strength and cognitive functions compared to HC. The PLS1 score mapping demonstrated anterior–posterior gradients of gene expression in EOS1, whereas inverse distributions were observed in EOS2 cohorts. Genetic commonalities were identified in autistic disorder and adult schizophrenia with EOS1 and inflammatory bowel diseases with EOS2 cohorts. The EOS1 PLS1- genes (Z < -5) were significantly enriched in synaptic signaling-related functions, whereas EOS2 demonstrated enrichments in virtual infection-related pathways. Furthermore, the majority of observed associations with EOS1-specific MSN strength differences contributed to specific transcriptional changes in astrocytes and neurons.ConclusionsThe findings of this study provide a comprehensive analysis of neuroanatomical subtypes in EOS, shedding light on the intricate relationships between macrostructural and molecular aspects of the EOS disease.

Long-Short Range Adaptive Transformer With Dynamic Sampling for 3D Object Detection

3D object detection in point cloud aims at simultaneously localizing and recognizing 3D objects from a 3D point set. However, since point clouds are usually sparse, unordered, and irregular, it is challenging to learn robust point representations and sample high-quality object queries. To deal with the above issues, we propose a Long-short rangE Adaptive transformer with Dynamic sampling (LeadNet), including a point representation encoder, a dynamic object query sampling decoder, and an object detection decoder in a unified architecture for 3D object detection. Specifically, in the point representation encoder, we combine an attention layer and a channel attentive kernel convolution layer to consider the local structure and the long-range context simultaneously. In the dynamic object query sampling decoder, we utilize multiple dynamic prototypes to adapt to various point clouds. In the object detection decoder, we incorporate a dynamic Gaussian weight map into the cross-attention mechanism to help the detection decoder focus on the proper visual regions near the object, further accelerating the training process. Extensive experimental results on two standard benchmarks show that our LeadNet outperforms the 3DETR baseline by 11.6% mAP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</sub> on the ScanNet v2 dataset and achieves the new state-of-the-art results on ScanNet v2 and SUN RGB-D benchmarks for the geometric-only approaches.

A Dynamic Air Combat Situation Assessment Model Based on Situation Knowledge Extraction and Weight Optimization

Air combat situation assessment is the basis of target assignment and maneuver decisions. The current air combat situation assessment models, whether nonparametric or parametric, ignore the continuity and timing of situation changes, making the situation assessment results lose tactical significance. Aimed at the shortcomings of current air combat situation assessment, a dynamic air combat situation assessment model based on situation knowledge extraction and weight optimization was proposed by combining a multiple regression model of hidden logic process, a weight optimization model based on grey prospect theory, a weight mapping model based on autoencoder and extreme learning machine (AE-ELM) and an air combat situation characteristic parameter prediction model based on dynamic weight online extreme learning machine (DWOSELM). Firstly, considering the timing and continuity of air combat situation change, a hidden logic process multiple regression model was introduced to realize the segmentation of air combat situation time series data and the extraction of air combat situation primitives. Secondly, the weight optimization method based on grey prospect theory was used to obtain the weight of the evaluation index under different air combat situations. On this basis, the dynamic mapping model between air combat situation characteristic parameters and the weight of index was constructed by using AE-ELM. Then, the dynamic weighted extreme learning machine was used to build the target maneuver trajectory prediction model, and the future position information of the target was predicted. On this basis, the future situation information between the enemy and us was obtained. Finally, the time weight calculation model based on normal cumulative distribution was used to determine the importance of the situation at each time. The situation information at multiple times in the air combat process was fused to obtain the comprehensive air combat situation assessment results at the current time. The simulation results show that the model can fully exploit the influence of historical information, effectively integrate the air combat situation information at multiple moments, and generate the air combat situation assessment results with practical tactical significance according to the individual differences of different pilots.

A defocus and similarity attention-based cascaded network for multi-focus and misaligned image fusion

Multi-focus image fusion uses multiple images focused on different depths to generate a clear image covering the whole scene. The existing multi-focus image fusion methods do not consider the defocus successive variation in close-range photography and the camera shaking in sequence image shooting, and most methods cannot process multiple images simultaneously. We proposed an end-to-end deep learning network to generate the all-in-focus image from multi-focus and misaligned images to solve these problems. Specifically, taking multiple multi-focus and misaligned source images as input, our Defocus and Similarity Attention Fusion Network (DSAF-Net) first generates the corresponding defocus map through the Defocus-Net, and warps the source images and defocus maps to a unified camera view according to the optical flow estimated by the OpticalFlow-Net. Finally, our model applies a coarse-to-fine correspondence matching scheme to obtain the similarity weight maps, combined with the warped source images and defocus maps to fuse into a clear image. For training and testing our DSAF-Net, a multi-focus and misaligned cultural heritage photography dataset (WHU-MFM) is constructed using Blender Cycles renderer. Experiments results demonstrate that our method outperforms the state-of-the-art methods both qualitatively and quantitatively. DSAF-Net is available at https://github.com/PeimingCHEN/DSAF-Net. And WHU-MFM is available at https://github.com/PeimingCHEN/WHU-MFM-Dataset.

End-to-end Remote Sensing Image Aircraft Target Detection and Fine-grained Recognition Framework

In view of the practical application of target detection and recognition tasks for remote sensing, an end-to-end aircraft target detection and fine-grained recognition framework is proposed. It can accurately and quickly implement detection and recognition in an end-to-end way. The main network of the framework adopts the design ideas of target detection and fine-grained recognition methods using candidate region extract and visual attention, making sure the accuracy of detection and recognition. Then, to solve the problem of high false detection rate and missed detection rate of densely arranged targets, we propose the re-detection mechanism. To minimize the large amounts of calculations of deep networks and improve real-time performance, we introduce depthwise separable convolution to optimize networks. Finally, a weight mapping idea based on transfer learning is adopted to solve the problem of data labeling and also helps the detection and fine-grained recognition. The results prove that the proposed framework has good robustness, versatility, and efficiency in aircraft detection and fine-grained recognition tasks.

All-Solid-State Synaptic Transistors with Lithium-Ion-Based Electrolytes for Linear Weight Mapping and Update in Neuromorphic Computing Systems.

Neuromorphic computing, an innovative technology inspired by the human brain, has attracted increasing attention as a promising technology for the development of artificial intelligence systems. This study proposes synaptic transistors with a Li1-xAlxTi2-x(PO4)3 (LATP) layer to analyze the conductance modulation linearity, which is essential for weight mapping and updating during on-chip learning processes. The high ionic conductivity of the LATP electrolyte provides a large hysteresis window and enables linear weight update in synaptic devices. The results demonstrate that optimizing the LATP layer thickness improves the conductance modulation and linearity of synaptic transistors during potentiation and degradation. A 20 nm-thick LATP layer results in the most nonlinear depression (αd = -6.59), whereas a 100 nm-thick LATP layer results in the smallest nonlinearity (αd = -2.22). Additionally, a device with the optimal 100 nm-thick LATP layer exhibits the highest average recognition accuracy of 94.8% and the smallest fluctuation, indicating that the linearity characteristics of a device play a crucial role in weight update during learning and can significantly affect the recognition accuracy.

BEA-Net: Body and Edge Aware Network with Multi-Scale Short-Term Concatenation for Medical Image Segmentation.

Medical image segmentation is indispensable for diagnosis and prognosis of many diseases. To improve the segmentation performance, this study proposes a new 2D body and edge aware network with multi-scale short-term concatenation for medical image segmentation. Multi-scale short-term concatenation modules which concatenate successive convolution layers with different receptive fields, are proposed for capturing multi-scale representations with fewer parameters. Body generation modules with feature adjustment based on weight map computing via enlarging the receptive fields, and edge generation modules with multi-scale convolutions using Sobel kernels for edge detection, are proposed to separately learn body and edge features from convolutional features in decoders, making the proposed network be body and edge aware. Based on the body and edge modules, we design parallel body and edge decoders whose outputs are fused to achieve the final segmentation. Besides, deep supervision from the body and edge decoders is applied to ensure the effectiveness of the generated body and edge features and further improve the final segmentation. The proposed method is trained and evaluated on six public medical image segmentation datasets to show its effectiveness and generality. Experimental results show that the proposed method achieves better average Dice similarity coefficient and 95% Hausdorff distance than several benchmarks on all used datasets. Ablation studies validate the effectiveness of the proposed multi-scale representation learning modules, body and edge generation modules and deep supervision. The code is available at https://github.com/hulinkuang/BEA-Net.

A Hybrid Weight Quantization Strategy for Memristive Neural Networks

Due to the ability to store data and process information, the memristor-based neuromorphic system has attracted extensive attention. Its efficient parallel computing approach allows it to implement neural networks in hardware. However, due to the limitation of the range of memristor conductance, it is difficult to represent high-precision weights in memristive neural network. During the off-chip learning, it is crucial to find an efficient weight quantization scheme and map it to the memristor array. Therefore, a hybrid weight quantization strategy that combines uniform and non-uniform quantization is proposed to overcome these problems. Specifically, the curve fitting of pulse modulation for tantalum oxide-based memristor is carried out, and the mapping rules of weights are proposed to simplify the process of reading verification. Furthermore, the hybrid quantization strategy is proposed and applied to a multilayer perceptron and a convolutional neural network, respectively. The effectiveness and robustness of the hybrid quantization scheme are verified in the MNIST dataset. Experiments show that the proposed hybrid quantization scheme can achieve 99.26% accuracy at 4 bits and tolerate 20% noise interference. The simulation results in this paper also provide an effective solution for the hardware implementation of memristive neural networks.

Research on seamless image stitching based on fast marching method

AbstractImage stitching is an important way to achieve large‐field high‐resolution imaging. The inconsistencies in brightness and structure and defects in ghosting, blurring and misalignment between images, which are inevitable and difficult to eliminate, make a challenge to image stitching, due to the external lighting environment and changes in camera pose and parameters. Here, a novel method is proposed to search for the optimal seamline based on the fast marching method, which can stitch large parallax images with high quality. A feature weight map is first formed based on the similarity in colour, edge, texture and saliency of the images. Then it is used as the cost value of the seamline to search for the optimal seamline by fast marching method. The results show that this new method is more efficient to reduce defects, such as ghosting, misalignment and chromatic aberration, and realize high quality image stitching compared with traditional stitching tools and methods, which provides a new perspective for image stitching technology.

Groupwise structural sparsity for discriminative voxels identification.

This paper investigates the selection of voxels for functional Magnetic Resonance Imaging (fMRI) brain data. We aim to identify a comprehensive set of discriminative voxels associated with human learning when exposed to a neutral visual stimulus that predicts an aversive outcome. However, due to the nature of the unconditioned stimuli (typically a noxious stimulus), it is challenging to obtain sufficient sample sizes for psychological experiments, given the tolerability of the subjects and ethical considerations. We propose a stable hierarchical voting (SHV) mechanism based on stability selection to address this challenge. This mechanism enables us to evaluate the quality of spatial random sampling and minimizes the risk of false and missed detections. We assess the performance of the proposed algorithm using simulated and publicly available datasets. The experiments demonstrate that the regularization strategy choice significantly affects the results' interpretability. When applying our algorithm to our collected fMRI dataset, it successfully identifies sparse and closely related patterns across subjects and displays stable weight maps for three experimental phases under the fear conditioning paradigm. These findings strongly support the causal role of aversive conditioning in altering visual-cortical activity.

Short-term cooling and heating loads forecasting of building district energy system based on data-driven models

Accurate forecasting of cooling and heating loads is critical for optimizing the energy usage of devices and planning for energy storage in building district energy systems (BDESs). Data-driven models offering high prediction accuracy and efficiency are receiving significant attention for addressing short-term load prediction problems. However, load forecasting for district building levels, as well as interpretable studies of deep learning models, are still the minority of existed research. To satisfy the cooling and heating loads forecasting of an actual BDES, a framework that includes raw data acquisition, data processing, model development, and evaluation is developed and analyzed. The performance of a multi-input-multi-output prediction strategy evaluated using five data-driven models, namely extreme gradient boost (XGB), long short-term memory (LSTM), gated recurrent units (GRU), and attention-added LSTM/GRU, is investigated. The results show that XGB performs the best, with CV-RMSE values of 14.51% and 11.95% for the one-hour-ahead forecasting of cooling and heating loads, respectively. In terms of 24-hour ahead prediction, the most accurate condition is “48–24” attention-added LSTM in both cooling and heating loads forecasting, where the CV-RMSE values recorded are 26.61% and 27.58%, respectively. The accuracy and interpretability of the models improved after an attention mechanism is introduced. The most significant enhancement in the prediction accuracy is reflected by a reduction in the CV-RMSE by 2.01%. The visualized heat maps of attention weights indicate the mechanism by which the models learn and extract the load patterns, where LSTM and GRU indicate different priorities. Further analysis of the attention weight distribution reveals the effects of building thermal inertia and load periodicity.

Adopting multiple vision transformer layers for fine-grained image representation

Accurate discriminative regions proposal has an important effect for fine-grained image recognition. The vision transformer (ViT) brings about a striking effect in computer vision duo to its innate muti-head self-attention mechanism. However, the attention maps are gradually similar after certain layers and since ViT adds classification token for perform classification, it is unable to effectively select discriminative image patches for fine-grained image classification. To accurately detect discriminative regions, we propose a novel network AMTrans, which efficiently increases layers to learn diverse features and utilizes integrated raw attention maps to capture more salient feature. Specifically, we employ DeepViT as backbone to solve the attention collapse issue. Then, we fuse each head attention weight within each layer to produce attention weight map. After that, we alternatively use recurrent residual refinement blocks to promote salient feature detection and then utilize semantic grouping method to propose the discriminative feature region. A lot of experiments prove that AMTrans acquires the SOTA performance on three widely used fine-grained datasets under the same settings, involving Stanford-Cars, Stanford-Dogs and CUB-200-2011.

Feature Alignment in Anchor-Free Object Detection

Most anchor-free methods perform object detection using dense recommendation, which assumes that one point can simultaneously conduct accurate category prediction and regression estimation. However, due to different task drivers, valid features for classification and regression may locate at distinct areas in the training phase. This problem is called feature misalignment. To solve it, we propose a new feature alignment method based on anchor-free object detector. Firstly, a global receptive field adaptor (G-RFA) is designed by incorporating the feature pyramid networks (FPN) with the global attention mechanism, and forward features are further fine-tuned with a deformable-subnet (De-Subnet) to remove the influence of redundant contextual information. Then, a new feature filter strategy with a misalignment score is proposed to guide the network to focus on sampling points with aligned features. In addition, we establish mutually independent multi-layer quality distributions to model the priori information of an object on different FPN levels. Equipped with our method, the classification and regression features are aligned, and the generated foreground weight map converges to the centers of classification and regression heatmaps. Experimental results show that without bells and whistles, our method achieves 49.3% AP on MS COCO test-dev under the default 2x training schedule, outperforming related methods. Besides, experiments on PASCAL VOC demonstrate the generalization ability of our method. Code is available at https://github.com/GFENGG/featurealign.

Multi exposure image fusion based on exposure correction and input refinement using limited low dynamic range images

Multi-Exposure Image Fusion (MEF) is an image processing technique that combines several images taken at various exposure levels into one high-quality image. The generation of artifacts and diminished perceptual quality of the fused image under few input images is an inevitable problem in MEF approaches. Although numerous MEF techniques have been proposed in the literature, little effort has been devoted to generate high quality fused images with minimal input images. The proposed work unveils a method for producing a fused image that exhibits superior visual quality with a minimum of two input images. The proposed work begins with refining the input images using an exposure correction strategy and a recursive filter. The refined images along with a selected exposure corrected image form an intermediate exposure stack. A weight map is constructed from the intermediate exposure stack using a set of weighting terms and a fast-guided filter is used to prevent the weight maps from being distorted by external anomalies. Finally, the fusion is performed using the pyramidal decomposition of weight maps and intermediate images. The resultant fused images obtained have been tested and proven to hold superior performance over the state-of-the-art methods in perceptual and empirical analysis. Benefiting from this approach is a framework that seeks fewer input images, an intermediate exposure stack, and a detail-enriched fused image.