Great Generalization Ability Research Articles

Dynamic modeling of thermo-sensitive hydrogel describing its complex energy conversion mechanism with hysteresis nonlinearity

Thermo-sensitive hydrogel (TSH) demonstrates a lot of promise for soft robots. However, the TSH presents challenges for modeling due to its complex energy conversion mechanism and hysteresis nonlinearity. To address this issue, a dynamic model consisting of an electro-thermal model and a thermo-deformation model is developed in this paper. In which, the electro-thermal model is established for describing the relationship between the driving voltage and the temperature of the TSH. The thermo-deformation model consisting of a Prandtl-Ishlinskii model, a polynomial model and a transfer function model is established between the temperature and the deformation of the TSH. In addition, the thermo-deformation model describes the asymmetric and rate-dependent hysteresis nonlinearity of the TSH. Moreover, due to the analytic inverse of the thermo-deformation model, it is convenient to design a model-based controller to achieve the high-precision deformation control of the TSH. Next, unknown parameters in the above models are determined based on the nonlinear least squares algorithm by using the data from the experiment. Finally, model validations are performed, and the fit values are all greater than 89.40%. Thus, the established dynamic model can accurately describe the complex energy conversion mechanism with hysteresis nonlinearity of the TSH and has great generalization ability.

Stack-HDAC3i: A high-precision identification of HDAC3 inhibitors by exploiting a stacked ensemble-learning framework

Epigenetics involves reversible modifications in gene expression without altering the genetic code itself. Among these modifications, histone deacetylases (HDACs) play a key role by removing acetyl groups from lysine residues on histones. Overexpression of HDACs is linked to the proliferation and survival of tumor cells. To combat this, HDAC inhibitors (HDACi) are commonly used in cancer treatments. However, pan-HDAC inhibition can lead to numerous side effects. Therefore, isoform-selective HDAC inhibitors, such as HDAC3i, could be advantageous for treating various medical conditions while minimizing off-target effects. To date, computational approaches that use only the SMILES notation without any experimental evidence have become increasingly popular and necessary for the initial discovery of novel potential therapeutic drugs. In this study, we develop an innovative and high-precision stacked-ensemble framework, called Stack-HDAC3i, which can directly identify HDAC3i using only the SMILES notation. Using an up-to-date benchmark dataset, we first employed both molecular descriptors and Mol2Vec embeddings to generate feature representations that cover multi-view information embedded in HDAC3i, such as structural and contextual information. Subsequently, these feature representations were used to train baseline models using nine popular ML algorithms. Finally, the probabilistic features derived from the selected baseline models were fused to construct the final stacked model. Both cross-validation and independent tests showed that Stack-HDAC3i is a high-accuracy prediction model with great generalization ability for identifying HDAC3i. Furthermore, in the independent test, Stack-HDAC3i achieved an accuracy of 0.926 and Matthew’s correlation coefficient of 0.850, which are 0.44–6.11% and 0.83–11.90% higher than its constituent baseline models, respectively.

Reconstructing long-term (1980–2022) daily ground particulate matter concentrations in India (LongPMInd)

Abstract. Severe airborne particulate matter (PM, including PM2.5 and PM10) pollution in India has caused widespread concern. Accurate PM concentrations are fundamental for scientific policymaking and health impact assessment, while surface observations in India are limited due to scarce sites and uneven distribution. In this work, a simple structured, efficient, and robust model based on the Light Gradient-Boosting Machine (LightGBM) was developed to fuse multisource data and estimate long-term (1980–2022) historical daily ground PM concentrations in India (LongPMInd). The LightGBM model shows good accuracy with out-of-sample, out-of-site, and out-of-year cross-validation (CV) test R2 values of 0.77, 0.70, and 0.66, respectively. Small performance gaps between PM2.5 training and testing (delta RMSE of 1.06, 3.83, and 7.74 µg m−3) indicate low overfitting risks. With great generalization ability, the openly accessible, long-term, and high-quality daily PM2.5 and PM10 products were then reconstructed (10 km, 1980–2022). This showed that India has experienced severe PM pollution in the Indo-Gangetic Plain (IGP), especially in winter. PM concentrations have significantly increased (p<0.05) in most regions since 2000 (0.34 µgm-3yr-1). The turning point occurred in 2018 when the Indian government launched the National Clean Air Programme, and PM2.5 concentrations declined in most regions (−0.78 µgm-3yr-1) during 2018–2022. Severe PM2.5 pollution caused continuous increased attributable premature mortalities, from 0.73 (95 % confidence interval (CI) [0.65, 0.80]) million in 2000 to 1.22 (95 % CI [1.03, 1.41]) million in 2019, particularly in the IGP, where attributable mortality increased from 0.36 million to 0.60 million. LongPMInd has the potential to support multiple applications of air quality management, public health initiatives, and efforts to address climate change. The daily and monthly PM2.5 and PM10 concentrations are publicly accessible at https://doi.org/10.5281/zenodo.10073944 (Wang et al., 2023a).

Logging Lithology Discrimination with Enhanced Sampling Methods for Imbalance Sample Conditions

In the process of lithology discrimination from a conventional well logging dataset, the imbalance in sample distribution restricts the accuracy of log identification, especially in the fine-scale reservoir intervals. Enhanced sampling balances the distribution of well logging samples of multiple lithologies, which is of great significance to precise fine-scale reservoir characterization. This study employed data over-sampling and under-sampling algorithms represented by the synthetic minority over-sampling technique (SMOTE), adaptive synthetic sampling (ADASYN), and edited nearest neighbors (ENN) to process well logging dataset. To achieve automatic and precise lithology discrimination on enhanced sampled well logging dataset, support vector machine (SVM), random forest (RF), and gradient boosting decision tree (GBDT) models were trained using cross-validation and grid search methods. Aimed to objectively evaluate the performance of different models on different sampling results from multiple perspectives, the lithology discrimination results were evaluated and compared based on the Jaccard index and F1 score. By comparing the predictions of eighteen lithology discrimination workflows, a new discrimination process containing ADASYN, ENN, and RF has the most precise lithology discrimination result. This process improves the discrimination accuracy of fine-scale reservoir interval lithology, has great generalization ability, and is feasible in a variety of different geological environments.

Domain base dynamic convolution and distance map guidance for anterior mediastinal lesion segmentation

The automatic segmentation of anterior mediastinal lesions in the enhanced CT image plays a significant role in clinical diagnosis. Due to the low incidence rate of anterior mediastinal disease, the data for training segmentation network may come from different hospitals, different equipment, and different operators, which leads to lower generalization and robustness of the trained network. This paper proposes an anterior mediastinal lesion segmentation method based on domain-adaptive dynamic convolution and distance map guidance. Considering the diversity of clinical data sources and equipment differences, the domain prior knowledge is added to the global average-pooling layer of the network, and the dynamic convolution forms specific network parameters for specific domains to enhance domain-specific information. Meanwhile, the Singed Distance Map (SDM) head is involved in the network according to the shape, size and position of the anterior mediastinal lesions to constraint the boundary and assist in localization. Ablation experiments demonstrate that the distance map can effectively constrain the segmentation target and reduce false positives. Both qualitative and quantitative experimental results indicate that our method can achieve more accurate anterior mediastinal lesion segmentation with greater generalization ability. Our approach achieves an overall Dice coefficient of 88.33%, which is 2.31% higher than existing state-of-the-art methods, and has achieved good performance in terms of ASSD evaluation for segmentation edges.

The Contrastive Network With Convolution and Self-Attention Mechanisms for Unsupervised Cell Segmentation.

Deep learning for cell instance segmentation is a significant research direction in biomedical image analysis. The traditional supervised learning methods rely on pixel-wise annotation of object images to train the models, which is often accompanied by time-consuming and labor-intensive. Various modified segmentation methods, based on weakly supervised or semi-supervised learning, have been proposed to recognize cell regions by only using rough annotations of cell positions. However, it is still hard to achieve the fully unsupervised in most approaches that the utilization of few annotations for training is still inevitable. In this article, we propose an end-to-end unsupervised model that can segment individual cell regions on hematoxylin and eosin (H&E) stained slides without any annotation. Compared with weakly or semi-supervised methods, the input of our model is in the form of raw data without any identifiers and there is no need to generate pseudo-labelling during training. We demonstrated that the performance of our model is satisfactory and also has a great generalization ability on various validation sets compared with supervised models. The ablation experiment shows that our backbone has superior performance in capturing object edge and context information than pure CNN or transformer under our unsupervised method.

Exploring exohedral functionalization of fullerene with automation and Neural Network Potential

Exohedral functionalized fullerenes have shown superior physicochemical properties over pristine carbon cages. The functional groups could significantly improve solubility, electron affinity, and photoelectric properties. However, their numerous distribution patterns have long been puzzling for theoretical chemists, and there are unmet needs for tools to unveil their functionalization mechanism. This work automates the traditional stepwise model as well as various analysis workflows within an open-source package AutoSteper. Besides, a Neural Network Potential (NNP) is trained, validated, and proven to have great generalization ability and transferability. Several case studies are performed with AutoSteper to explore functionalization mechanisms, such as the selectivity and the functionalization pathway of a specific isomer, and the Stone-Wales Rearrangement (SWR) in co-crystallization systems. In addition, to reasonably narrow down the screen scope for giant nanoclusters, a new variant stepwise model is proposed. Fruitful scientific discoveries demonstrate that AutoSteper is capable of providing complete, comprehensive, and cost-effective analysis. With the combination of Automation and NNP, a data-driven research paradigm is realized for fullerene chemistry.

Edibility and species discrimination of wild bolete mushrooms using FT-NIR spectroscopy combined with DD-SIMCA and RF models

Mushroom poisoning is a highly publicized food safety concern, and commercial fraud by adulterating cheap species in the wild mushroom supply chain often occurs. To safeguard the lives and property of consumers, it is necessary to develop accurate methods to discriminate the edibility and species of wild mushrooms. Near-infrared (NIR) spectroscopy has become a popular analytical tool due to its key advantages. This work aimed to investigate the feasibility of discriminating the edibility and species of wild bolete mushrooms using NIR spectroscopy-based two-dimensional correlation spectroscopy (2DCOS) combined with Red-Green-Blue (RGB) image analysis and multivariate analysis methods. The results showed that the data driven version of soft independent modeling of class analogy (DD-SIMCA) model had excellent sensitivity (0.98) but the specificity was only 0.59. This model was not suitable to accurately discriminate the edibility of boletes, but can be used for screening a potentially poisonous species of boletes (Caloboletus calopus). As for species discrimination, random forests (RF) model had great classification and generalization ability and the accuracy of the training set and test set was 97.22% and 100%, respectively. Therefore, FT-NIR spectroscopy combined with DD-SIMCA and RF models had the potential to discriminate the edibility and species of wild bolete mushrooms.

Android malware detection based on sensitive patterns

In recent years, the rapid increase in the number and type of Android malware has brought great challenges and pressure to malware detection systems. As a widely used method in android malware detection, static detecting has been a hot topic in academia and industry. However, in order to improve the accuracy of detection, the existing static detecting methods sacrifice the excessively high analysis complexity and time cost. Moreover, the correlation between static features leads to redundancy of a large amount of data. Therefore, this paper proposes a static detecting method of Android malware based on sensitive pattern. It uses an improved FP-growth algorithm to mine frequent combinations of sensitive permissions and API calls in malicious apps and benign apps, which avoids the generation of redundant information. In addition, this paper adopts multi-layered gradient boosting decision trees algorithm to train the detection model. And a dual similarity combination method is proposed to measure the similarity between different sensitive patterns. The experimental results show that our proposed detection method has high accuracy and great generalization ability.

A review on multimodal zero‐shot learning

AbstractMultimodal learning provides a path to fully utilize all types of information related to the modeling target to provide the model with a global vision. Zero‐shot learning (ZSL) is a general solution for incorporating prior knowledge into data‐driven models and achieving accurate class identification. The combination of the two, known as multimodal ZSL (MZSL), can fully exploit the advantages of both technologies and is expected to produce models with greater generalization ability. However, the MZSL algorithms and applications have not yet been thoroughly investigated and summarized. This study fills this gap by providing an objective overview of MZSL's definition, typical algorithms, representative applications, and critical issues. This article will not only provide researchers in this field with a comprehensive perspective, but it will also highlight several promising research directions.This article is categorized under: Algorithmic Development > Multimedia Technologies > Classification Technologies > Machine Learning

A benchmark for hypothalamus segmentation on T1-weighted MR images

The hypothalamus is a small brain structure that plays essential roles in sleep regulation, body temperature control, and metabolic homeostasis. Hypothalamic structural abnormalities have been reported in neuropsychiatric disorders, such as schizophrenia, amyotrophic lateral sclerosis, and Alzheimer's disease. Although mag- netic resonance (MR) imaging is the standard examination method for evaluating this region, hypothalamic morphological landmarks are unclear, leading to subjec- tivity and high variability during manual segmentation. Due to these limitations, it is common to find contradicting results in the literature regarding hypothalamic volumetry. To the best of our knowledge, only two automated methods are available in the literature for hypothalamus segmentation, the first of which is our previous method based on U-Net. However, both methods present performance losses when predicting images from different datasets than those used in training. Therefore, this project presents a benchmark consisting of a diverse T1-weighted MR image dataset comprising 1381 subjects from IXI, CC359, OASIS, and MiLI (the latter created specifically for this benchmark). All data were provided using automatically generated hypothalamic masks and a subset containing manually annotated masks. As a baseline, a method for fully automated segmentation of the hypothalamus on T1-weighted MR images with a greater generalization ability is presented. The pro- posed method is a teacher-student-based model with two blocks: segmentation and correction, where the second corrects the imperfections of the first block. After using three datasets for training (MiLI, IXI, and CC359), the prediction performance of the model was measured on two test sets: the first was composed of data from IXI, CC359, and MiLI, achieving a Dice coefficient of 0.83; the second was from OASIS, a dataset not used for training, achieving a Dice coefficient of 0.74. The dataset, the baseline model, and all necessary codes to reproduce the experiments are available at https://github.com/MICLab-Unicamp/HypAST and https://sites.google.com/ view/calgary-campinas-dataset/hypothalamus-benchmarking. In addition, a leaderboard will be maintained with predictions for the test set submitted by anyone working on the same task.

Adaptive multiscale convolutional neural network model for chemical process fault diagnosis

Intelligent fault recognition techniques are essential to ensure the long-term reliability of manufacturing. Due to the variations in material, equipment and environment, the process variables monitored by sensors contain diverse data characteristics at different time scales or in multiple operating modes. Despite much progress in statistical learning and deep learning for fault recognition, most models are constrained by abundant diagnostic expertise, inefficient multiscale feature extraction and unruly multimode condition. To overcome the above issues, a novel fault diagnosis model called adaptive multiscale convolutional neural network (AMCNN) is developed in this paper. A new multiscale convolutional learning structure is designed to automatically mine multiple-scale features from time-series data, embedding the adaptive attention module to adjust the selection of relevant fault pattern information. The triplet loss optimization is adopted to increase the discrimination capability of the model under the multimode condition. The benchmarks CSTR simulation and Tennessee Eastman process are utilized to verify and illustrate the feasibility and efficiency of the proposed method. Compared with other common models, AMCNN shows its outstanding fault diagnosis performance and great generalization ability.

Interactive Mode of Visual Communication Based on Information Visualization Theory.

In the modern environment, visual communication design has become a comprehensive subject that combines technology and art. The development of various technologies has promoted the emergence of new design art forms, and at the same time, it has also promoted the change of design concepts and thinking modes. Visual representation, as a representation practice to express the meaning of information in the form of visual symbols, is a method and means to realize information visualization. Based on visual interface design, this article takes interactivity and user experience as innovation points and digs into the design process of interactive information visualization to achieve the purpose of information transmission. By analyzing the cognitive tasks of users in each stage of visual thinking activities in visual terminals, this article proposes to optimize the interactive design of visual communication from the key points of attention, consciousness, and memory. In order to verify the feasibility of the interactive mode of visual communication based on information visualization theory, this article has carried out several comparative experiments with different models. Experimental results show that this algorithm has a faster convergence speed and a greater generalization ability, the accuracy of the algorithm reaches 96.03%, and the highest evaluation of user satisfaction is 95.93%. The interactive mode of visual communication in this article can provide means and reference value for the development direction of visual communication design.

Crafting universal adversarial perturbations with output vectors

Recently, researches on universal adversarial perturbations have deepened the public’s attention to the security of deep neural networks (DNNs). Most researchers use iterative methods or generative adversarial networks (GANs) to find universal adversarial perturbations (UAPs). However, just focusing on outputs still cannot separate adversarial examples in feature space and there is almost no correlation between the perturbations generated by random noise and the target network. To cope with these problems, we propose a novel end-to-end UAPs generation framework that uses constant activation information as the input of the generator and adopts a feature adaptive loss function. Experiments show that the attack success rate of our UAPs has reached a competitive level against different classification networks on CIFAR-10 and ImageNet where the average fooling rate can reach 86% and 96%, leading to 19% and 16% improvements over UAP, respectively. Furthermore, we demonstrate the great generalization ability and strong transferability of proposed attacker. Finally, the proposed attacker can achieve lower mAP and mean IoU than several classical white box attackers in object detection task and image segmentation task.

Predictive Modeling of Millimeter-Wave Vegetation-Scattering Effect Using Hybrid Physics-Based and Data-Driven Approach

System design and deployment of millimeter-wave (mmWave) wireless communication systems for outdoor coverage need sophisticated channel models to describe the wave interaction with illuminated physical objects. However, the conventional physical–statistical model cannot accurately predict the site-specific mmWave channel characteristics when involving complex system configurations and geometric information of transceiver relative locations. A framework of machine learning (ML) assisted channel modeling approach is proposed, in which the statistical models are leveraged for inter-cluster-level channel characterization and the propagation properties within each kind of clusters are predicted using a hybrid physics-based and data-driven approach. In particular, with a focus on the mmWave through-vegetation-scattering effect, a set of dedicated directional channel measurements and ray-tracing simulations is performed in an identical vegetated street canyon environment at 28 GHz for the performance evaluation of the proposed approach. Moreover, the training results and model validation in different environments show that comparing with the physical–statistical model, the proposed hybrid model, which adds the environment features to the artificial neural network (ANN) as inputs, has higher prediction accuracy and greater generalization ability in terms of the site-specific through-vegetation cluster parameters, such as vegetation attenuation, delay spread (DS), and angular spread.

Nonsingular fast terminal sliding mode control based on neural network with adaptive robust term for robotic manipulators with actuators

For accurate trajectory tracking of robotic manipulators with actuators, a novel nonsingular fast terminal sliding mode control (NFTSMC) strategy based on radial basis function neural network (RBFNN) is put forward and investigated in this paper. Because of the existence of nonsingular fast terminal sliding mode (NFTSM) manifold, the controller possesses high precision and fast convergence. Considering that it is difficult to obtain accurate model parameters owing to modeling errors or external disturbances, RBFNN is used to approximate the nonlinear uncertainties due to its simple structure and great generalization ability. A new adaptive law is designed to adjust RBFNN. In order to compensate the estimation errors and suppress other unstable factors, a robust term is introduced. A new adaptive law is developed to flexibly adjust the robust term. Then, Lyapunov theory is applied to prove the system stability and finite-time convergence. Finally, a small-sized industrial robotic manipulator Epson LS3-401S with its first two joints is taken as the simulation plant, and several simulations between the proposed controller and the other two controllers are performed. External disturbances and other two conditions are considered to simulate the real environment, and the corresponding results verify the effectiveness and superiority of the proposed controller.

Establishment of Economic Forecasting Model of High-tech Industry Based on Genetic Optimization Neural Network.

Scientific and accurate prediction of high-tech industries is of great practical significance for government departments to grasp the future economic operation and formulate development strategies. In this paper, aiming at some shortcomings of neural network (NN) applied in economic forecasting, GANN was introduced to construct the economic forecasting model of high-tech industry. Genetic algorithm (GA) has simple calculation and strong robustness and can generally ensure convergence to the global optimum, which effectively overcomes the shortcomings of NN using gradient descent method. In order to verify the feasibility of the economic forecasting model in this paper, the comparative experiments of different models are carried out in this paper. Experimental results show that the proposed algorithm has faster convergence speed and greater generalization ability, and the average error rate is reduced to about 1%. The prediction accuracy of this model reached 95.14%, which was about 11.93% higher than the previous model. Applying the economic forecasting model in this paper to the economic forecasting of high-tech industries can provide the means and reference value for the government to formulate regional future economic development plans, forecast, and control the economic growth and development direction.

MinkLoc3D-SI: 3D LiDAR Place Recognition With Sparse Convolutions, Spherical Coordinates, and Intensity

The 3D LiDAR place recognition aims to estimate a coarse localization in a previously seen environment based on a single scan from a rotating 3D LiDAR sensor. The existing solutions to this problem include hand-crafted point cloud descriptors (e.g., ScanContext, M2DP, LiDAR IRIS) and deep learning-based solutions (e.g., PointNetVLAD, PCAN, LPD-Net, DAGC, MinkLoc3D), which are often only evaluated on accumulated 2D scans from the Oxford RobotCar dataset. We introduce MinkLoc3D-SI, a sparse convolution-based solution that utilizes spherical coordinates of 3D points and processes the intensity of 3D LiDAR measurements, improving the performance when a single 3D LiDAR scan is used. Our method integrates the improvements typical for hand-crafted descriptors (like ScanContext) with the most efficient 3D sparse convolutions (MinkLoc3D). Our experiments show improved results on single scans from 3D LiDARs (USyd Campus dataset) and great generalization ability (KITTI dataset). Using intensity information on accumulated 2D scans (RobotCar Intensity dataset) improves the performance, even though spherical representation doesn’t produce a noticeable improvement. As a result, MinkLoc3D-SI is suited for single scans obtained from a 3D LiDAR, making it applicable in autonomous vehicles.

Physics-Consistent Data-Driven Waveform Inversion With Adaptive Data Augmentation

Seismic full-waveform inversion (FWI) is a nonlinear computational imaging technique that can provide detailed estimates of subsurface geophysical properties. Solving the FWI problem can be challenging due to its ill-posedness and high computational cost. In this work, we develop a new hybrid computational approach to solve FWI that combines physics-based models with data-driven methodologies. In particular, we develop a data augmentation strategy that can not only improve the representativity of the training set but also incorporate important governing physics into the training process and therefore improve the inversion accuracy. To validate the performance, we apply our method to synthetic elastic seismic waveform data generated from a subsurface geologic model built on a carbon sequestration site at Kimberlina, California. We compare our physics-consistent data-driven inversion method to both purely physics-based and purely data-driven approaches and observe that our method yields higher accuracy and greater generalization ability.

Image Deep Learning Assisted Prediction of Mechanical and Corrosion Behavior for Al-Zn-Mg Alloys

The use of metallographic images to predict the mechanical properties of materials and their corrosion behavior is helpful in achieving nondestructive detection and quality control. However, after a long-term attempt, the traditional methods cannot accurately correlate the mechanical properties and corrosion behavior of materials with the corresponding microstructure images. In this study, we propose a deep learning strategy to predict the mechanical property and corrosion behavior of large-scale extruded aluminum profiles using surface optical microstructure images. The proposed models with remarkable properties were established through experimental dataset collection, dataset preprocessing, deep learning network modification, and key parameter screening. Taking extruded Al-Zn-Mg alloys with different surface microstructures as example materials, 4,800 sets of “metallographic image – hardness (HV) – corrosion potential (E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">corr</sub> )” data were experimentally collected to establish the HV and E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">corr</sub> models with prediction accuracies of 90% and 82%, respectively. The proposed HV and E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">corr</sub> models exhibit great generalization ability with mean average errors of 1.8 HV and 7.0 mV on experimental validation sets, respectively. The proposed model can accurately correlate the metallographic images, mechanical property, and corrosion behavior, which can provide theoretical support for intelligent and nondestructive testing methods to further prevent unexpected material failure.