Domain Datasets Research Articles

Con2Mix: A semi-supervised method for imbalanced tabular security data1

Con2Mix (Contrastive Double Mixup) is a new semi-supervised learning methodology that innovates a triplet mixup data augmentation approach for finding code vulnerabilities in imbalanced, tabular security data sets. Tabular data sets in cybersecurity domains are widely known to pose challenges for machine learning because of their heavily imbalanced data (e.g., a small number of labeled attack samples buried in a sea of mostly benign, unlabeled data). Semi-supervised learning leverages a small subset of labeled data and a large subset of unlabeled data to train a learning model. While semi-supervised methods have been well studied in image and language domains, in security domains they remain underutilized, especially on tabular security data sets which pose especially difficult contextual information loss and balance challenges for machine learning. Experiments applying Con2Mix to collected security data sets show promise for addressing these challenges, achieving state-of-the-art performance on two evaluated data sets compared with other methods.

Research on the Bearing Lifespan Prediction Method for Ship Propulsion Shaft Systems Based on an Enhanced Domain Adversarial Neural Network

To address the challenge of accurate lifespan prediction for bearings in different operating conditions within ship propulsion shaft systems, a two-stage prediction model based on an enhanced domain adversarial neural network (DANN) is proposed. Firstly, pre-training features containing comprehensive degradation information are extracted from the entire source domain dataset encompassing all operational conditions. Subsequently, DANN is employed to extract domain-invariant features that are difficult to distinguish. Following this, a feature alignment process is utilized to align high-dimensional features with pre-training features, thereby mitigating the adverse effects caused by missing data in the incomplete target operational condition dataset. Finally, the effectiveness of this approach is validated using operational data from bearings under multiple operating conditions. The experimental results demonstrate that the method presented in this paper achieves an average error reduction of 0.0626 and 0.0845 compared to the MK-MMD transfer learning method and self-attention ConvLSTM algorithms, respectively, and exhibits higher predictive reliability. This method can provide valuable insights for lifespan prediction challenges concerning bearings in ship propulsion shaft systems under various operational conditions, as well as similar cross-domain lifespan prediction problems.

Research on the service condition monitoring method of rolling bearings based on adaptive deconvolution and transformer

Aiming at the complexity of the rolling bearing operation state in actual working conditions and the limitation of the feature extraction capability of single-scale model, this paper proposes a fault diagnosis method based on two-channel adaptive deconvolution and transformer module (TADAT). First, a frequency domain dataset is built and multiscale feature information is extracted using two-channel adaptive deconvolution at the beginning stage of the model; the global features of the signal are further extracted by combining them in series with the transformer (multi-head attention). Next, the loss function is optimized by introducing a regularized loss function with label smoothing and L2 regularization penalty term. Finally, the model is validated against an open-source dataset and operating condition data collected from an unbalanced bearing load test bed with an accuracy of over 98.5%. The model also maintains high accuracy under noisy samples. The test shows that the rolling bearing service condition monitoring method based on TADAT feature extraction model proposed by this method is characterized by simple model structure, high diagnostic accuracy and strong generalization performance, which provides a useful reference for the research of rolling bearing service condition monitoring.

Modeling PM2.5 forecast using a self-weighted ensemble GRU network: Method optimization and evaluation

Due to the rapid industrial development and global concern about air pollution, understanding the dynamics of PM2.5 concentration has become a key aspect of air quality prediction. Many deep learning and mode decomposition techniques have been explored to capture the temporal and nonlinear features of PM2.5 concentration data. However, most of the existing methods ignore the differences in prediction losses of individual subsequences, resulting in lower prediction accuracy. To address this limitation, we proposed an ensemble gated recurrent unit (GRU) model that incorporated a self-weighted total loss function based on variational mode decomposition (VMD). In this approach, the PM2.5 concentration series were decomposed using the VMD, and then each decomposed subsequence (including the residual sequence) was fed into the GRU and the predicted loss of the subsequence was then calculated. For the model to output optimal predictions, we used a self-weighted ensemble loss function to adaptively optimize the prediction loss for each subsequence. Specifically, larger weights were assigned to the model's subsequences with higher predictive losses to better focus on those with higher predictive losses. In addition, the hyperparameter of the model was adjusted to adapt to various datasets in different domains. Experimental results on the three datasets show that our model performs better than the VMD-GRU and single GRU models. This validates the effectiveness of our model. Our approach has the advantage of plug-and-play, making it easier to seamlessly integrate deep learning techniques and pattern decomposition methods into air quality prediction.

Inception-VR70: An Advanced Inception-Net Artificial Intelligence based Novel Hybrid Architecture

In the present day, Convolutional Neural Network (CNN) architectures are undergoing a great deal of development, which has resulted in the creation of models like VGG16, ResNet50, InceptionV3 etc., that significantly increase accuracy. Yet, a network that can deal with overfitting, a significant challenge in deep learning besides having greater accuracy and extracting useful features is required. In this paper, we propose a Deep hybrid model which is an inception of pretrained models with a different input image size, significantly leading to improved accuracy which has been tested on various datasets of different domains including health care, agriculture, and the remote sensing. The performance of this hybrid model is superior to the standalone pretrained models. It is observed that the hybrid model proposed in this paper merely has overfitting despite of having very deep layers compared to all other deep architectures. Best accuracy achieved for this model is 99.64% being train accuracy and 96.33% being test accuracy for the Satellite Images of Hurricane Damage dataset with minimum overfitting.

VerifyTL: Secure and Verifiable Collaborative Transfer Learning

Getting access to labeled datasets in certain sensitive application domains can be challenging. Hence, one may resort to transfer learning to transfer knowledge learned from a source domain with sufficient labeled data to a target domain with limited labeled data. However, most existing transfer learning techniques only focus on one-way transfer which may not benefit the source domain. In addition, there is the risk of a malicious adversary corrupting a number of domains, which can consequently result in inaccurate prediction or privacy leakage. In this paper, we construct a secure and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Verif</b> iable collaborative <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</b> ransfer <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> earning scheme, VerifyTL, to support two-way transfer learning over potentially untrusted datasets by improving knowledge transfer from a target domain to a source domain. Furthermore, we equip VerifyTL with a secure and verifiable transfer unit employing SPDZ computation to provide privacy guarantee and verification in the multi-domain setting. Thus, VerifyTL is secure against malicious adversary that can compromise up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n-1$</tex-math></inline-formula> out of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> data domains. We analyze the security of VerifyTL and evaluate its performance over four real-world datasets. Experimental results show that VerifyTL achieves significant performance gains over existing secure learning schemes.

Research on sentiment analysis method of opinion mining based on multi-model fusion transfer learning

With the popularity of social media, opinion mining has gradually become a popular research field. Among these fields, sentiment analysis mining is an important research direction in the field of opinion mining. It aims to reveal the public's sentiment tendency, and attitude towards specific topics or events by analyzing text data generated by users on online platforms and digital media. However, the large amount of opinion data usually lacks effective annotation, which limits the learning and construction of opinion models. Therefore, focusing on the problem of the scarcity of labeled data in opinion analysis, this paper proposes a mining method for public opinion sentiment analysis based on multi-model fusion transfer learning, that can make full use of the limited labeled data to improve the learning efficiency of sentiment features by integrating the advantages of different models. Additionally, it introduces a transfer learning strategy to enable the models of the target domains to perform better in the absence of labeled data. Furthermore, the attention mechanism is combined to strengthen the acquisition of key features of the emotional colors and improve the accuracy of sentiment analysis. Specifically, the paper uses the ERNIE model to generate dynamic representations of the text word vectors in the dataset. It also uses TextCNN and BiGRU to construct a joint model for extracting local and overall features of the text word vectors. The parameters of the feature layer of the trained model are migrated to the target domain through transfer learning. The attention mechanism is combined with the model to identify the extreme elements of the sentiment. Finally, the local and overall features are fused to achieve comprehensive mining of public opinion and emotional information. This method can effectively improve the accuracy and generalization of public opinion analysis in cases of data scarcity. In the experimental part, the paper conducts comparisons and analyses in eight aspects: word embedding model, model combination, attention mechanism, transfer learning, source domain dataset, target domain dataset, model training, and baseline model. The four indicators, namely accuracy, precision, recall, and F1-measure are used to evaluate the performance of the method. The experiments are thorough and detailed, demonstrating the effective improvement of opinion mining performance.

Role-aware random walk for network embedding

Network embedding is a fundamental part of many network analysis tasks, including node classification and link prediction. The existing random walk-based embedding methods aim to learn node embedding that preserves information on either node proximity or structural similarity. However, the information on both role and community is important to network nodes. To address the shortcomings of the existing methods, this paper proposes a novel method for network embedding called the RARE, which can be used for the analysis of different types of networks and even disconnected networks. The proposed method uses the role and community information of nodes to preserve both node proximity and structural similarity in the learned node embeddings. The walks generated through the role-aware random walk can capture the role and community information of nodes. The obtained walks are input to the Skip-gram model to learn the final embedding of nodes. In addition, the RARE is extended to the CRARE that adds the sampling of high-order community members to the customized random walk so that the node’s representation can preserve more structural information of the network. The performances of the proposed methods are evaluated on multi-class node classification, link prediction, and network visualization tasks. Experimental results on different domain datasets indicate that the proposed methods outperform the baseline methods. The proposed methods can be further accelerated using parallelization in the random walk generation process. The source code: https://github.com/HeguiZhang/RARE.

A Comparative Analysis for Filter-Based Feature Selection Techniques with Tree-based Classification

The selection of features is crucial as an essential pre-processing method, used in the area of research as Data Mining, Text mining, and Image Processing. Raw datasets for machine learning, comprise a combination of multidimensional attributes which have a huge amount of size. They are used for making predictions. If these datasets are used for classification, due to the majority of the presence of features that are inconsistent and redundant, it occupies more resources according to time and produces incorrect results and effects on the classification. With the intention of improving the efficiency and performance of the classification, these features have to be eliminated. A variety of feature subset selection methods had been presented to find and eliminate as many redundant and useless features as feasible. A comparative analysis for filter-based feature selection techniques with tree-based classification is done in this research work. Several feature selection techniques and classifiers are applied to different datasets using the Weka Tool. In this comparative analysis, we evaluated the performance of six different feature selection techniques and their effects on decision tree classifiers using 10-fold cross-validation on three datasets. After the analysis of the result, It has been found that the feature selection method ChiSquaredAttributeEval + Ranker search with Random Forest classifier beats other methods for effective and efficient evaluation and it is applicable to numerous real datasets in several application domains

Hyperplane projection network for few-shot relation classification

Recently meta-learning-based few-shot learning methods have been widely used for relation classification. Previous work reveals that meta-learning performs poorly in scenarios where the edge probability distribution of the target domain dataset appears to be significantly different from the source domain. In this paper, we enhance the meta-learning framework with high-dimensional semantic feature extraction and hyperplane projection metrics for meta-tasks. First, we enhance the focus of BERT on entity words by adding entity markers and vector pooling. After that, the high-dimensional semantic features of the support set are extracted and transformed into hyperplanes. Finally, we obtain the classification results by calculating the projection distance between the query sample and the hyperplane. In addition, we design a auxiliary function with a plane correction factor, which can better amplify the plane spacing and reduce the degree of category confusion, which is important for solving the problem of metric spatial loss. Experiments on two real-world few-shot datasets show that our model HPN is more effective in classifying few-shot relations in the same domain and domain-adapted scenarios. And HPN is more stable on NOTA tasks.

Curing process monitoring of polymeric composites with Gramian angular field and transfer learning-boosted convolutional neural networks

Continuous and accurate monitoring of the degree of curing (DoC) is essential for ensuring the structural integrity of fabricated composites during service. Although machine learning (ML) has shown effectiveness in DoC monitoring, its generalization and extendibility are limited when applied to other curing-related scenarios not included in the previous learning process. To break through this bottleneck, we propose a novel DoC monitoring approach that utilizes transfer learning (TL)-boosted convolutional neural networks alongside Gramian angular field-based imaging processing. The effectiveness of the proposed approach is validated through experiments on metal/polymeric composite co-bonded structures and carbon fiber reinforced polymers using raw sensor data separately collected through the electromechanical impedance and fiber Bragg grating (FBG) measurements. Four indicators, accuracy, precision, recall, and F1-score are introduced to evaluate the performance of generalization and extendibility of the proposed approach. The indicator scores of the proposed approach exceed 0.9900 and outperform other conventional ML algorithms on the FBG dataset of the target domain, demonstrating the effectiveness of the proposed approach in reusing the pre-trained base model on the composite curing monitoring issues.

Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches, and Future Directions

Abstractive dialogue summarization generates a concise and fluent summary covering the salient information in a dialogue among two or more interlocutors. It has attracted significant attention in recent years based on the massive emergence of social communication platforms and an urgent requirement for efficient dialogue information understanding and digestion. Different from news or articles in traditional document summarization, dialogues bring unique characteristics and additional challenges, including different language styles and formats, scattered information, flexible discourse structures, and unclear topic boundaries. This survey provides a comprehensive investigation of existing work for abstractive dialogue summarization from scenarios, approaches to evaluations. It categorizes the task into two broad categories according to the type of input dialogues, i.e., open-domain and task-oriented, and presents a taxonomy of existing techniques in three directions, namely, injecting dialogue features, designing auxiliary training tasks, and using additional data. A list of datasets under different scenarios and widely accepted evaluation metrics are summarized for completeness. After that, the trends of scenarios and techniques are summarized, together with deep insights into correlations between extensively exploited features and different scenarios. Based on these analyses, we recommend future directions, including more controlled and complicated scenarios, technical innovations and comparisons, publicly available datasets in special domains, and so on.

Domain adaptive fruit detection method based on multiple alignments

While deep learning based object detection methods have achieved high accuracy in fruit detection, they rely on large labeled datasets to train the model and assume that the training and test samples come from the same domain. This paper proposes a cross-domain fruit detection method with image and feature alignments. It first converts the source domain image into the target domain through an attention-guided generative adversarial network to achieve the image-level alignment. Then, the knowledge distillation with mean teacher model is fused in the yolov5 network to achieve the feature alignment between the source and target domains. A contextual aggregation module similar to a self-attention mechanism is added to the detection network to improve the cross-domain feature learning by learning global features. A source domain (orange) and two target domain (tomato and apple) datasets are used for the evaluation of the proposed method. The recognition accuracy on the tomato and apple datasets are 87.2% and 89.9%, respectively, with an improvement of 10.3% and 2.4%, respectively, compared to existing methods on the same datasets.

Retina-TransNet: A Gradient-Guided Few-Shot Retinal Vessel Segmentation Net.

Due to the high labor cost of physicians, it is difficult to collect a rich amount of manually-labeled medical images for developing learning-based computer-aided diagnosis (CADx) systems or segmentation algorithms. To tackle this issue, we reshape the image segmentation task as an image-to-image (I2I) translation problem and propose a retinal vascular segmentation network, which can achieve good cross-domain generalizability even with a small amount of training data. We devise primarily two components to facilitate this I2I-based segmentation method. The first is the constraints provided by the proposed gradient-vector-flow (GVF) loss, and, the second is a two-stage Unet (2Unet) generator with a skip connection. This configuration makes 2Unet's first-stage play a role similar to conventional Unet, but forces 2Unet's second stage to learn to be a refinement module. Extensive experiments show that by re-casting retinal vessel segmentation as an image-to-image translation problem, our I2I translator-based segmentation subnetwork achieves better cross-domain generalizability than existing segmentation methods. Our model, trained on one dataset, e.g., DRIVE, can produce segmentation results stably on datasets of other domains, e.g., CHASE-DB1, STARE, HRF, and DIARETDB1, even in low-shot circumstances.

Generative adversarial minority enlargement—A local linear over-sampling synthetic method

The imbalanced data problem is widely recognized in real-world datasets. To avoid learning bias on imbalanced data, the over-sampling strategy is well studied and adopted for generating synthetic minority samples. Wherein, the Synthetic Minority Over-sampling Technology and its improved algorithms become standard baselines. In recent years, the popular Generative Adversarial Networks and its enhanced variants, introduced from the computer vision community, are believed to generate better samples, by approximating the true data distribution. Yet, we notice that the synthetic samples for the minority category in these existing methods is usually restrained in a limited samples space known in advance, which may mislead the classifiers trained on them to take data in the unsampled region of the minority category as from the majority category. Given such limitations, we propose a Generative Adversarial Minority Enlargement (GAME) method to intentionally extend the sampling margin during data generative and adversarial phases. This is accomplished by adjusting the parameters of a local linear model to approach the majority category. We conduct an extensive evaluation on 28 datasets of different domains, extracted from the UCI real-world datasets. The results show that GAME can achieve more balanced and stable results efficiently than 18 state-of-the-art methods.

An evolutionary filter approach to feature selection in classification for both single- and multi-objective scenarios

The high-dimensional datasets in various domains, such as text categorization, information retrieval and bioinformatics, have highlighted the importance of feature selection in data mining. Despite the numerous existing approaches to feature selection, there is still a need for further research in this field. In this paper, we propose an evolutionary filter feature selection approach that can be used for both single- and multi-objective scenarios by introducing an objective function inspired by Neighborhood Component Analysis (NCA)-based method and then integrating it into the differential evolution framework. The proposed approach applicable to two scenarios aims to identify an optimal feature subset through an evolutionary search process that maximizes class separation while minimizing the dimensionality. Through comprehensive experimental studies conducted on diverse datasets, the results show that the proposed approach outperforms recently proposed evolutionary information-theoretic, rough set-based and state-of-the-art feature selection approaches in both scenarios. Notably, this study is the first to integrate an NCA-based strategy into an evolutionary feature selection approach. Furthermore, you can access the source code of this approach at https://github.com/ehancer06/DENCA this link.

Autonomous complex knowledge mining and graph representation through natural language processing and transfer learning

Regulatory documents play a significant role in securing engineering project quality, standard process management and long-term sustainable developments. With the digitisation of knowledge in the AEC industry, the demand for automated knowledge mining has emerged when confronted with substantial regulations. However, the current interpretation approaches for regulatory documents are still mostly labour-intensive and flawed in complex knowledge. Based on transfer learning (BERT) and natural language processing (e.g., NLP-Syntactic Parsing), this paper proposes a fully automated knowledge mining framework to convert complex knowledge in textual regulations to graph-based knowledge representations. The framework uses a BERT-based engine to extract clauses from regulation documents through fine-tuning with the self-developed domain dataset. A constituent extractor is developed to process the provisions with complex knowledge and extract constituents. A knowledge modelling engine integrates the extracted constituents into a graph-based regulation knowledge model, which can be queried, visualised, and directly applied to downstream applications. The outcome has demonstrated promising performance in complex knowledge mining and knowledge graph modelling based on ISO 19650 case study. This research can effectively convert textual regulation documents to their counterpart regulatory knowledge base, contributing to automated knowledge acquisition and multi-domain knowledge fusion toward regulation digitalization.

Classification and Recognition Method of Non-Cooperative Object Based on Transfer Learning

Classifying and identifying non-cooperative targets plays an important role in space missions in the present age. A micro-Doppler characteristic signal is an effective means to identify the target. In order to better extract the micro-Doppler features of the target in the case of very little experimental data, a pose recognition network based on the theory of transfer learning is proposed. The source domain dataset is constructed of the time–frequency spectrums obtained by short-time Fourier transform of the simulated echo signals. The time–frequency spectrums from experiment are made as target domain. Pre-training and domain adaptation is applied in this work. Finally, the test results demonstrate that the classification accuracy of our pose recognition network based on transfer learning is improved due to the good use of simulation data and the small amount of experimental data.

Cross-domain policy adaptation with dynamics alignment

The implementation of robotic reinforcement learning is hampered by problems such as an unspecified reward function and high training costs. Many previous works have used cross-domain policy transfer to obtain the policy of the problem domain. However, these researches require paired and aligned dynamics trajectories or other interactions with the environment. We propose a cross-domain dynamics alignment framework for the problem domain policy acquisition that can transfer the policy trained in the source domain to the problem domain. Our framework aims to learn dynamics alignment across two domains that differ in agents’ physical parameters (armature, rotation range, or torso mass) or agents’ morphologies (limbs). Most importantly, we learn dynamics alignment between two domains using unpaired and unaligned dynamics trajectories. For these two scenarios, we propose a cross-physics-domain policy adaptation algorithm (CPD) and a cross-morphology-domain policy adaptation algorithm (CMD) based on our cross-domain dynamics alignment framework. In order to improve the performance of policy in the source domain so that a better policy can be transferred to the problem domain, we propose the Boltzmann TD3 (BTD3) algorithm. We conduct diverse experiments on agent continuous control domains to demonstrate the performance of our approaches. Experimental results show that our approaches can obtain better policies and higher rewards for the agents in the problem domains even when the dataset of the problem domain is small.

Metagenomic analysis of soybean endosphere microbiome to reveal signatures of microbes for health and disease

BackgroundSoil metagenomics is a cultivation-independent molecular strategy for investigating and exploiting the diversity of soil microbial communities. Soil microbial diversity is essential because it is critical to sustaining soil health for agricultural productivity and protection against harmful organisms. This study aimed to perform a metagenomic analysis of the soybean endosphere (all microbial communities found in plant leaves) to reveal signatures of microbes for health and disease. ResultsThe dataset is based on the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) release “microbial diversity in soybean”. The quality control process rejected 21 of the evaluated sequences (0.03% of the total sequences). Dereplication determined that 68,994 sequences were artificial duplicate readings, and removed them from consideration. Ribosomal Ribonucleic acid (RNA) genes were present in 72,747 sequences that successfully passed quality control (QC). Finally, we found that hierarchical classification for taxonomic assignment was conducted using MG-RAST, and the considered dataset of the metagenome domain of bacteria (99.68%) dominated the other groups. In Eukaryotes (0.31%) and unclassified sequence 2 (0.00%) in the taxonomic classification of bacteria in the genus group, Streptomyces, Chryseobacterium, Ppaenibacillus, Bacillus, and Mitsuaria were found. We also found some biological pathways, such as CMP-KDO biosynthesis II (from D-arabinose 5-phosphate), tricarboxylic acid cycle (TCA) cycle (plant), citrate cycle (TCA cycle), fatty acid biosynthesis, and glyoxylate and dicarboxylate metabolism. Gene prediction uncovered 1,180 sequences, 15,172 of which included gene products, with the shortest sequence being 131 bases and maximum length 3829 base pairs. The gene list was additionally annotated using Integrated Microbial Genomes and Microbiomes. The annotation process yielded a total of 240 genes found in 177 bacterial strains. These gene products were found in the genome of strain 7598. Large volumes of data are generated using modern sequencing technology to sample all genes in all species present in a given complex sample. ConclusionsThese data revealed that it is a rich source of potential biomarkers for soybean plants. The results of this study will help us to understand the role of the endosphere microbiome in plant health and identify the microbial signatures of health and disease. The MG-RAST is a public resource for the automated phylogenetic and functional study of metagenomes. This is a powerful tool for investigating the diversity and function of microbial communities.