Embedding Algorithm Research Articles

USteg-DSE: Universal quantitative Steganalysis framework using Densenet merged with Squeeze & Excitation net

Carrying concealed communication via media is termed as steganography and unraveling details of such covert transmission is known as steganalysis. Extracting details of hidden message like length, position, embedding algorithm etc. forms part of forensic steganalysis. Predicting length of payload in camouflaged interchange is termed as quantitative steganalysis and is an indispensable tool for forensic investigators. When payload length is estimated without any prior knowledge about cover media or used steganography algorithm, it is termed as universal quantitative steganalysis.Most of existing frameworks on quantitative steganalysis available in literature, work for a specific embedding algorithm or are domain specific. In this paper we propose and present USteg-DSE, a deep learning framework for performing universal quantitative image steganalysis using DenseNet with Squeeze & Excitation module (SEM). In deep learning techniques, deeper networks easily capture complex statistical properties. But as depth increases, networks suffer from vanishing gradient problem. In classic architectures, all channels are equally weighted to produce feature maps. Presented USteg-DSE framework overcomes these problems by using DenseNet and SEM. In DenseNet, each layer is directly connected with every other layer. DenseNet makes information and gradient flow easier with fewer feature maps. SEM incorporates content aware mechanism to adaptively regulate weight for every feature map. Presented framework has been compared with existing state-of-the-art techniques for spatial domain as well as transform domain and show better results in terms of Mean Absolute Error (MAE) and Mean Square Error (MSE).

Basic research for identification and classification of organophosphorus pesticides in water based on ultraviolet-visible spectroscopy information.

In this study, the goal was to develop a method for detecting and classifying organophosphorus pesticides (OPPs) in bodies of water. Sixty-five samples with different concentrations were prepared for each of the organophosphorus pesticides, namely chlorpyrifos, acephate, parathion-methyl, trichlorphon, dichlorvos, profenofos, malathion, dimethoate, fenthion, and phoxim, respectively. Firstly, the spectral data of all the samples was obtained using a UV-visible spectrometer. Secondly, five preprocessing methods, six manifold learning methods, and five machine learning algorithms were utilized to build detection models for identifying OPPs in water bodies. The findings indicate that the accuracy of machine learning models trained on data preprocessed using convolutional smoothing + first-order derivatives (SG + FD) outperforms that of models trained on data preprocessed using other methods. The backpropagation neural network (BPNN) model exhibited the highest accuracy rate at 99.95%, followed by the support vector machine (SVM) and convolutional neural network (CNN) models, both at 99.92%. The extreme learning machine (ELM) and K-nearest neighbors (KNN) models demonstrated accuracy rates of 99.84% and 99.81%, respectively. Following the application of a manifold learning algorithm to the full-wavelength data set for the purpose of dimensionality reduction, the data was then visualized in the first three dimensions. The results demonstrate that the t-distributed domain embedding (t-SNE) algorithm is superior, exhibiting dense clustering of similar clusters and clear classification of dissimilar ones. SG + FD-t-SNE-SVM ranks highest among the feature extraction models in terms of performance. The feature extraction dimension was set to 4, and the average classification accuracy was 99.98%, which slightly improved the prediction performance over the full-wavelength model. As shown in this study, the ultraviolet-visible (UV-visible) spectroscopy system combined with the t-SNE and SVM algorithms can effectively identify and classify OPPs in waterbodies.

Similarity measure method of near-infrared spectrum combined with multi-attribute information

Due to the high-dimensionality, redundancy, and non-linearity of the near-infrared (NIR) spectra data, as well as the influence of attributes such as producing area and grade of the sample, which can all affect the similarity measure between samples. This paper proposed a t-distributed stochastic neighbor embedding algorithm based on Sinkhorn distance (St-SNE) combined with multi-attribute data information. Firstly, the Sinkhorn distance was introduced which can solve problems such as KL divergence asymmetry and sparse data distribution in high-dimensional space, thereby constructing probability distributions that make low-dimensional space similar to high-dimensional space. In addition, to address the impact of multi-attribute features of samples on similarity measure, a multi-attribute distance matrix was constructed using information entropy, and then combined with the numerical matrix of spectral data to obtain a mixed data matrix. In order to validate the effectiveness of the St-SNE algorithm, dimensionality reduction projection was performed on NIR spectral data and compared with PCA, LPP, and t-SNE algorithms. The results demonstrated that the St-SNE algorithm effectively distinguishes samples with different attribute information, and produced more distinct projection boundaries of sample category in low-dimensional space. Then we tested the classification performance of St-SNE for different attributes by using the tobacco and mango datasets, and compared it with LPP, t-SNE, UMAP, and Fisher t-SNE algorithms. The results showed that St-SNE algorithm had the highest classification accuracy for different attributes. Finally, we compared the results of searching the most similar sample with the target tobacco for cigarette formulas, and experiments showed that the St-SNE had the highest consistency with the recommendation of the experts than that of the other algorithms. It can provide strong support for the maintenance and design of the product formula.

Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention

Detecting defects on a steel surface is crucial for the quality enhancement of steel, but its effectiveness is impeded by the limited number of high-quality samples, diverse defect types, and the presence of interference factors such as dirt spots. Therefore, this article proposes a fine-tuned deep learning approach to overcome these obstacles in unstructured few-shot settings. Initially, to address steel surface defect complexities, we integrated a serial multi-scale attention mechanism, concatenating attention and spatial modules, to generate feature maps that contain both channel information and spatial information. Further, a pseudo-label semi-supervised learning algorithm (SSL) based on a variant of the locally linear embedding (LLE) algorithm was proposed, enhancing the generalization capability of the model through information from unlabeled data. Afterwards, the refined model was merged into a fine-tuned few-shot object detection network, which applied extensive base class samples for initial training and sparsed new class samples for fine-tuning. Finally, specialized datasets considering defect diversity and pixel scales were constructed and tested. Compared with conventional methods, our approach improved accuracy by 5.93% in 7-shot detection tasks, markedly reducing manual workload and signifying a leap forward for practical applications in steel defect detection.

A vast space of compact strategies for effective decisions.

Inference-based decision-making, which underlies a broad range of behavioral tasks, is typically studied using a small number of handcrafted models. We instead enumerate a complete ensemble of strategies that could be used to effectively, but not necessarily optimally, solve a dynamic foraging task. Each strategy is expressed as a behavioral "program" that uses a limited number of internal states to specify actions conditioned on past observations. We show that the ensemble of strategies is enormous-comprising a quarter million programs with up to five internal states-but can nevertheless be understood in terms of algorithmic "mutations" that alter the structure of individual programs. We devise embedding algorithms that reveal how mutations away from a Bayesian-like strategy can diversify behavior while preserving performance, and we construct a compositional description to link low-dimensional changes in algorithmic structure with high-dimensional changes in behavior. Together, this work provides an alternative approach for understanding individual variability in behavior across animals and tasks.

Node2vec2rank: Large Scale and Stable Graph Differential Analysis via Multi-Layer Node Embeddings and Ranking.

Computational methods in biology can infer large molecular interaction networks from multiple data sources and at different resolutions, creating unprecedented opportunities to explore the mechanisms driving complex biological phenomena. Networks can be built to represent distinct conditions and compared to uncover graph-level differences-such as when comparing patterns of gene-gene interactions that change between biological states. Given the importance of the graph comparison problem, there is a clear and growing need for robust and scalable methods that can identify meaningful differences. We introduce node2vec2rank (n2v2r), a method for graph differential analysis that ranks nodes according to the disparities of their representations in joint latent embedding spaces. Improving upon previous bag-of-features approaches, we take advantage of recent advances in machine learning and statistics to compare graphs in higher-order structures and in a data-driven manner. Formulated as a multi-layer spectral embedding algorithm, n2v2r is computationally efficient, incorporates stability as a key feature, and can provably identify the correct ranking of differences between graphs in an overall procedure that adheres to veridical data science principles. By better adapting to the data, node2vec2rank clearly outperformed the commonly used node degree in finding complex differences in simulated data. In the real-world applications of breast cancer subtype characterization, analysis of cell cycle in single-cell data, and searching for sex differences in lung adenocarcinoma, node2vec2rank found meaningful biological differences enabling the hypothesis generation for therapeutic candidates. Software and analysis pipelines implementing n2v2r and used for the analyses presented here are publicly available.

Pixel-MPS: Stochastic Embedding and Density-Based Clustering of Image Patterns for Pixel-Based Multiple-Point Geostatistical Simulation

Multiple-point geostatistics (MPS) is an established tool for the uncertainty quantification of Earth systems modeling, particularly when dealing with the complexity and heterogeneity of geological data. This study presents a novel pixel-based MPS method for modeling spatial data using advanced machine-learning algorithms. Pixel-based multiple-point simulation implies the sequential modeling of individual points on the simulation grid, one at a time, by borrowing spatial information from the training image and honoring the conditioning data points. The developed methodology is based on the mapping of the training image patterns database using the t-Distributed Stochastic Neighbor Embedding (t-SNE) algorithm for dimensionality reduction, and the clustering of patterns by applying the Density-based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, as an efficient unsupervised classification technique. For the automation, optimization, and input parameter tuning, multiple stages are implemented, including entropy-based determination of the template size and a k-nearest neighbors search for clustering parameter selection, to ensure the proposed method does not require the user’s interference. The proposed model is validated using synthetic two- and three-dimensional datasets, both for conditional and unconditional simulations, and runtime information is provided. Finally, the method is applied to a case study gold mine for stochastic orebody modeling. To demonstrate the computational efficiency and accuracy of the proposed method, a two-dimensional training image with 101 by 101 pixels is simulated for 100 conditional realizations in 453 s (~4.5 s per realization) using only 361 hard data points (~3.5% of the simulation grid), and the resulting average simulation has a good visual match and only an 11.8% pixel-wise mismatch with the training image.

A Spectral Clustering Algorithm for Non-Linear Graph Embedding in Information Networks

With the development of network technology, information networks have become one of the most important means for people to understand society. As the scale of information networks expands, the construction of network graphs and high-dimensional feature representation will become major factors affecting the performance of spectral clustering algorithms. To address this issue, in this paper, we propose a spectral clustering algorithm based on similarity graphs and non-linear deep embedding, named SEG_SC. This algorithm introduces a new spectral clustering model that explores the underlying structure of graphs through sparse similarity graphs and deep graph representation learning, thereby enhancing graph clustering performance. Experimental analysis with multiple types of real datasets shows that the performance of this model surpasses several advanced benchmark algorithms and performs well in clustering on medium- to large-scale information networks.

Privacy-preserving outsourcing scheme of face recognition based on locally linear embedding

Face recognition is a biometric-based technology that identifies or verifies a person’s identity by analyzing and comparing patterns in their facial features. Among kinds of face recognition algorithms, Locally Linear Embedding (LLE) algorithm has the advantage of preserving the local structure of face data. However, the LLE algorithm requires some computationally intensive matrix calculations, which might introduce latency to the device. In this paper, we propose a privacy-preserving outsourcing scheme of face recognition based on LLE algorithm for the first time. By outsourcing matrix multiplication, linear equations, and eigenvalue decomposition of the LLE algorithm to a cloud server, we reduce the computational load on the client device. To ensure data security, we generate a novel type of sparse orthogonal matrix through square root application and random permutation of diagonal elements, used as secret keys to blind the client’s data. Meanwhile, due to the sparsity of key matrices, the complexity of client-side computation complexity is reduced from O(n3) to O(n2). The experimental results show that our proposed scheme reduces computational costs while maintaining almost identical face recognition accuracy as the original LLE-based algorithm.

Automatic grid topology detection method based on Lasso algorithm and t-SNE algorithm

For a long time, the low-voltage distribution network has the problems of untimely management and complex and frequently changing lines, which makes the problem of missing grid topology information increasingly serious. This study proposes an automatic grid topology detection model based on lasso algorithm and t-distributed random neighbor embedding algorithm. The model identifies the household-variable relationship through the lasso algorithm, and then identifies the grid topology of the station area through the t-distributed random neighbor embedding algorithm model. The experimental results indicated that the lasso algorithm, the constant least squares algorithm and the ridge regression algorithm had accuracies of 0.88, 0.80, and 0.71 and loss function values of 0.14, 0.20, and 0.25 for dataset sizes up to 500. Comparing the time spent on identifying household changes in different regions, in Region 1, the training time for the Lasso algorithm, the Constant Least Squares algorithm, and the Ridge Regression algorithm is 2.8 s, 3.0 s, and 3.1 s, respectively. The training time in region 2 is 2.4s, 3.6s, and 3.4s, respectively. The training time in region 3 is 7.7 s, 1.9 s, and 2.8 s, respectively. The training time in region 4 is 3.1 s, 3.6 s, and 3.3 s, respectively. The findings demonstrate that the suggested algorithmic model performs better than the other and can identify the structure of LV distribution networks.

MathEagle: Accurate prediction of drug-drug interaction events via multi-head attention and heterogeneous attribute graph learning

BackgroundDrug-drug interaction events influence the effectiveness of drug combinations and can lead to unexpected side effects or exacerbate underlying diseases, jeopardizing patient prognosis. Most existing methods are restricted to predicting whether two drugs interact or the type of drug-drug interactions, while very few studies endeavor to predict the specific risk levels of side effects of drug combinations. MethodsIn this study, we propose MathEagle, a novel approach to predict accurate risk levels of drug combinations based on multi-head attention and heterogeneous attribute graph learning. Initially, we model drugs and three distinct risk levels between drugs as a heterogeneous information graph. Subsequently, behavioral and chemical structure features of drugs are utilized by message passing neural networks and graph embedding algorithms, respectively. Ultimately, MathEagle employs heterogeneous graph convolution and multi-head attention mechanisms to learn efficient latent representations of drug nodes and estimates the risk levels of pairwise drugs in an end-to-end manner. ResultsTo assess the effectiveness and robustness of the model, five-fold cross-validation, ablation experiments, and case studies were conducted. MathEagle achieved an accuracy of 85.85 % and an AUC of 0.9701 on the drug risk level prediction task and is superior to all comparative models. The MathEagle predictor is freely accessible at http://120.77.11.78/MathEagle/. ConclusionsThe experimental results indicate that MathEagle can function as an effective tool for predicting accurate risk of drug combinations, aiding in guiding clinical medication, and enhancing patient outcomes.

The axes of biology: a novel axes-based network embedding paradigm to decipher the functional mechanisms of the cell.

Common approaches for deciphering biological networks involve network embedding algorithms. These approaches strictly focus on clustering the genes' embedding vectors and interpreting such clusters to reveal the hidden information of the networks. However, the difficulty in interpreting the genes' clusters and the limitations of the functional annotations' resources hinder the identification of the currently unknown cell's functioning mechanisms. We propose a new approach that shifts this functional exploration from the embedding vectors of genes in space to the axes of the space itself. Our methodology better disentangles biological information from the embedding space than the classic gene-centric approach. Moreover, it uncovers new data-driven functional interactions that are unregistered in the functional ontologies, but biologically coherent. Furthermore, we exploit these interactions to define new higher-level annotations that we term Axes-Specific Functional Annotations and validate them through literature curation. Finally, we leverage our methodology to discover evolutionary connections between cellular functions and the evolution of species. Data and source code can be accessed at https://gitlab.bsc.es/sdoria/axes-of-biology.git.

Gaussian-kernel weighted neighborhood preserving embedding algorithm and its application in fault detection

Fault detection in industrial processes is essential for enhancing production safety. Despite the application of the neighborhood preserving embedding (NPE) algorithm in fault detection as a manifold learning technique, a notable limitation exists-NPE overlooks local geometric structure, leading to suboptimal fault detection and occasional false alarms. This paper introduces the Gaussian kernel weighted NPE (KW-NPE) algorithm to address this challenge. Specifically designed for precise weight assignment in local structures, KW-NPE strategically employs the Gaussian kernel method to project the spatial neighborhood set and capture comprehensive local structural characteristics. The weight assignment, dependent on feature values, enhances the retention of intrinsic structure during dimensionality reduction. A novel objective function further augments this process.To assess performance, a comprehensive composite index is introduced in a case study, amalgamating the false alarm rate and fault detection rate. The effectiveness of the KW-NPE algorithm is demonstrated through extensive simulations and its application to the Tennessee Eastman process dataset, highlighting its superiority over conventional approaches.

Pulsar candidate identification using advanced transformer-based models

Rapid and accurate identification of pulsars is a significant topic for large radio telescope surveys. With the enhancement of astronomical instruments, modern radio telescopes are witnessing an exponential increase in pulsar candidate detections. The application of artificial intelligence for the identification of pulsar candidates is an automated and highly effective solution to tackle the challenge of processing and recognizing vast volumes of data. In this work, using the data released by two surveys, the Commensal Radio Astronomy FasT Survey (CRAFTS) and High-Time Resolution Universe (HTRU), we propose a new framework to identify pulsar candidates. Firstly, due to the small number of real pulsars, we compare the performance of different data augmentation methods and find that the pulsar samples generated by the Deep Convolutional Generative Adversarial Network (DCGAN) based on deep learning techniques are closer to real pulsars. Secondly, we use two transformer-based classification models, Vision Transformer (ViT) and Convolutional Vision Transformer (CvT), to classify pulsar candidates, and find that the evaluation indexes of pulsar candidate classification based on two transformers can reach 100%. Finally, we use the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm to visualize the results of our identification framework. The results showed that pulsar and non-pulsar samples are separated from each other in multidimensional space. Therefore, it is a new attempt to apply transformer technology to pulsar candidate classification, and it could be of great significance to subsequent theoretical research.

Reversible Image Fragile Watermarking with Dual Tampering Detection

The verification of image integrity has attracted increasing attention. Irreversible algorithms embed fragile watermarks into cover images to verify their integrity, but they are not reversible due to unrecoverable loss. In this paper, a new dual tampering detection scheme for reversible image fragile watermarking is proposed. The insect matrix reversible embedding algorithm is used to embed the watermark into the cover image. The cover image can be fully recovered when the dual-fragile-watermarked images are not tampered with. This study adopts two recovery schemes and adaptively chooses the most appropriate scheme to recover tampered data according to the square errors between the tampered data and the recovered data of two watermarked images. Tampering coincidence may occur when a large region of the fragile-watermarked image is tampered with, and the recovery information corresponding to the tampered pixels may be missing. The tampering coincidence problem is solved using image-rendering techniques. The experimental results show that the PSNR value of the watermarked image obtained using our scheme can reach 46.37 dB, and the SSIM value is 0.9942. In addition, high-accuracy tampering detection is achieved.

Predicting Alloying Element Yield in Converter Steelmaking Using t-SNE-WOA-LSTM

The performance and quality of steel products are significantly impacted by the alloying element control. The efficiency of alloy utilization in the steelmaking process was directly related to element yield. This study analyses the factors that influence the yield of elements in the steelmaking process using correlation analysis. A yield prediction model was developed using a t-distributed stochastic neighbor embedding (t-SNE) algorithm, a whale optimization algorithm (WOA), and a long short-term memory (LSTM) neural network. The t-SNE algorithm was used to reduce the dimensionality of the original data, while the WOA optimization algorithm was employed to optimize the hyperparameters of the LSTM neural network. The t-SNE-WOA-LSTM model accurately predicted the yield of Mn and Si elements with hit rates of 71.67%, 96.67%, and 99.17% and 57.50%, 89.17%, and 97.50%, respectively, falling within the error range of ±1%, ±2%, and ±3% for Mn and ±1%, ±3%, and ±5% for Si. The results demonstrate that the t-SNE-WOA-LSTM model outperforms the backpropagation (BP), LSTM, and WOA-LSTM models in terms of prediction accuracy. The model was applied to actual production in a Chinese plant. The actual performance of the industrial application is within a ±3% error range, with an accuracy of 100%. Furthermore, the elemental yield predicted by the model and then added the ferroalloys resulted in a reduction in the elemental content of the product by 0.017%. The model enables accurate prediction of alloying element yields and was effectively applied in industrial production.

Scalable and unbiased sequence-informed embedding of single-cell ATAC-seq data with CellSpace

Standard scATAC sequencing (scATAC-seq) analysis pipelines represent cells as sparse numeric vectors relative to an atlas of peaks or genomic tiles and consequently ignore genomic sequence information at accessible loci. Here we present CellSpace, an efficient and scalable sequence-informed embedding algorithm for scATAC-seq that learns a mapping of DNA k-mers and cells to the same space, to address this limitation. We show that CellSpace captures meaningful latent structure in scATAC-seq datasets, including cell subpopulations and developmental hierarchies, and can score transcription factor activities in single cells based on proximity to binding motifs embedded in the same space. Importantly, CellSpace implicitly mitigates batch effects arising from multiple samples, donors or assays, even when individual datasets are processed relative to different peak atlases. Thus, CellSpace provides a powerful tool for integrating and interpreting large-scale scATAC-seq compendia.

Text steganography: enhanced character-level embedding algorithm using font attribute with increased resilience to statistical attacks

Text steganography: enhanced character-level embedding algorithm using font attribute with increased resilience to statistical attacks

VCGERG

Vulnerability can lead to data loss, privacy leakage and financial loss. Accurate detection and identification of vulnerabilities is essential to prevent information leakage and APT attacks. This paper explores the possibility of digging the valuable information in vulnerability reports deeply. We propose a new model, VCGERG, which products a graph using key information from vulnerability reports and embeds the graph into the vector space using a keywords-LINE graph embedding algorithm based on the attention of neighboring nodes. VCGERG model uses the OVR random forest algorithm to classify vulnerabilities. Our model can get the complicated local and global information of the graph in large-scale dataset and achieve better results. In order to verify the effectiveness of our model, it is evaluated on many experiments. Compared with other models, our method has a higher accuracy rate of 0.975.

Immunaut: Immunophenotype Analysis with Machine Learning for LAIV Vaccine Response Prediction

Abstract The Immunaut approach represents a groundbreaking application of machine learning in immunology, focusing on predicting immune responses to Live Attenuated Influenza Vaccines (LAIV). This novel method combines unsupervised machine learning techniques, including t-distributed Stochastic Neighbor Embedding (tSNE) and advanced clustering algorithms, with the Sequential Iterative Modeling OverNight (SIMON) platform for predictive analysis. By identifying and categorizing immunophenotypic groups based on baseline features, Immunaut offers a sophisticated, data-driven means of enhancing our understanding and prediction of vaccine efficacy. This approach promises to pave the way for more personalized vaccine strategies and a deeper insight into immune response mechanisms.