Learned Feature Vectors Research Articles

Deep incomplete multi-view clustering via attention-based direct contrastive learning

Incomplete Multi-View Clustering (IMVC), enhanced by contrastive learning, stands out in unsupervised learning for its notable performance. However, it faces challenges: over-reliance on additional projection heads to avoid dimensionality collapse, leading to redundant parameters, and the risk of encoder-derived features, which merge view-specific information, misleading the learning of common semantics due to simultaneous learning and reconstruction in the same feature space. To address these issues, we propose a novel framework for incomplete multi-view contrastive clustering. This framework employs an encoder network with a self-attention mechanism, allowing both reconstruction loss and contrastive loss to act on the learned feature vectors and their sub-vectors, respectively. This approach effectively mitigates the impact of extraneous private information. By leveraging sub-vectors for consistency learning, our model directly refines the latent feature subspace, thus circumventing dimensionality collapse without the dependence on projection heads. Additionally, our method incorporates a cross-view prediction mechanism to recuperate missing information in incomplete datasets. Comprehensive experiments on public datasets demonstrate that our method achieves state-of-the-art clustering performance.

Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles

Cell and organelle shape are driven by diverse genetic and environmental factors and thus accurate quantification of cellular morphology is essential to experimental cell biology. Autoencoders are a popular tool for unsupervised biological image analysis because they learn a low-dimensional representation that maps images to feature vectors to generate a semantically meaningful embedding space of morphological variation. The learned feature vectors can also be used for clustering, dimensionality reduction, outlier detection, and supervised learning problems. Shape properties do not change with orientation, and thus we argue that representation learning methods should encode this orientation invariance. We show that conventional autoencoders are sensitive to orientation, which can lead to suboptimal performance on downstream tasks. To address this, we develop O2-variational autoencoder (O2-VAE), an unsupervised method that learns robust, orientation-invariant representations. We use O2-VAE to discover morphology subgroups in segmented cells and mitochondria, detect outlier cells, and rapidly characterise cellular shape and texture in large datasets, including in a newly generated synthetic benchmark.

FedPT-V2G: Security enhanced federated transformer learning for real-time V2G dispatch with non-IID data

The rising popularity of electric vehicles (EVs) underscores the potential of vehicle-to-grid (V2G) technology to contribute to load peak-shaving, valley-filling, and photovoltaic (PV) self-consumption. Effective V2G control strategies can be obtained by data-driven techniques, which is able to leverage historical and current data to inform future decision-making amidst uncertainties. However, the centralized collection and sharing of data among charging stations face challenges due to data asset concerns. Furthermore, even if data sharing hurdles are overcome, the non-independent and non-identically distributed (Non-IID) nature of data across charging stations can still negatively impact performance. In this study, we introduce FedPT-V2G, a security-enhanced federated transformer learning approach for real-time V2G dispatch that addresses Non-IID data. We employ deep learning models trained on historical and current data to enable real-time decision-making, facilitating both load shifting and PV self-consumption. Additionally, we utilize federated learning to jointly train a global model across all charging stations without collecting or sharing any local private data. We pioneer the application of the Proximal algorithm and Transformer model to tackle data distribution discrepancies within the V2G scheduling prediction task. The Proximal algorithm employs regularization techniques to align local models at each charging station more closely with the global model during updates. Concurrently, the multi-head attention mechanism within the Transformer model allows learned feature vectors to diverge, enabling better exploitation of variations across the entire feature space. Finally, we validate the proposed FedPT-V2G approach through extensive numerical simulations, demonstrating comparable performance to centralized learning on both balanced (98.93% vs 98.65%) and imbalanced (92.15% vs 92.20% in label skew) datasets.

Baking Neural Radiance Fields for Real-Time View Synthesis.

Neural volumetric representations such as Neural Radiance Fields (NeRF) have emerged as a compelling technique for learning to represent 3D scenes from images with the goal of rendering photorealistic images of the scene from unobserved viewpoints. However, NeRF's computational requirements are prohibitive for real-time applications: rendering views from a trained NeRF requires querying a multilayer perceptron (MLP) hundreds of times per ray. We present a method to train a NeRF, then precompute and store (i.e. "bake") it as a novel representation called a Sparse Neural Radiance Grid (SNeRG) that enables real-time rendering on commodity hardware. To achieve this, we introduce 1) a reformulation of NeRF's architecture, and 2) a sparse voxel grid representation with learned feature vectors. The resulting scene representation retains NeRF's ability to render fine geometric details and view-dependent appearance, is compact (averaging less than 90MB per scene), and can be rendered in real-time (higher than 30 frames per second on a laptop GPU). Actual screen captures are shown in our video.

The prediction of drug sensitivity by multi-omics fusion reveals the heterogeneity of drug response in pan-cancer

Cancer drug response prediction based on genomic information plays a crucial role in modern pharmacogenomics, enabling individualized therapy. Given the expensive and complexity of biological experiments, computational methods serve as effective tools in predicting cancer drug sensitivity. In this study, we proposed a novel method called Multi-Omics Integrated Collective Variational Autoencoders (MOICVAE), which leverages integrated omics knowledge, including genomic and transcriptomic data, to fill in missing cancer-drug associations and enhance drug sensitivity prediction. Our method employs an encoder-decoder network to learn latent feature representations from cell lines. These learned feature vectors are then fed into a collective variational autoencoder network to train an association matrix. We evaluated MOICVAE on the GDSC and CCLE benchmark datasets using 10-fold cross-validation and achieved impressive AUCs of 0.856 and 0.808, respectively, outperforming state-of-the-art methods. Furthermore, on the TCGA dataset, consisting of 25 drugs across 7 cancer types, MOICVAE exhibited an average AUC of 0.91 in predicting drug sensitivity. Additionally, significant differences were observed in survival, tumor inflammatory assessment, and tumor microenvironment between the predicted drug-sensitive and drug-resistant groups. These results are consistent with predictions made on the METABRIC dataset. Moreover, we discovered that fusing omics data based on mRNA and CNV (copy number variations) yielded superior results in drug sensitivity prediction. MOICVAE not only achieved higher accuracy in drug sensitivity prediction but also provided additional value for combining immunotherapy with chemotherapy, offering patients with more precise treatment options. The code and dataset for MOICVAE are freely available at https://github.com/wanggnoc/MOICVAE.

Deep Learning of Image Representations with Convolutional Neural Networks Autoencoder for Image Retrieval with Relevance Feedback

mage retrieval with traditional relevance feedback encounters problems: (1) ability to represent handcrafted features which is limited, and (2) inefficient withhigh-dimensional data such as image data. In this paper,we propose a framework based on very deep convolutionalneural network autoencoder for image retrieval, called AIR(Autoencoders for Image Retrieval). Our proposed frameworkallows to learn feature vectors directly from the raw imageand in an unsupervised manner. In addition, our frameworkutilizes a hybrid approach of unsupervised and supervisedto improve retrieval performance. The experimental resultsshow that our method gives better results than some existingmethods on the CIFAR-100 image set, which consists of 60,000images.

Effective recognition of facial emotions using dual transfer learned feature vectors and support vector machine

Effective recognition of facial emotions using dual transfer learned feature vectors and support vector machine

ADOP

In this paper we present ADOP, a novel point-based, differentiable neural rendering pipeline. Like other neural renderers, our system takes as input calibrated camera images and a proxy geometry of the scene, in our case a point cloud. To generate a novel view, the point cloud is rasterized with learned feature vectors as colors and a deep neural network fills the remaining holes and shades each output pixel. The rasterizer renders points as one-pixel splats, which makes it very fast and allows us to compute gradients with respect to all relevant input parameters efficiently. Furthermore, our pipeline contains a fully differentiable physically-based photometric camera model, including exposure, white balance, and a camera response function. Following the idea of inverse rendering, we use our renderer to refine its input in order to reduce inconsistencies and optimize the quality of its output. In particular, we can optimize structural parameters like the camera pose, lens distortions, point positions and features, and a neural environment map, but also photometric parameters like camera response function, vignetting, and per-image exposure and white balance. Because our pipeline includes photometric parameters, e.g. exposure and camera response function, our system can smoothly handle input images with varying exposure and white balance, and generates high-dynamic range output. We show that due to the improved input, we can achieve high render quality, also for difficult input, e.g. with imperfect camera calibrations, inaccurate proxy geometry, or varying exposure. As a result, a simpler and thus faster deep neural network is sufficient for reconstruction. In combination with the fast point rasterization, ADOP achieves real-time rendering rates even for models with well over 100M points. https://github.com/darglein/ADOP

A Novel Method to Inspect 3D Ball Joint Socket Products Using 2D Convolutional Neural Network with Spatial and Channel Attention.

Product defect inspections are extremely important for industrial manufacturing processes. It is necessary to develop a special inspection system for each industrial product due to their complexity and diversity. Even though high-precision 3D cameras are usually used to acquire data to inspect 3D objects, it is hard to use them in real-time defect inspection systems due to their high price and long processing time. To address these problems, we propose a product inspection system that uses five 2D cameras to capture all inspection parts of the product and a deep learning-based 2D convolutional neural network (CNN) with spatial and channel attention (SCA) mechanisms to efficiently inspect 3D ball joint socket products. Channel attention (CA) in our model detects the most relevant feature maps while spatial attention (SA) finds the most important regions in the extracted feature map of the target. To build the final SCA feature vector, we concatenated the learned feature vectors of CA and SA because they complement each other. Thus, our proposed CNN with SCA provides high inspection accuracy as well as it having the potential to detect small defects of the product. Our proposed model achieved 98% classification accuracy in the experiments and proved its efficiency on product inspection in real-time.

Exploring Sub-Action Granularity for Weakly Supervised Temporal Action Localization

Modeling cross-video relationship is an important issue for the weakly supervised temporal action localization task. To this end, traditional methods operate at the action level and rely on complicated strategies to prepare triplet samples, which only mines the cross-video relationships among three videos from two categories. In this work, we observe that action instances from different categories could exhibit similar motion patterns, i.e. subaction, and propose to operate at the sub-action granularity to elaborately explore cross-video relationships. However, only given video-level category labels, the sub-actions are undefined and not annotated. To tackle this challenge, we represent video features via a group of sub-actions, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> the sub-action family. Specifically, the sub-action family contains multiple feature vectors, where each vector is in charge of representing a specific sub-action. The sub-action family is shared among all videos in the dataset, while all videos contribute to the learning of the sub-action family. Consequently, we can not only get rid of the complicated sampling strategy but also thoroughly mine cross-video relationships from all available videos in the dataset. To learn feature vectors within the sub-action family, we employ a bottom-up temporal action localization paradigm and introduce an extra top-down branch. The sub-action family is introduced into the top-down branch, and it learns feature vectors via representing raw video features. Moreover, we propose a consistency loss to guide the learning process and a diversity loss to mine distinct sub-actions. Extensive experiments are carried out on three benchmark datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> THUMOS14, ActivityNet v1.2 and ActivityNet v1.3, and the proposed method builds new high performance.

Multi-scale Salient Features for Person Re-identification

Person re-identification is a technology that uses computer vision to match and retrieve the same target person across cameras, which has a wide range of applications in the fields of intelligent video surveillance, intelligent security, and unmanned driving. Aiming at the problem of low accuracy caused by complex background information, occlusion and other factors in person re-identification, the extracted features are not discriminative and robust, resulting in low accuracy, a person re-identification method based on multi-scale saliency features is proposed. Firstly, the saliency detection is used to extract the distinctive feature areas with discriminative power in person, so that the features of distinguishing person inages are more prominent. Secondly, these saliency features are merged with global features and the fused features are divided into different scales, and the global features are divided into three different scales to enhance the expression of features. Finally, according to the difference of global and local features, three kinds of loss functions are combined for learning. In the inference stage, the global feature and the local feature are fused into a new feature vector for metric learning. The proposed method has been used in the person re-identification public datasets Market1501 and DukeMTMC-reID to do a lot of effective verification experiments. The experimental results show that the features extracted by the proposed method have strong distinguishability and robustness.

Semi-supervised two-phase familial analysis of Android malware with normalized graph embedding

With the widespread use of smartphones, Android malware has posed serious threats to its security. Given the explosive growth of Android malware variants, detecting malware families are crucial for identifying new security threats, triaging, and building reference datasets. Building behavior profiles of Android applications (apps) with holistic graph-based features would help to retain program semantics and resist obfuscation. It is more effective to use representation with the low-dimensional feature, which could reduce calculation cost and improve the efficiency of downstream analytics tasks. To achieve this goal, we design and develop a practical system for the familial analysis of Android malware named GSFDroid. We first use graph-based features that contain structural information to analyze app behavior. Then, we employ Graph Convolutional Networks (GCNs) to embed nodes into a continuous and low-dimensional space, which improves the efficiency of downstream analytics tasks. Note that distributions of the learned feature vectors of APKs are not aligned and centered caused by the random initialization and propagation strategy of GCN, whose different scales can harm the performance of downstream tasks. Inspired by the z-score, we propose a simple graph feature normalization to standardize the embedded APK features. Finally, instead of fully supervised or unsupervised learning, we propose a two-phased familial analysis method fusing a semi-supervised classifier with a cluster operation on high uncertain score samples respect to the classifier. Promising experimental results based on real-world datasets demonstrate that our approach significantly outperforms state-of-the-art approaches, and can effectively cluster new malware samples from unknown families.

One-dimension hierarchical local receptive fields based extreme learning machine for radar target HRRP recognition

Radar automatic target recognition (RATR) aims at extracting meaningful target features from the electromagnetic echo signal and utilizing the features to automatically recognize the target types. The high-resolution range profile (HRRP) plays an important role in RATR field, HRRP is the amplitude of the echo summation for target scattering centers in each range cell of wideband radar. Using deep neural networks for HRRP radar target recognition encounters the problem of storage overhead and slow convergence rate, to resolve those issues, we propose a one-dimension local receptive fields based extreme learning auto-encoder (1D ELM-LRF-AE) network for HRRP local structures and meaningful representations learning. ELM-LRF-AE consists of an input layer, a random convolution layer, a pooling layer, several local connected layers and an output layer, it reconstructs the input with a greedy strategy that the input feature vectors are divided into several subgroups and the i-th pooling feature vector is used to reconstruct the i-th grouping input feature vector. Then we use the learned pooling feature vectors to replace the random pooling feature vectors as the learned representations. We also stack several 1D ELM-LRF-AEs to build 1D hierarchical local receptive fields based extreme learning machine (1D H-ELM-LRF) for high level HRRP abstract representations learning and recognition. Experimental results on simulated HRRP data set demonstrate the superior high recognition performance and high computational efficiency of our algorithm.

Deobfuscation, unpacking, and decoding of obfuscated malicious JavaScript for machine learning models detection performance improvement

Obfuscation is rampant in both benign and malicious JavaScript (JS) codes. It generates an obscure and undetectable code that hinders comprehension and analysis. Therefore, accurate detection of JS codes that masquerade as innocuous scripts is vital. The existing deobfuscation methods assume that a specific tool can recover an original JS code entirely. For a multi-layer obfuscation, general tools realize a formatted JS code, but some sections remain encoded. For the detection of such codes, this study performs Deobfuscation, Unpacking, and Decoding (DUD-preprocessing) by function redefinition using a Virtual Machine (VM), a JS code editor, and a python int_to_str() function to facilitate feature learning by the FastText model. The learned feature vectors are passed to a classifier model that judges the maliciousness of a JS code. In performance evaluation, the authors use the Hynek Petrak's dataset for obfuscated malicious JS codes and the SRILAB dataset and the Majestic Million service top 10,000 websites for obfuscated benign JS codes. They then compare the performance to other models on the detection of DUD-preprocessed obfuscated malicious JS codes. Their experimental results show that the proposed approach enhances feature learning and provides improved accuracy in the detection of obfuscated malicious JS codes.

Deep supervised multimodal semantic autoencoder for cross‐modal retrieval

AbstractCross‐modal retrieval aims to do flexible retrieval among different modals, whose main issue is how to measure the semantic similarities among multimodal data. Though many existing methods have been proposed to enable cross‐modal retrieval, they rarely consider the preservation of content information among multimodal data. In this paper, we present a three‐stage cross‐modal retrieval method, named MMCA‐CMR. To reduce the discrepancy among multimodal data, we first attempt to embed multimodal data into a common representation space. We then combine the feature vectors with the content information into the semantic‐aware feature vectors. We finally obtain the feature‐aware and content‐aware projections via multimodal semantic autoencoders. With semantic deep autoencoders, MMCA‐CMR promotes a more reliable cross‐modal retrieval by learning feature vectors from different modalities and content information simultaneously. Extensive experiments demonstrate that the proposed method is valid in cross‐modal retrieval, which significantly outperforms state‐of‐the‐art on four widely‐used benchmark datasets.

Learning Triple Embeddings from Knowledge Graphs

Graph embedding techniques allow to learn high-quality feature vectors from graph structures and are useful in a variety of tasks, from node classification to clustering. Existing approaches have only focused on learning feature vectors for the nodes and predicates in a knowledge graph. To the best of our knowledge, none of them has tackled the problem of directly learning triple embeddings. The approaches that are closer to this task have focused on homogeneous graphs involving only one type of edge and obtain edge embeddings by applying some operation (e.g., average) on the embeddings of the endpoint nodes. The goal of this paper is to introduce Triple2Vec, a new technique to directly embed knowledge graph triples. We leverage the idea of line graph of a graph and extend it to the context of knowledge graphs. We introduce an edge weighting mechanism for the line graph based on semantic proximity. Embeddings are finally generated by adopting the SkipGram model, where sentences are replaced with graph walks. We evaluate our approach on different real-world knowledge graphs and compared it with related work. We also show an application of triple embeddings in the context of user-item recommendations.

A neural multi-context modeling framework for personalized attraction recommendation

In attraction recommendation scenarios, how to model multifaceted tourism contexts so as to accurately learn tourist preferences and attraction tourism features is a keystone of generating personalized recommendations. However, most of existing works generally focused on modeling spatiotemporal contexts of historical travel trajectories to learn tourists’ preferences, while neglected rich heterogeneous tourism side information, i.e., personal tourism constraints of tourists and tourism attributes of attractions. To this end, we propose a Neural Multi-context Modeling Framework (NMMF) to learn tourism feature representations of tourists and attractions by modeling multiple tourism contexts. Initially, we leverage a travel knowledge graph and massive original travelogues to construct the tourism attribute context of attractions and the travel trajectory context of tourists. Then, we design two context embedding models, named TKG2vec and Traj2vec, to model two kinds of context respectively. Both models learn feature vectors of tourist and attraction in contexts by elaborating neural networks to project each tourist and attraction into a uniform latent feature space. Finally, our framework integrates feature vectors derived from two models to acquire complete feature representations of tourists and attractions, and recommends personalized attractions by calculating the similarity between tourist and candidate attractions in the latent space. Experimental results on a real-world tourism dataset demonstrate our framework outperforms state-of-the-art methods in two personalized attraction recommendation tasks.

Automated risk assessment for material movement in manufacturing

Abstract: Proximity movements between vehicles transporting materials in manufacturing plants, or “interfaces”, result in occupational injuries and fatalities. Risk assessment for interfaces is currently limited to techniques such as safety audits, originally designed for static environments. A data-driven alternative for dynamic environments is desirable to quantify interface risks and to enable the development of effective countermeasures. We present a method to estimate the Risk Prioritization Number (RPN) for mobile vehicle interfaces in manufacturing environments, based on the Probability-Severity-Detectability (PSD) formulation. The highlight of the method is the estimation of the probability of occurrence (P) of vehicle interfaces using machine learning and computer vision techniques. A PCA-based sparse feature vector for machine learning characterizes vehicle geometry from a top-down perspective. Supervised classification on sparse feature vectors using Support Vector Machines (SVMs) is employed to detect vehicles. Computer vision techniques are used for position tracking to identify interfaces and to calculate their probability of occurrence (P). This leads to an automated calculation of RPN based on the PSD formulation. Experimental data is collected in the laboratory using a sample work area layout and scale versions of vehicles. Vehicle interfaces and movements were physically simulated to train and test the machine learning model. The performance of the automated system is compared with human annotation to validate the approach.

Topo2Vec: A Novel Node Embedding Generation Based on Network Topology for Link Prediction

Link prediction of a scale-free network has become relevant for problems relating to social network analysis, recommendation system, and in the domain of bioinformatics. In recently proposed approaches, the sampling of nodes of a network is done by simulating random walk. The generated node samples are used to train a neural network to learn the contextual information through an embedding vector. This method has gained popularity as the embedding vector is produced from the primitive node adjacency information and has achieved an outstanding performance. In this article, a naive and scalable approach for generating the node samples based on the principle of goal-oriented greedy searching has been proposed. The generated node samples have low noisy structures, which can represent the relation of edges of the network in a better way, compared to the state-of-the-art methods. Consequently, better representation of feature embedding of nodes of the network is generated from the samples. The learned feature vectors are used for solving the link prediction problem using a pairwise kernel support vector machine (SVM) classifier, which is computationally and spatially expensive in nature. Therefore, as an alternative, we have chosen to deploy a random forest (RF) classifier with a new algebraic operation to obtain the symmetric pairwise feature representation of a node pair. We demonstrated the efficacy of the proposed Topo2vec by testing it against the state-of-the-art network context generation algorithms in several real-world networks. Altogether, the proposed Topo2vec algorithm is a new method for solving the link prediction problem with entirely different settings.

Byte2vec: Malware Representation and Feature Selection for Android

AbstractMalware detection based on static features and without code disassembling is a challenging path of research. Obfuscation makes the static analysis of malware even more challenging. This paper extends static malware detection beyond byte level $n$-grams and detecting important strings. We propose a model (Byte2vec) with the capabilities of both binary file feature representation and feature selection for malware detection. Byte2vec embeds the semantic similarity of byte level codes into a feature vector (byte vector) and also into a context vector. The learned feature vectors of Byte2vec, using skip-gram with negative-sampling topology, are combined with byte-level term-frequency (tf) for malware detection. We also show that the distance between a feature vector and its corresponding context vector provides a useful measure to rank features. The top ranked features are successfully used for malware detection. We show that this feature selection algorithm is an unsupervised version of mutual information (MI). We test the proposed scheme on four freely available Android malware datasets including one obfuscated malware dataset. The model is trained only on clean APKs. The results show that the model outperforms MI in a low-dimensional feature space and is competitive with MI and other state-of-the-art models in higher dimensions. In particular, our tests show very promising results on a wide range of obfuscated malware with a false negative rate of only 0.3% and a false positive rate of 2.0%. The detection results on obfuscated malware show the advantage of the unsupervised feature selection algorithm compared with the MI-based method.