Transformed Feature Vectors Research Articles

Memory and Communication Efficient Federated Kernel k-Means.

A federated kernel k -means (FedKKM) algorithm is developed in this article to conduct distributed clustering with low memory consumption on user devices. In FedKKM, a federated eigenvector approximation (FEA) algorithm is designed to iteratively determine the low-dimensional approximate vectors of the transformed feature vectors, using only low-dimensional random feature vectors. To maintain high communication efficiency in each iteration of FEA, a communication-efficient Lanczos algorithm (CELA) is further designed in FEA to reduce the communication cost. Based on the low-dimensional approximate vectors, the clustering result is obtained by leveraging a distributed linear k -means algorithm. A theoretical analysis shows that: 1) FEA has a convergence rate of O(1/T) , where T is the number of iterations; 2) the scalability of FedKKM is not affected by the dataset size since the communication cost of FedKKM is independent of the number of users' data; and 3) FedKKM is a (1+ϵ) approximation algorithm. The experimental results show that FedKKM achieves the comparable clustering quality to that of a centralized kernel k -means. Compared with state-of-the-art schemes, FedKKM reduces the memory consumption on user devices by up to 94% and also reduces the communication cost by more than 40%.

Abstract 3526: GAIN-BRCA: A graphical explainable AI-net framework for breast cancer subtype classification using multiomics data

Abstract Higher standards for precise distinction and classification among the four breast cancer subtypes, i.e., Luminal A. Luminal B, HER-2-enriched, and Basal, are essential to delivering customized treatment in precision oncology. The effective integration of readily accessible multi-omics datasets from the same patient may improve the accuracy of subtype prediction algorithms and help decipher reliable molecular features for the prediction and prognosis of breast cancer. Previous models for predicting breast cancer subtypes relied mostly on concatenation and autoencoder methods but rarely utilized graph-based integration to leverage the biological associations among omics datatypes. To overcome these limitations, we developed a graph-based multi-omics integration method using features from mRNA, DNA methylation, and miRNA data and synthesized features from their interactions. GAIN-BRCA computes weightage scores from miRNA-mRNA and DNA methylation-mRNA interactions to derive a new transformed feature vector without compromising biological information. The neural network architecture was designed and trained on the transformed features to classify breast cancer subtypes. Neural network prediction accuracies were compared using concatenated, autoencoder, and GAIN-BRCA-based integrated datasets. GAIN-BRCA outperforms the existing similar models MOGONET (Acc: 0.72) and moBRCA-net (Acc: 0.86) with an accuracy of 0.91. These transformed features were then processed using SHAP, and the top 357 features were selected for functional characterization and pathway analysis. We identified specific pathways and genes related to each breast cancer subtype separately. The most enriched pathways found in each subtype are ribonucleotide reductase signaling and glucocorticoid signaling pathways in luminal A, PFKB4 signaling pathway in luminal B, adipogenesis and DNA methylation and transcriptional signaling in HER2+, and the role of OCT4 in mammalian embryonic stem cell pluripotency in basal. Similarly, the most discriminated gene features are BRAF, ESR1, KRAS, MAPK1, SMARCE1, PPP3CA, and TGFBR2 in luminal A, TGFB3 and NCOA3 in luminal B, HNF4A, SIN3A, and SOX9 in HER2+, and POU5F1 and FOXA2 in basal subtypes. The GAIN-BRCA framework, combined with SHAP, weighs independent omics as cross-disciplinary rather than multidisciplinary and presents a distinctive picture for each subtype. As a result, the framework presents the importance of features about the corresponding subtype, which might lead to a greater understanding of the molecular basis of breast cancer subtypes. Citation Format: Jai Chand Patel, Sushil Kumar Shakyawar, Sahil Sethi, Chittibabu Guda. GAIN-BRCA: A graphical explainable AI-net framework for breast cancer subtype classification using multiomics data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3526.

Broad learning: A GPU-free image-based malware classification

In today’s cybersecurity landscape, software security companies encounter a significant challenge in detecting new and unknown malware. Despite the introduction of various machine learning and deep learning tools designed to identify malicious software based on static and dynamic features, achieving the desired level of accuracy remains elusive. This challenge is exacerbated by factors such as encryption, packing, limited distribution, and uneven allocation of malware samples across different families. Moreover, deep learning techniques demand substantial time, computational resources (specifically GPUs), and expertise from data scientists for practical malware analysis. In response to these challenges, we propose a novel GPU-free approach called Image-based Malware Classification using Broad Learning (IMCBL) to address these issues. Our method integrates visualization, feature decomposition, and broad learning architecture to enhance malware detection and classification. We convert raw malware binaries into images, reducing the necessity for extensive feature engineering. These images transform using truncated Singular Value Decomposition (SVD) to reduce the feature vector size, expediting the training process while mitigating model overfitting. The transformed feature vector is then input into our proposed Broad Learning (BL) system, which facilitates malware detection and classification. The BL architecture, structured as a flat network mapping original inputs to feature nodes and expanding the structure in enhancement nodes, ensures efficient and effective classification without the need for retraining. This dynamic and incremental learning capability sets IMCBL apart, making it superior to existing deep learning architectures. To validate our approach, we conducted extensive experiments using five benchmark malware datasets, including the Microsoft Windows malware challenge dataset, the Malimg Windows malware dataset, the IoT-Android mobile malware dataset, the Big Windows malware dataset, and an obfuscated Windows malware dataset. The results demonstrate IMCBL’s remarkable success in classifying most malware samples, even under obfuscation attacks, performing comparably or outperforming current methods using similar benchmarks. Specifically, IMCBL achieved 95.58% accuracy for the Microsoft Windows malware dataset, 97.64% accuracy for the Malimg Windows malware dataset, 96.51% accuracy for IoT Android malware datasets, and 96.19% accuracy for the extensive Windows malware dataset. Additionally, IMCBL demonstrated 93.04% accuracy for an obfuscated Windows malware dataset, which contains both packed and unpacked malware samples. Notably, IMCBL exhibits an exponential advantage in computation overhead, including training and prediction time, when compared to traditional machine learning and advanced state-of-the-art deep learning architectures such as VGG16, ResNet50, and InceptionV3.

Deep learning based sarcasm detection and classification model

Social media is becoming a crucial part of our everyday lives, whether it’s for product advertising, developing brand value, or reaching out to users. At the same time, sentiment analysis (SA) is a method for determining the emotions associated with online information. The main obstacle to SA’s success is the presence of sarcasm in the text. Previous studies on the identification of sarcasm use lexical and pragmatic signs such as interjection, punctuation, and sentimental change, amongst others. Deep learning (DL) models can be used to learn the lexical and contextual aspects of informal language because handcrafted features cannot be generalised. In addition, word embedding can be used to train the DL models and provide effective results on big datasets at the same time. Optimal Deep Learning based Sarcasm detection and classification using an ODL-SDC method is presented in this study. ODL-SDC analyses social media data to look for and classify any sarcasm that may have been used there. In addition, the Glove embedding approach is used to transform feature vectors. A approach known as the chaotic crow search optimization on deep belief network (CCSO-DBN) is also used to classify and detect satire. Many benchmark datasets were used to evaluate the ODL-SDC method, and the results show it to be more effective than existing approaches in a number of performance metrics.

Spatial linear discriminant analysis approaches for remote-sensing classification

Linear Discriminant Analysis (LDA) is a popular and simple classification tool that often outperforms more sophisticated modern machine learning techniques in remote sensing. We introduce a novel LDA method that uses spatial autocorrelation of all pixels of an object to be classified but also of other objects of the training set that are spatially close to improve classification performance. To simplify spatial modelling and model fitting, the methodology is applied to the transformed feature vectors. We term this method conditional spatial LDA. Much alike universal Kriging in geostatistical interpolation, the combined use of feature data and conditioning on labelled training data in conditional spatial LDA was best able to exploit the available geospatial data. The method is illustrated on a crop classification case study from the Aconcagua agricultural region in central Chile.

Grey wolf optimization (GWO) with the convolution neural network (CNN)-based pattern recognition system

ABSTRACT The dynamic video frame dataset’s automated feature analysis addresses the complexity of intensity mapping with normal and abnormal classes. Iterative modelling is needed to learn the component of a video frame in several patterns for various video frame data types for threshold-based data clustering and feature analysis. GWO optimises the Convoluted Pattern of Wavelet Transform (CPWT) feature vectors employed in this paper's CNN feature analysis technique. A median filter reduces noise and smooths the video frame before normalising it. Edge information represents the video frame's bright spot boundary. Neural network based video frame classification clusters pixels using feature recurrent learning with minimal dataset training. The filtered video frame's features were evaluated using complex wavelet transformation feature extraction algorithms. These features demonstrate video frame spatial and textural classifications. CNN classifiers help analyse video frame instances and classify action labels. Categorization improves with the fewest training datasets. This strategy may be beneficial if compared to optimal practises.

A Hyperparameter-Free, Fast and Efficient Framework to Detect Clusters From Limited Samples Based on Ultra High-Dimensional Features.

Clustering is a challenging problem in machine learning in which one attempts to group N objects into K0 groups based on P features measured on each object. In this article, we examine the case where N ≪ P and K0 is not known. Clustering in such high dimensional, small sample size settings has numerous applications in biology, medicine, the social sciences, clinical trials, and other scientific and experimental fields. Whereas most existing clustering algorithms either require the number of clusters to be known a priori or are sensitive to the choice of tuning parameters, our method does not require the prior specification of K0 or any tuning parameters. This represents an important advantage for our method because training data are not available in the applications we consider (i.e., in unsupervised learning problems). Without training data, estimating K0 and other hyperparameters-and thus applying alternative clustering algorithms-can be difficult and lead to inaccurate results. Our method is based on a simple transformation of the Gram matrix and application of the strong law of large numbers to the transformed matrix. If the correlation between features decays as the number of features grows, we show that the transformed feature vectors concentrate tightly around their respective cluster expectations in a low-dimensional space. This result simplifies the detection and visualization of the unknown cluster configuration. We illustrate the algorithm by applying it to 32 benchmarked microarray datasets, each containing thousands of genomic features measured on a relatively small number of tissue samples. Compared to 21 other commonly used clustering methods, we find that the proposed algorithm is faster and twice as accurate in determining the "best" cluster configuration.

Dynamic contrast enhanced‐magnetic resonance imaging radiomics combined with a hybrid adaptiveneuro‐fuzzyinference system‐particle swarm optimization approach for breast tumour classification

AbstractThe authors propose a method for breast dynamic contrast enhanced‐magnetic resonance imaging classification by combining radiomic texture analysis with a hybrid adaptive neuro‐fuzzy inference system (ANFIS)‐particle swarm optimization (PSO) classifier. The fast discrete curvelet transform is utilized as a decomposition scheme in multiple scales. The mean and entropy features extracted from the produced scheme are used as texture descriptors. Principal component analysis (PCA) involves reduction of the dimensionality of the initial feature set. The transformed feature vector is subsequently introduced to a hybrid ANFIS‐PSO classifier. The average overall classification power of the proposed hybrid ANFIS‐PSO classifier is comparatively assessed to that obtained using several classifiers (ANFIS, linear discriminant analysis, Naïve Bayes, artificial neural networks, random forest and support vector machine) by using the 70 training‐30 testing data ratio. The comparison performed highlights the superiority of the proposed methodology, thus underlying the potential of ANFIS‐PSO for the breast cancer diagnosis with a classification accuracy of 94%.

Investigating and improving the utility of probabilistic linear discriminant analysis for acoustic signal classification

Probabilistic linear discriminant analysis (PLDA) has achieved good performance in face recognition and speaker recognition. However, the computation of PLDA using the original formulation is inefficient when there are many training data, especially when the dimensionality of the data is high. Faced with this inefficiency issue, we propose scalable formulations for PLDA. The computation of PLDA using the scalable formulations is more efficient than using the original formulation when dealing with many training data. Using the scalable formulations, the PLDA model can significantly outperform other popular classifiers for speaker recognition, such as Support Vector Machine (SVM) and Gaussian Mixture Model (GMM). Besides of directly using PLDA as a classifier, we may also use PLDA as a feature transformation technique. This PLDA-based feature transformation technique can reduce or expand the original feature dimensionality, and at the same time keep the transformed feature vector approximately following the Gaussian distribution. Our experimental results on speaker recognition and acoustic scene classification demonstrate the effectiveness of applying PLDA for feature transformation. It is then promising to combine PLDA with other classification models for improved performance, extending the utility of PLDA to a wider range of areas.

A cancelable biometric authentication system based on feature-adaptive random projection

Biometric template data protection is critical in preventing user privacy and identity from leakage. Random projection based cancelable biometrics is an efficient and effective technique to achieve biometric template protection. However, traditional random projection based cancelable template design suffers from the attack via record multiplicity (ARM), where an adversary obtains multiple transformed templates from different applications and the associated parameter keys so as to assemble them into a full-rank linear equation system, thereby retrieving the original feature vector. To address this issue, in this paper we propose a feature-adaptive random projection based method, in which the projection matrixes, the key to the ARM, are generated from one basic matrix in conjunction with local feature slots. The generated projection matrixes are discarded after use, thus making it difficult for the adversary to launch the ARM. Moreover, the random projection in the proposed method is performed on a local-feature basis. This feature-adaptive random projection can mitigate the negative impact of biometric uncertainty on recognition accuracy, as it limits the error to part of the transformed feature vector rather than the entire vector. The proposed method is evaluated on four public available databases FVC2002 DB1-DB3 and FVC2004 DB2. The experimental results and security analysis show the validity of the proposed method.

A tfidfvectorizer and singular value decomposition based host intrusion detection system framework for detecting anomalous system processes

Host based intrusion detection systems (HIDSs) are indispensable tools for providing a comprehensive security solution. They are capable of detecting host specific attacks, which cannot be detected using network based intrusion detection systems (NIDSs). This paper proposes a novel tfidfvectorizer and truncated singular value decomposition (SVD) based host intrusion detection system (HIDS) framework for identification of anomalous system processes in real time. The proposed HIDS framework takes the system call trace files as its input and transforms them into n-gram feature vector representational models. The framework then uses a vectorization technique called the tfidfvectorizer to compute the tfidf values of the n-gram terms of the transformed feature vectors. Dimensionality reduction of the transformed n-gram feature vectors are then carried out using truncated SVD based on their tfidf values. The dimensionality reduced tfidfvectorized n-gram feature vectors are finally provided as inputs to various machine learning based classifier models to determine whether the corresponding system call trace files are normal or anomalous. Experimental results on the benchmark ADFA-LD and ADFA-WD datasets show that the proposed HIDS framework effectively detects anomalous system processes with high accuracy and low processing overhead. It is also shown to outperform other HIDS frameworks proposed in the literature.

Building Deep Random Ferns Without Backpropagation

Randomized sampling-based ensemble learning is emerging as a new alternative to deep neural networks (DNNs) because it supports diversity and locality and does not require backpropagation in the learning process. By connecting randomized weak classifiers in a layer-by-layer manner, it is possible to create models having a performance similar to that of DNNs, but with less overfitting and better generalizability than before. Therefore, in this paper, we propose a deep random ferns (d-RFs) model, in which extremely randomized ferns are connected to multilayers, allowing a high classification performance and a lightweight and fast structure. The input vector is first encoded as a transformed feature vector in the feature encoder layer and then is input to the cascade layers. The feature encoding process is similar to the DNN convolution and helps improve the performance of the final output layer. Unlike in the backpropagation paradigm, the cascade layer adjusts the number of ferns and layers required for the d-RFs adaptively, using only a small scale of data. RFs ensemble approaches have considerably fewer hyper-parameters than a DNN or deep forest model, and the complexity can be determined automatically in a data-dependent manner. In addition, experimental results show that an ensemble of multiple weak classifiers reduces the bias between models through an averaging of the weakly correlated classifiers, resulting in a better overall model. The proposed lightweight d-RFs was successfully applied to benchmark datasets and yielded a similar or better accuracy level and a smaller number of parameters and operations as compared with state-of-the-art methods.

Novel Polarimetric Contrast Enhancement Method Based on Minimal Clutter to Signal Ratio Subspace

Enhancing the contrast of target and clutter is a crucial issue in synthetic aperture radar (SAR) image target detection. In this paper, we define a novel subspace, called minimal clutter-to-signal ratio (MCSR) subspace, which can minimize the clutter-to-signal ratio (CSR) by projecting the feature vector to the subspace. Based on MCSR, a novel polarimetric contrast enhancement method is proposed. The MCSR subspace is learned based on the commonly used polarimetric feature vectors extracted from the labeled training SAR image pixels. The feature vectors extracted form candidate SAR image pixels are projected to the MCSR subspace. By calculating the square norm of each transformed feature vector, an enhanced image can be obtained. It is demonstrated that the existing optimization of polarimetric contrast enhancement (OPCE) is a special case of the proposed method to some extent. Experimental results show that our method outperforms the traditional OPCE method on the RadarSat-2 SAR data.

Linear Feature Transform and Enhancement of Classification on Deep Neural Network

A weighted and convex regularized nuclear norm model is introduced to construct a rank constrained linear transform on feature vectors of deep neural networks. The feature vectors of each class are modeled by a subspace, and the linear transform aims to enlarge the pairwise angles of the subspaces. The weight and convex regularization resolve the rank degeneracy of the linear transform. The model is computed by a difference of convex function algorithm whose descent and convergence properties are analyzed. Numerical experiments are carried out in convolutional neural networks on CAFFE platform for 10 class handwritten digit images (MNIST) and small object color images (CIFAR-10) in the public domain. The transformed feature vectors improve the accuracy of the network in the regime of low dimensional features subsequent to dimensional reduction via principal component analysis. The feature transform is independent of the network structure, and can be applied to reduce complexity of the final fully-connected layer without retraining the feature extraction layers of the network.

머신러닝을 이용한 앉은 자세 분류 연구

According to recent studies, poor sitting posture of the spine has been shown to lead to a variety of spinal disorders. For this reason, it is important to measure the sitting posture. We proposed a strategy for classification of sitting posture using machine learning. We retrieved acceleration data from single tri-axial accelerometer attached on the back of the subject’s neck in 5-types of sitting posture. 6 subjects without any spinal disorder were participated in this experiment. Acceleration data were transformed to the feature vectors of principle component analysis. Support vector machine (SVM) and K-means clustering were used to classify sitting posture with the transformed feature vectors. To evaluate performance, we calculated the correct rate for each classification strategy. Although the correct rate of SVM in sitting back arch was lower than that of K-means clustering by 2.0%, SVM’s correct rate was higher by 1.3%, 5.2%, 16.6%, 7.1% in a normal posture, sitting front arch, sitting cross-legged, sitting leaning right, respectively. In conclusion, the overall correction rates were 94.5% and 88.84% in SVM and K-means clustering respectively, which means that SVM have more advantage than K-means method for classification of sitting posture.

An efficient low vision plant leaf shape identification system for smart phones

In computer vision research, the first most important step is to represent the captured object into some mathematical transformed feature vector describing the proper shape, texture and/or color information for the classification. To understand the nature's biodiversity, together with computer vision (CV), the emerging ubiquitous mobile technologies are now used. Therefore, in this paper, a novel low computational, efficient, and accurate rotation-scale-translation invariant shape profile transform called Angle View Projection (AVP) is proposed. The leaf images captured via mobile devices are transformed to an AVP shape profile curve (a set of four shapelets) and then compacted using Discrete Cosine Transform (DCT) to improve the performance of the system. It also reduces the energy consumption of the device. The algorithm is tested on five different types of leaf datasets: Flavia dataset, 100 plant species leaves dataset, Swedish database, Intelligent Computing Laboratory leaf dataset and Diseased leaf dataset. An `Agent' on mobile device decides whether the module needs to offload to the Server or to compute on the device itself. The experiments carried out clearly indicates that the proposed system outperforms the state-of-the-art with a fast response time even in a low vision environment. AVP also outperforms other methods when tested over incomplete leaves caused due to the physiological or pathological phenomenon. This AVP shape profile based mobile plant biometric system is developed for general applications in our society to better understand the nature and helps in botanical studies and researches.

Human pose recovery by supervised spectral embedding

In this paper we propose a subspace learning algorithm based on supervised manifold learning techniques to address the problem of inferring 3D human poses from monocular video frames. Low-dimensional representations of visual features are computed via spectral embedding, regularized by the pairwise relationship of poses for simultaneously preserving the locality in the feature space and taking account of similarities in the pose space. To deal with the “out-of-sample” problem, we obtain a global linear projection from the embedding whereby the Euclidean distances between transformed feature vectors can faithfully reflect the corresponding pose distances. To retrieve the most similar candidate from the exemplar database, weighted sum of Euclidean distances of features is employed to achieve better accuracy instead of simply summing up the squared distances of all feature types. The experimental results on HumanEva dataset validate the efficacy of our proposed method.

Hidden Markov model‐based speech enhancement using multivariate Laplace and Gaussian distributions

In this paper, statistical speech enhancement using hidden Markov model (HMM) is studied and new techniques for applying non-Gaussian distributions are proposed. The superiority of using non-Gaussian distributions in online adaptive noise suppression algorithms has been proven; however, in this study, this approach is formulated in an HMM-based mean-square error estimator (MMSE) estimator in which a priori models are trained in an off-line manner. In addition, an analytical study of using different distributions other than autoregressive (AR) Gaussian distribution, such as Laplace, is presented in order to construct an accurate HMM as a priori model for discrete Fourier transform and discrete cosine transform feature vectors of speech signal. In the proposed framework, an HMM-based MMSE estimator bassed on Gaussian assumption using diagonal covariance matrix is provided rather than AR hypothesis which is employed in the conventional AR-HMM-based speech enhancement algorithm. Experimental evaluations of the proposed methods are done in the presence of four different noise types at various signal-to-noise ratio levels which demonstrate the superiority of the proposed methods in most conditions in comparison with AR-HMM.

Cluster-based analysis for characterizing dynamic functional connectivity.

Different regions in the resting brain exhibit non-stationary functional connectivity (FC) over time. In this paper, a simple and efficient framework of clustering the variability in FC of a rat's brain at rest is proposed. This clustering process reveals areas that are always connected with a chosen region, called seed voxel, along with the areas exhibiting variability in the FC. This addresses an issue common to most dynamic FC analysis techniques, which is the assumption that the spatial extent of a given network remains constant over time. We increase the voxel size and reduce the spatial resolution to analyze variable FC of the whole resting brain. We hypothesize that the adjacent voxels in resting state functional magnetic resonance imaging (rsfMRI), just as in task-based fMRI, exhibit similar intensities, so they can be averaged to obtain larger voxels without any significant loss of information. Sliding window correlation is used to compute variable patterns of the rat's whole brain FC with the seed voxel in the sensorimotor cortex. These patterns are grouped based on their spatial similarities using binary transformed feature vectors in k-means clustering, not only revealing the variable and nonvariable portions of FC in the resting brain but also detecting the extent of the variability of these patterns.

A Similarity Search Scheme over Encrypted Cloud Images based on Secure Transformation

With the growing popularity of cloud computing, more and more users outsource their private data to the cloud. To ensure the security of private data, data owners usually encrypt their private data before outsourcing the data to the cloud server, which brings incommodity of data operating. This paper proposes a scheme for similar search on encrypted images. In the setup phase, image owner extracts feature vectors to represent the images as usual image retrieval system does. Then, the feature vectors are transformed by an invertible matrix, which not only protect the information of feature vector but also support similarity evaluation between the vectors. The encrypted vectors and image identifies are used to construct inverted index, which is finally uploaded along with the encrypted image to the cloud. In the search phase, with a query image, the authorized image user extracts and encrypts feature vector to generate the trapdoor. The trapdoor is submitted to the cloud and can be used to calculate the similarity with the transformed feature vectors. The encryption on features does not degrade the result accuracy. Moreover, the image owner could update the encrypted image database as well as the secure index very easily.