Feature Space Research Articles

An optimized heterogeneous multi-access edge computing framework based on transfer learning and artificial internet of things

In most practical applications, the feature space of the training datasets and the target domain datasets are inconsistent, or the data distribution between them is inconsistent, which leads to the problem of data starvation and makes it difficult for terminal devices to obtain high accurate results. Aiming at the problems of limited terminal device resources, low accuracy of data processing results, and unsatisfactory processing speed, a Heterogeneous Multi-access Edge Computing (MEC) Framework based on Transfer Learning (TL) is proposed, abbreviated as HMECF-TL. This framework adopts a cloud-edge-end three-layer architecture. It uses model transfer to optimize the Convolutional Neural Networks (CNN) model at each layer to achieve the goal of improving data processing speed and accuracy. Furthermore, a multi-agent Deep Reinforcement Learning Algorithm having Attention Mechanism (DRLAAM) is designed to further increase the timeliness performance of computation-intensive applications. The performance of HMECSF-TL framework is verified by simulation experiments, which not only reduces the delay by more than 24.66 %, but also improves the accuracy by more than 8.34 %. The framework not only increase the computing capacity to solve the shortage of terminal device resources, but also improve the quality of data processing to solve the problem of data starvation.

Online imbalance learning with unpredictable feature evolution and label scarcity

Recently, online learning with imbalanced data streams has aroused wide concern, which reflects an uneven distribution of different classes in data streams. Existing approaches have conventionally been conducted on stationary feature space and they assume that we can obtain the entire labels of data streams in the case of supervised learning. However, in many real scenarios, e.g., the environment monitoring task, new features flood in, and old features are partially lost during the changing environment as the different lifespans of different sensors. Besides, each instance needs to be labeled by experts, resulting in expensive costs and scarcity of labels. To address the above problems, this paper proposes a novel Online Imbalance learning with unpredictable Feature evolution and Label scarcity (OIFL) algorithm. First, we utilize margin-based online active learning to selectively label valuable instances. After obtaining the labels, we handle imbalanced class distribution by optimizing F-measure and transforming F-measure optimization into a weighted surrogate loss minimization. When data streams arrive with augmented features, we combine the online passive-aggressive algorithm and structural risk minimization to update the classifier in the divided feature space. When data streams arrive with incomplete features, we leverage variance to identify the most informative features following the empirical risk minimization principle and continue to update the existing classifier as before. Finally, we obtain a sparse but reliable learner by the strategy of projecting truncation. We derive theoretical analyses of OIFL. Also, experiments on the synthetic datasets and real-world data streams to validate the effectiveness of our method.

ASF-Net: Robust video deraining via temporal alignment and online adaptive learning

In recent times, learning-based methods for video deraining have demonstrated commendable results. However, there are two critical challenges that these methods are yet to address: exploiting temporal correlations among adjacent frames and ensuring adaptability to unknown real-world scenarios. To overcome these challenges, we explore video deraining from a paradigm design perspective to learning strategy construction. Specifically, we propose a new computational paradigm, Alignment-Shift-Fusion Network (ASF-Net), which incorporates a temporal shift module. This module is novel to this field and provides deeper exploration of temporal information by facilitating the exchange of channel-level information within the feature space. To fully discharge the model’s characterization capability, we further construct a LArge-scale RAiny video dataset (LARA) which also supports the development of this community. On the basis of the newly-constructed dataset, we explore the parameters learning process by developing an innovative re-degraded learning strategy. This strategy bridges the gap between synthetic and real-world scenes, resulting in stronger scene adaptability. Our approach exhibits superior performance in three benchmarks and compelling visual quality in real-world scenarios, underscoring its efficacy. The code is available at https://github.com/vis-opt-group/ASF-Net.

Machine Reading Comprehension Model Based on Fusion of Mixed Attention

To address the problems of the insufficient semantic fusion between text and questions and the lack of consideration of global semantic information encountered in machine reading comprehension models, we proposed a machine reading comprehension model called BERT_hybrid based on the BERT and hybrid attention mechanism. In this model, BERT is utilized to separately map the text and questions into the feature space. Through the integration of Bi-LSTM, an attention mechanism, and a self-attention mechanism, the proposed model achieves a comprehensive semantic fusion between text and questions. The probability distribution of answers is computed using Softmax. The experimental results on the public dataset DuReader demonstrate that the proposed model achieves improvements in BLEU-4 and ROUGE-L scores compared to existing models. Furthermore, to validate the effectiveness of the proposed model design, we analyze the factors influencing the model’s performance.

Practical feature filter strategy to machine learning for small datasets in chemistry

Many potential use cases for machine learning in chemistry and materials science suffer from small dataset sizes, which demands special care for the model design in order to deliver reliable predictions. Hence, feature selection as the key determinant for dataset design is essential here. We propose a practical and efficient feature filter strategy to determine the best input feature candidates. We illustrate this strategy for the prediction of adsorption energies based on a public dataset and sublimation enthalpies using an in-house training dataset. The input of adsorption energies reduces the feature space from 12 dimensions to two and still delivers accurate results. For the sublimation enthalpies, three input configurations are filtered from 14 possible configurations with different dimensions for further productive predictions as being most relevant by using our feature filter strategy. The best extreme gradient boosting regression model possesses a good performance and is evaluated from statistical and theoretical perspectives, reaching a level of accuracy comparable to density functional theory computations and allowing for physical interpretations of the predictions. Overall, the results indicate that the feature filter strategy can help interdisciplinary scientists without rich professional AI knowledge and limited computational resources to establish a reliable small training dataset first, which may make the final machine learning model training easier and more accurate, avoiding time-consuming hyperparameter explorations and improper feature selection.

A modified CNN-IDS model for enhancing the efficacy of intrusion detection system

As the security of computer networks in enterprises worldwide is dependent on the proper functioning of intrusion detection systems (IDSs) and intrusion prevention systems (IPSs), this effectiveness of both of them is of utmost priority. Leveraging diverse techniques, these network security systems are created to keep the reliability, the availability, and the integrity of the organizational networks safe. One plus point of using ML in intrusion detection system (IDS) is that it has successfully weeded out all the IDS attacks with a high degree of accuracy. In contrast, such systems may be believed to operate to their least competent levels when supersized data spaces have to be dealt with. In the process to solve this, application of feature selection techniques will play the crucial role to ignore non-relevant features which do not impact the issue of classification much. One more thing to keep in mind is that the ML-based IDSs often have problems with high false alarms and percentage accuracy because of the imbalanced training sets. The undertaking of this paper involves a through the analysis of the UNSW-NB15 intrusion detection data set as upon which our models will be tested and trained. We utilize two feature selection approaches: the PCA method, which is denoted as PCA, and the SVD method, called SVD. Furthermore, we categorize the datasets using these methods— Ridge Regression (RR), Stochastic Gradient Descent, and Convolutional Neural Network (CNN)-- on the transformed feature space. What is the most widely used for, is that it deals with both, binary and multiclass classification. The result measure that PCA and SVD are succeeded in getting better performance of IDS than others with enhancing the accuracy of classification models. More specifically, the RR classifier's precise was outstanding for the binary classification problem experiencing a rise in the accuracy from 98.13 % to 99.85 %. This shows the critical role of feature selection approaches and is also demonstrates the modeling capabilities of RR, SGD, and CNN classifiers and stands out as a solution to intrusion detection.

ECF-Net: Enhanced, Channel-Based, Multi-Scale Feature Fusion Network for COVID-19 Image Segmentation

Accurate segmentation of COVID-19 lesion regions in lung CT images aids physicians in analyzing and diagnosing patients’ conditions. However, the varying morphology and blurred contours of these regions make this task complex and challenging. Existing methods utilizing Transformer architecture lack attention to local features, leading to the loss of detailed information in tiny lesion regions. To address these issues, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement. Specifically, we leverage the learning capabilities of both CNN and Transformer architectures to design parallel channel extraction blocks in three different ways, effectively capturing diverse lesion features. Additionally, to minimize irrelevant information in the high-dimensional feature space and focus the network on useful and critical information, we develop adaptive feature generation blocks. Lastly, a bidirectional pyramid-structured feature fusion approach is introduced to integrate features at different levels, enhancing the diversity of feature representations and improving segmentation accuracy for lesions of various scales. The proposed method is tested on four COVID-19 datasets, demonstrating mIoU values of 84.36%, 87.15%, 83.73%, and 75.58%, respectively, outperforming several current state-of-the-art methods and exhibiting excellent segmentation performance. These findings provide robust technical support for medical image segmentation in clinical practice.

Multi-scale anatomical regularization for domain-adaptive segmentation of pelvic CBCT images.

Cone beam computed tomography (CBCT) image segmentation is crucial in prostate cancer radiotherapy, enabling precise delineation of the prostate gland for accurate treatment planning and delivery. However, the poor quality of CBCT images poses challenges in clinical practice, making annotation difficult due to factors such as image noise, low contrast, and organdeformation. The objective of this study is to create a segmentation model for the label-free target domain (CBCT), leveraging valuable insights derived from the label-rich source domain (CT). This goal is achieved by addressing the domain gap across diverse domains through the implementation of a cross-modality medical image segmentationframework. Our approach introduces a multi-scale domain adaptive segmentation method, performing domain adaptation simultaneously at both the image and feature levels. The primary innovation lies in a novel multi-scale anatomical regularization approach, which (i) aligns the target domain feature space with the source domain feature space at multiple spatial scales simultaneously, and (ii) exchanges information across different scales to fuse knowledge from multi-scaleperspectives. Quantitative and qualitative experiments were conducted on pelvic CBCT segmentation tasks. The training dataset comprises 40 unpaired CBCT-CT images with only CT images annotated. The validation and testing datasets consist of 5 and 10 CT images, respectively, all with annotations. The experimental results demonstrate the superior performance of our method compared to other state-of-the-art cross-modality medical image segmentation methods. The Dice similarity coefficients (DSC) for CBCT image segmentation results is %, and the average symmetric surface distance (ASSD) is . Statistical analysis confirms the statistical significance of the improvements achieved by ourmethod. Our method exhibits superiority in pelvic CBCT image segmentation compared to its counterparts.

Significance of Machine Learning Algorithms to Improve Predictive Analytics in Chronic Disease Management through Pharmacogenomics

The treatment of chronic diseases is an important concern in global health. Machine learning (ML)-based disease prediction models are becoming more important for making informed medical decisions in light of the paradigm shift towards preventative care. Integrating genetic, pharmacogenomic, personal health, and psychosocial data can greatly assist healthcare practitioners in making treatment-related decisions for patients with chronic diseases. This study utilizes a Deep Convolutional Neural Network-assisted Chronic Disease Management (DCNN-CDM) through pharmacogenomics and an improved predictive analytics model to enable informed real-time decision-making at the point of care. Data augmentation in terms of feature space allows the DCNN model to avoid over-fitting while effectively capturing high-level features submerged in chronic disease datasets. Afterwards, this work suggests an attention-empowered DCNN model to enhance sick case diagnosis accuracy, which augments data regarding sample space, thereby alleviating the class imbalance issue. Electronic health information data mining is now using predictive analytics to determine individuals at risk of acquiring chronic disease problems. The suggested model may aid in the early and accurate diagnosis of chronic diseases. The numerical outcomes demonstrate that the recommended DCNN-CDM model increases the accuracy rate of 98.7%, patient monitoring rate of 97.5%, F1-score rate of 96.3% and predictive performance rate of 95.1% compared to other existing methodologies.

SRL‐ProtoNet: Self‐supervised representation learning for few‐shot remote sensing scene classification

AbstractUsing a deep learning method to classify a large amount of labelled remote sensing scene data produces good performance. However, it is challenging for deep learning based methods to generalise to classification tasks with limited data. Few‐shot learning allows neural networks to classify unseen categories when confronted with a handful of labelled data. Currently, episodic tasks based on meta‐learning can effectively complete few‐shot classification, and training an encoder that can conduct representation learning has become an important component of few‐shot learning. An end‐to‐end few‐shot remote sensing scene classification model based on ProtoNet and self‐supervised learning is proposed. The authors design the Pre‐prototype for a more discrete feature space and better integration with self‐supervised learning, and also propose the ProtoMixer for higher quality prototypes with a global receptive field. The authors’ method outperforms the existing state‐of‐the‐art self‐supervised based methods on three widely used benchmark datasets: UC‐Merced, NWPU‐RESISC45, and AID. Compare with previous state‐of‐the‐art performance. For the one‐shot setting, this method improves by 1.21%, 2.36%, and 0.84% in AID, UC‐Merced, and NWPU‐RESISC45, respectively. For the five‐shot setting, this method surpasses by 0.85%, 2.79%, and 0.74% in the AID, UC‐Merced, and NWPU‐RESISC45, respectively.

Probabilistic Contrastive Learning for Long-Tailed Visual Recognition.

Long-tailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Such imbalance issue considerably impairs the performance of standard supervised learning algorithms, which are mainly designed for balanced training sets. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. However, the performance of supervised contrastive learning is plagued by an inherent challenge: it necessitates sufficiently large batches of training data to construct contrastive pairs that cover all categories, yet this requirement is difficult to meet in the context of class-imbalanced data. To overcome this obstacle, we propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space, and samples contrastive pairs accordingly. In fact, estimating the distributions of all classes using features in a small batch, particularly for imbalanced data, is not feasible. Our key idea is to introduce a reasonable and simple assumption that the normalized features in contrastive learning follow a mixture of von Mises-Fisher (vMF) distributions on unit space, which brings two-fold benefits. First, the distribution parameters can be estimated using only the first sample moment, which can be efficiently computed in an online manner across different batches. Second, based on the estimated distribution, the vMF distribution allows us to sample an infinite number of contrastive pairs and derive a closed form of the expected contrastive loss for efficient optimization. Other than long-tailed problems, ProCo can be directly applied to semi-supervised learning by generating pseudo-labels for unlabeled data, which can subsequently be utilized to estimate the distribution of the samples inversely. Theoretically, we analyze the error bound of ProCo. Empirically, extensive experimental results on supervised/semi-supervised visual recognition and object detection tasks demonstrate that ProCo consistently outperforms existing methods across various datasets.

Deep Learning Features Can Improve Radiomics-Based Prostate Cancer Aggressiveness Prediction

PURPOSE Emerging evidence suggests that the use of artificial intelligence can assist in the timely detection and optimization of therapeutic approach in patients with prostate cancer. The conventional perspective on radiomics encompassing segmentation and the extraction of radiomic features considers it as an independent and sequential process. However, it is not necessary to adhere to this viewpoint. In this study, we show that besides generating masks from which radiomic features can be extracted, prostate segmentation and reconstruction models provide valuable information in their feature space, which can improve the quality of radiomic signatures models for disease aggressiveness classification. MATERIALS AND METHODS We perform 2,244 experiments with deep learning features extracted from 13 different models trained using different anatomic zones and characterize how modeling decisions, such as deep feature aggregation and dimensionality reduction, affect performance. RESULTS While models using deep features from full gland and radiomic features consistently lead to improved disease aggressiveness prediction performance, others are detrimental. Our results suggest that the use of deep features can be beneficial, but an appropriate and comprehensive assessment is necessary to ensure that their inclusion does not harm predictive performance. CONCLUSION The study findings reveal that incorporating deep features derived from autoencoder models trained to reconstruct the full prostate gland (both zonal models show worse performance than radiomics only models), combined with radiomic features, often lead to a statistically significant increase in model performance for disease aggressiveness classification. Additionally, the results also demonstrate that the choice of feature selection is key to achieving good performance, with principal component analysis (PCA) and PCA + relief being the best approaches and that there is no clear difference between the three proposed latent representation extraction techniques.

TriLA: Triple-Level Alignment Based Unsupervised Domain Adaptation for Joint Segmentation of Optic Disc and Optic Cup.

Cross-domain joint segmentation of optic disc and optic cup on fundus images is essential, yet challenging, for effective glaucoma screening. Although many unsupervised domain adaptation (UDA) methods have been proposed, these methods can hardly achieve complete domain alignment, leading to suboptimal performance. In this paper, we propose a triple-level alignment (TriLA) model to address this issue by aligning the source and target domains at the input level, feature level, and output level simultaneously. At the input level, a learnable Fourier domain adaptation (LFDA) module is developed to learn the cut-off frequency adaptively for frequency-domain translation. At the feature level, we disentangle the style and content features and align them in the corresponding feature spaces using consistency constraints. At the output level, we design a segmentation consistency constraint to emphasize the segmentation consistency across domains. The proposed model is trained on the RIGA+ dataset and widely evaluated on six different UDA scenarios. Our comprehensive results not only demonstrate that the proposed TriLA substantially outperforms other state-of-the-art UDA methods in joint segmentation of optic disc and optic cup, but also suggest the effectiveness of the triple-level alignment strategy.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Online Feature Selection With Varying Feature Spaces

Online Feature Selection With Varying Feature Spaces

NeRF-Texture: Synthesizing Neural Radiance Field Textures.

Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.

Delving into CLIP latent space for Video Anomaly Recognition

We tackle the complex problem of detecting and recognising anomalies in surveillance videos at the frame level, utilising only video-level supervision. We introduce the novel method AnomalyCLIP, the first to combine Vision and Language Models (VLMs), such as CLIP, with multiple instance learning for joint video anomaly detection and classification. Our approach specifically involves manipulating the latent CLIP feature space to identify the normal event subspace, which in turn allows us to effectively learn text-driven directions for abnormal events. When anomalous frames are projected onto these directions, they exhibit a large feature magnitude if they belong to a particular class. We also leverage a computationally efficient Transformer architecture to model short- and long-term temporal dependencies between frames, ultimately producing the final anomaly score and class prediction probabilities. We compare AnomalyCLIP against state-of-the-art methods considering three major anomaly detection benchmarks, i.e. ShanghaiTech, UCF-Crime, and XD-Violence, and empirically show that it outperforms baselines in recognising video anomalies. Project website and code are available at https://lucazanella.github.io/AnomalyCLIP/.

HEnsem_DTIs: A heterogeneous ensemble learning model for drug-target interactions prediction

Drug discovery is the process by which a drug is discovered. Drug-target interactions prediction is a major part of drug discovery. Unfortunately, producing new drugs is time-consuming and expensive; Because it requires a lot of human and laboratory resources. Recently, predictions have been made using computational methods to solve these problems and prevent blindly examining all interactions. Various experiences using computational methods show that no single algorithm can be suitable for all applications; Hence, ensemble learning is expressed. Although various ensemble methods have been proposed, it is still not easy to find a suitable ensemble method for a particular dataset. In general, the existing algorithms in aggregation and combination method are selected manually based on experience. Reinforcement learning can be one way to meet this challenge. High-dimensional feature space and class imbalance are among the challenges of drug-target interactions prediction. This paper proposes HEnsem_DTIs, a heterogeneous ensemble model, for predicting drug-target interactions using dimensionality reduction and concepts of recommender systems to address these challenges. HEnsem_DTIs is configured with reinforcement learning. Dimensionality reduction is applied to handle the challenge of high-dimensional feature space and recommender systems to improve under-sampling and solve the class imbalance challenge. Six datasets are used to evaluate the proposed model; Results of the evaluation on datasets show that HEnsem_DTIs works better than other models in this field. Results of evaluation of the proposed model on the first dataset using 10-fold cross-validation experiments show the amount of sensitivity 0.896, specificity 0.954, GM 0.924, AUC 0.930 and AUPR 0.935.

FC-StackGNB: A novel machine learning modeling framework for forest fire risk prediction combining feature crosses and model fusion algorithm

Forest fire risk prediction is a crucial link in maintaining forest ecological security. Machine learning, due to its powerful non-linear modeling capabilities, has been widely applied in forest fire risk prediction research. However, existing studies often focus on the direct information provided by multiple environmental factor features when constructing the feature space, while overlooking the deeper information conveyed by feature cross-correlations. Additionally, fire risk prediction predominantly relies on single-model forecasting, exhibiting slightly insufficient generalization and stability in models. Model fusion algorithms (MFA) can combine the advantages of multiple models to compensate for this limitation. In this study, a machine learning framework, FC-StackGNB, combining feature crosses (FC) and model fusion, is proposed. This framework employs the FC method to analyze the temporal trends of various environmental factors influencing fire occurrence, constructing multiple seasonal cross features (SCFs) capable of effectively capturing the non-linear relationship between environmental factors and time. Moreover, the framework develops a Gaussian Naive Bayes (GNB) optimized stacking MFA to fully leverage the strengths of different ML algorithms. Results demonstrate that the introduction of SCFs effectively enhances the prediction performance of six machine learning models, with the mean values of five evaluation metrics (Accuracy, Precision, Recall, F1-score, and ROC_AUC) increasing by 1.58% to 6.30%. The fusion model constructed based on the StackGNB algorithm can effectively handle the multicollinearity issue of features, exhibiting significantly better prediction performance than single models, particularly in improving the Recall metric (increasing by around 3% and 5% compared to the top two ranked single models respectively), which signifies the model’s ability to predict positive samples (i.e., high-risk fire areas). The proposed modeling framework effectively enhances the robustness and prediction performance of the models, offering new modeling insights for subsequent research. This study holds significant importance for enhancing the level of forest fire risk warning.

Microarray classification using genetic algorithm and latin hypercube sampling

Cancer, the second leading cause of global death, requires advanced diagnostic technology. Microarray gene expression technology plays an important role in comprehensively analyzing the genetic aspects of cancer. However, challenges such as high-dimensional attributes, limited samples, and varying gene presence rates hinder the accurate classification of microarray data. This study proposes a model that uses latin hypercube sampling (LHS) in genetic algorithms (GA) for Feature Selection in microarray data classification. LHS makes the chromosome samples in the initial population of GAs representative and diverse. The study used three microarray datasets with different numbers of features and classes. The results reveal that first, the use of GA alone tends to limit the exploration of the resulting feature space, while the use of LHS can expand the feature selection possibilities in the context of feature selection. Secondly, this study shows that microarray classification using GA with LHS (GALHS) consistently outperforms other feature selection methods such as based correlation features (BCF), principal component analysis (PCA), relief, and lasso. Thus, this research contributes to feature selection by applying LHS and GA to optimize the performance of microarray data classification models.