Feature Space Research Articles

The speech neuroprosthesis.

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore naturalcommunication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinicalepilepsy monitoringand subsequently in individuals with paralysis as part ofearlyfeasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapidandsubstantial progress indecoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

High Efficiency Deep-learning Based Video Compression

Although deep learning technique has achieved significant improvement on image compression, but its advantages are not fully explored in video compression, which leads to the performance of deep-learning-based video compression (DLVC) is obviously inferior to that of hybrid video coding framework. In this article, we proposed a novel network to improve the performance of DLVC from its most important modules, including Motion Process (MP), Residual Compression (RC), and Frame Reconstruction (FR). In MP, we design a split second-order attention and multi-scale feature extraction module to fully remove the warping artifacts from multi-scale feature space and pixel space, which can help reduce the distortion in the following process. In RC, we propose a channel selection mechanism to gradually drop redundant information while preserving informative channels for a better rate-distortion performance. Finally, in FR, we introduce a residual multi-scale recurrent network to improve the quality of the current reconstructed frame by progressively exploiting temporal context information between it and its several previous reconstructed frames. Extensive experiments are conducted on the three widely used video compression datasets (HEVC, UVG, and MCL-JVC), and the performance demonstrates the superiority of our proposed approach over the state-of-the-art methods.

Prediction model of early recurrence of multimodal hepatocellular carcinoma with tensor fusion

Objective. In oncology, clinical decision-making relies on a multitude of data modalities, including histopathological, radiological, and clinical factors. Despite the emergence of computer-aided multimodal decision-making systems for predicting hepatocellular carcinoma (HCC) recurrence post-hepatectomy, existing models often employ simplistic feature-level concatenation, leading to redundancy and suboptimal performance. Moreover, these models frequently lack effective integration with clinically relevant data and encounter challenges in integrating diverse scales and dimensions, as well as incorporating the liver background, which holds clinical significance but has been previously overlooked. Approach. To address these limitations, we propose two approaches. Firstly, we introduce the tensor fusion method to our model, which offers distinct advantages in handling multi-scale and multi-dimensional data fusion, potentially enhancing overall performance. Secondly, we pioneer the consideration of the liver background’s impact, integrating it into the feature extraction process using a deep learning segmentation-based algorithm. This innovative inclusion aligns the model more closely with real-world clinical scenarios, as the liver background may contain crucial information related to postoperative recurrence. Main results. We collected radiomics (MRI) and histopathological images from 176 cases diagnosed by experienced clinicians across two independent centers. Our proposed network underwent training and 5-fold cross-validation on this dataset before validation on an external test dataset comprising 40 cases. Ultimately, our model demonstrated outstanding performance in predicting early recurrence of HCC postoperatively, achieving an AUC of 0.883. Significance. These findings signify significant progress in addressing challenges related to multimodal data fusion and hold promise for more accurate clinical outcome predictions. In this study, we exploited global 3D liver background into modelling which is crucial to to the prognosis assessment and analyzed the whole liver background in addition to the tumor region. Both MRI images and histopathological images of HCC were fused at high-dimensional feature space using tensor techniques to solve cross-scale data integration issue.

Pre-stack seismic inversion based on model-constrained generative adversarial network

Pre-stack seismic inversion usually uses various traditional algorithms to estimate elastic parameters such as P-wave velocity, S-wave velocity, and density. It is hard to derive accurate elastic parameters due to their non-uniqueness and high dimensionality between elastic parameters and seismic data, the calculation of elastic parameters is inaccurate. Convolutional Neural Networks (CNNs) have high-dimensional feature space mapping capabilities, which are utilized to establish mapping relationships between seismic data and elasticity parameters. However, their effectiveness is greatly affected by label data, and at the same time, due to the lack of enough label data, resulting in a low degree of fitting between prediction results and real data. In addition, conventional seismic inversion methods based on CNNs lack physical model constraints, resulting in low accuracy and poor interpretability of prediction results. We propose a Cycle-consistent Generative Adversarial Network based on a geophysical mechanism (SeisInv-CycleGAN). Deterministic inversion results and labeled data are combined into hybrid geophysical data as a training set of SeisInv-CycleGAN with geophysical constraints. At the same time, the residual (seismic loss) between the seismic data synthesized by forward modeling and the actual data is used as part of the loss function. The SeisInv-CycleGAN does not require building an initial model, and it can achieve higher accuracy in prediction results with a small amount of labeled data.

Advanced facial recognition with LBP-URIGL hybrid descriptors

AbstractFacial recognition technology is transformative in security and human-machine interaction, reshaping societal interactions. Robust descriptors, essential for high precision in machine learning tasks like recognition and recall, are integral to this transformation. This paper presents a hybrid model enhancing local binary pattern descriptors for facial representation. By integrating rotation-invariant local binary pattern with uniform rotation-invariant grey-level co-occurrence, employing linear discriminant analysis for feature space optimization, and utilizing an artificial neural network for classification, the model achieves exceptional accuracy rates of 100% for Olivetti Research Laboratory, 99.98% for Maastricht University Computer Vision Test, and 99.17% for Extended Yale B, surpassing traditional methods significantly.

Head pose estimation with particle swarm optimization‐based contrastive learning and multimodal entangled GCN

AbstractHead pose estimation is an especially challenging task due to the complexity nonlinear mapping from 2D feature space to 3D pose space. To address the above issue, this paper presents a novel and efficient head pose estimation framework based on particle swarm optimized contrastive learning and multimodal entangled graph convolution network. Firstly, a new network, the region and difference‐aware feature pyramid network (RD‐FPN), is proposed for 2D keypoints detection to alleviate the background interference and enhance the feature expressiveness. Then, particle swarm optimized contrastive learning is constructed to alternatively match 2D and 3D keypoints, which takes the multimodal keypoints matching accuracy as the optimization objective, while considering the similarity of cross‐modal positive and negative sample pairs from contrastive learning as a local contrastive constraint. Finally, multimodal entangled graph convolution network is designed to enhance the ability of establishing geometric relationships between keypoints and head pose angles based on second‐order bilinear attention, in which point‐edge attention is introduced to improve the representation of geometric features between multimodal keypoints. Compared with other methods, the average error of our method is reduced by 8.23%, indicating the accuracy, generalization, and efficiency of our method on the 300W‐LP, AFLW2000, BIWI datasets.

A New Framework of Collaborative Learning for Adaptive Metric Distillation.

This article presents a new adaptive metric distillation approach that can significantly improve the student networks' backbone features, along with better classification results. Previous knowledge distillation (KD) methods usually focus on transferring the knowledge across the classifier logits or feature structure, ignoring the excessive sample relations in the feature space. We demonstrated that such a design greatly limits performance, especially for the retrieval task. The proposed collaborative adaptive metric distillation (CAMD) has three main advantages: 1) the optimization focuses on optimizing the relationship between key pairs by introducing the hard mining strategy into the distillation framework; 2) it provides an adaptive metric distillation that can explicitly optimize the student feature embeddings by applying the relation in the teacher embeddings as supervision; and 3) it employs a collaborative scheme for effective knowledge aggregation. Extensive experiments demonstrated that our approach sets a new state-of-the-art in both the classification and retrieval tasks, outperforming other cutting-edge distillers under various settings.

An Integrated Way to Monitor the Health Condition of Building Facades by Analyzing Thermal Infrared Images

Thermography is widely used for defect detection through analysing thermography. Thermography is used to record the surface temperatures with a thermal infrared camera. Visual interpretation can be employed to identify defects according to the differences in surface temperatures. Recently, image-processing techniques have provided a quantitative way to detect defects rather than visual interpretation. Robust principal component analysis (RPCA) is employed to analyze a series of thermal infrared images [1]. Traditionally, principal component analysis (PCA) is used to investigate a series of thermal infrared images by projecting the thermal data onto a low-dimension space such that a template image inherits the significant characteristics of the given thermal data. However, PCA introduces noise effects such that the template generated with PCA contains the noise effects. RPCA offers a similar way to project the given data onto a low-dimension space and a feature space and provides a robust way to depress the noise effects. With the introduction of RPCA, a template image can be selected from the low- dimension space, and image segmentation is employed to segment the template. Those segmented regions illustrated on the segmented template indicate the surface temperature differences, and those differences can be used as clues to identify the defects.

Diversity-Learning Block: Conquer Feature Homogenization of Multibranch.

Visual Geometry Group (VGG)-style ConvNet is an neural-network process units (NPU)-friendly network; however, the accuracy of this architecture cannot keep up with other well-designed network structures. Although some reparameterization methods are proposed to remedy this weakness, their performance suffers from the homogenization issue of parallel branches, and the preset shape of convolution kernels also influences spatial perception. To address this problem, we propose a diversity-learning (DL) block to build the DLNet, which could adaptively learn various features to enrich the feature space. To balance floating point of operations (FLOPs) and accuracy, groupwise operation is introduced and finally, a lightweight DL ConvNet DLGNet is obtained. Extensive evaluations have been conducted on different computer vision tasks, e.g., image classification [Canadian Institute For Advanced Research (CIFAR) and ImageNet], object detection [PASCAL visual object classes (VOC) and Microsoft Common Objects in Context (MS COCO)], and semantic segmentation (Cityscapes). The experimental results show that our proposed DLGNet can achieve comparable performance with the state-of-the-art networks while the speed is 183% faster than GhostNet and even over 600% faster than MobileNetV3 with similar accuracy when running on NPU.

Exploring the Temporal Consistency of Arbitrary Style Transfer: A Channelwise Perspective.

Arbitrary image stylization by neural networks has become a popular topic, and video stylization is attracting more attention as an extension of image stylization. However, when image stylization methods are applied to videos, unsatisfactory results that suffer from severe flickering effects appear. In this article, we conducted a detailed and comprehensive analysis of the cause of such flickering effects. Systematic comparisons among typical neural style transfer approaches show that the feature migration modules for state-of-the-art (SOTA) learning systems are ill-conditioned and could lead to a channelwise misalignment between the input content representations and the generated frames. Unlike traditional methods that relieve the misalignment via additional optical flow constraints or regularization modules, we focus on keeping the temporal consistency by aligning each output frame with the input frame. To this end, we propose a simple yet efficient multichannel correlation network (MCCNet), to ensure that output frames are directly aligned with inputs in the hidden feature space while maintaining the desired style patterns. An inner channel similarity loss is adopted to eliminate side effects caused by the absence of nonlinear operations such as softmax for strict alignment. Furthermore, to improve the performance of MCCNet under complex light conditions, we introduce an illumination loss during training. Qualitative and quantitative evaluations demonstrate that MCCNet performs well in arbitrary video and image style transfer tasks. Code is available at https://github.com/kongxiuxiu/MCCNetV2.

Plant pathology identification using local-global feature level based on transformer

Deep learning plays a crucial role in addressing the challenge of plant disease identification in the field of agriculture. Detecting diseases in plants requires extensive effort, along with a comprehensive understanding of various plant diseases and increased processing time. Balancing both speed and accuracy in predicting leaf diseases in plants can significantly improve crop production and reduce environmental damage. In this paper, we examined deseases on popular plants in agriculture. We proposed a novel model to predict crop pathology on a feature space of global-local based on transformer aggregation. Paticular, we use refined feature of different layer to correlate semantics from high-level feature and low-level feature. Besides, to capture the extended temporal scale across the entire image, we employ a transformer to discern long-range dependencies among frames. Subsequently, the enhanced features incorporating these dependencies are inputted into a classifier for preliminary crop pathology prediction. The plant village dataset and VietNam strawberry disease (VNStr) dataset were utilized for training and disease classification in the experiments. Extensive experiments show that the proposed method outperforms by 99.18% and 94.05% accuracy in plant village and VNStr, respectivly. The model after being judged was applied on Android devices and therefore is easy to use.

Evaluating the association between radiomic features on breast MRI and dormant tumor cell presence in the SURMOUNT trial.

561 Background: While breast cancer treatment is effective at eradicating primary disease, approximately 20% of patients will ultimately recur at a distant site. Recurrence has been linked to the presence of minimal residual disease (MRD) after treatment, which presents as circulating micrometastases and dormant tumor cells (DTCs) located primarily in the bone marrow. Radiomic features on MRI are imaging biomarkers that have been linked with lesion heterogeneity, which in turn has been associated with disease aggressiveness. Here, we examine the association between radiomic features from the primary lesion on MRI and the presence of post-treatment DTCs. Noninvasively identifying patients at high risk of MRD would facilitate optimized treatment strategies to reduce recurrence risk. Methods: Dynamic contrast-enhanced MRI data from breast cancer survivors enrolled in the SURMOUNT trial (N=104) were retrospectively analyzed. Participants with MRIs collected within 90 days of diagnosis who consented to provide a bone marrow aspirate (BMA) sample were eligible. Working with a board-certified radiologist, we successfully identified and segmented the primary lesion for N=70 patients with BMA outcomes. Using the publicly available CaPTk software, we computed 33 radiomic features for each segmented lesion based on pixel intensity, morphology, and grey level textures. To reduce the dimensionality of the feature space, we used these feature values to group the participants into clusters—termed radiomic phenotypes — via agglomerative hierarchical clustering with significance testing. Fisher’s exact test was applied to test the statistical significance of the association between radiomic phenotype and BMA outcome (± DTC). For all statistical tests, p<0.05 was considered significant. Results: Of the N=70 participants analyzed, 58 were negative and 12 were positive for DTCs present in the bone marrow. Two statistically significant radiomic phenotypes ( p<0.05) were identified that were strongly associated with BMA outcome ( p=0.0251). 11 out of 12 positive outcomes were grouped into the second radiomic phenotype (Table). Conclusions: We found a statistically significant association between DTC presence and radiomic phenotypes derived from MRI data. Given the large percentage of positive outcomes assigned to the second radiomic phenotype, the radiomic phenotypes may indicate low and high risk of MRD, respectively. Importantly, this analysis shows promise for the ability to predict MRD from noninvasive imaging features. These results motivate continued data collection to increase sample size and identify a robust set of radiomic features, which may ultimately be used as predictors in potential models of MRD risk. [Table: see text]

Negation handling for sentiment analysis task: approaches and performance analysis

Negation plays an essential role in sentiment analysis within natural language processing (NLP). Its integration involves two key aspects: identifying the scope of negation and incorporating this information into the sentiment model. Before delving into scope detection, the specific negation cue must be identified, with explicit and implicit negation cues being the two main types. Various methodologies, such as rule-based, machine learning, and hybrid approaches, address the negation scope detection challenge. Strategies for leveraging negation information in sentiment models encompass heuristic polarity modification, feature space augmentation, end-to-end approach, and hierarchical multi-task learning. Notably, there is a need for more studies addressing implicit negation cue detection, even within the state-of-the-art bidirectional encoder representation for transformers (BERT) approach. Some studies have employed reinforcement learning and hybrid techniques to address the implicit negation problem. Further exploration, particularly through a hybrid and multi-task learning approach, is warranted to make potential contributions to the nuanced challenges of handling negation in sentiment analysis, especially in complex sentence structures.

Multimodal Image Classification by Multiview Latent Pattern Extraction, Selection, and Correlation.

The large amount of data available in the modern big data era opens new opportunities to expand our knowledge by integrating information from heterogeneous sources. Multiview learning has recently achieved tremendous success in deriving complementary information from multiple data modalities. This article proposes a framework called multiview latent space projection (MVLSP) to integrate features extracted from multiple sources in a discriminative way to facilitate binary and multiclass classifications. Our approach is associated with three innovations. First, most existing multiview learning algorithms promote pairwise consistency between two views and do not have a natural extension to applications with more than two views. MVLSP finds optimum mappings from a common latent space to match the feature space in each of the views. As the matching is performed on a view-by-view basis, the framework can be readily extended to multiview applications. Second, feature selection in the common latent space can be readily achieved by adding a class view, which matches the latent space representations of training samples with their corresponding labels. Then, high-order view correlations are extracted by considering feature-label correlations. Third, a technique is proposed to optimize the integration of different latent patterns based on their correlations. The experimental results on the prostate image dataset demonstrate the effectiveness of the proposed method.

A genetic algorithm-based feature selection approach for diabetes prediction

<p>Genetic algorithms have emerged as a powerful optimization technique for feature selection due to their ability to search through a vast feature space efficiently. This study discusses the importance of feature selection for prediction in healthcare and prominently focuses on diabetes mellitus. Feature selection is essential for improving the performance of prediction models, by finding significant features and removing unnecessary among them. The study aims to identify the most informative subset of features. Diabetes is a chronic metabolic disorder that poses significant health challenges worldwide. For the experiment, two datasets related to diabetes were downloaded from Kaggle and the results of both (datasets) with and without feature selection using the genetic algorithm were compared. Machine learning classifiers and genetic algorithms were combined to increase the precision of diabetes risk prediction. In the preprocessing phase, feature selection, machine learning classifiers, and performance metrics methods were applied to make this study feasible. The results of the experiment showed that genetic algorithm + logistic regression i.e., 80% (accuracy) works better for PIMA diabetes, and for Germany diabetes dataset genetic algorithm + random forest and genetic algorithm + K-Nearest Neighbor i.e., 98.5% performed better than other chosen classifiers. The researchers can better comprehend the importance of feature selection in healthcare through this study.</p>

Online learning for data streams with bi-dynamic distributions

Data streams, as an important pattern of big data, require online real-time processing because instances arrive one by one and are fleeting. Existing online learning methods make distinctive assumptions, such as a fixed feature space, a varying feature space that follows specific patterns, and a fixed data distribution. However, data streams generated from real-world scenarios typically have both randomly changing feature spaces and data distributions, making existing methods inappropriate for practical applications. To fill this gap, this study proposes a novel Online Learning for Data Streams with Bi-dynamic Distributions (OLBD) algorithm. OLBD has a two-fold main idea: 1) it overcomes random changes in the feature space by building a mapping matrix to space transform and projects the original instances onto the global feature space; 2) it handles dynamic data distributions by constraining prior knowledge and transferring established mapping relationships to new distributions. To evaluate OLBD, we compared it with related state-of-the-art algorithms. First, we use 13 datasets to simulate three scenarios of dynamic feature space, namely trapezoidal, feature evolvable, and capricious data streams. Second, we simulated the data streams with dynamic data distributions using eight real and four generated datasets. We then conducted ablation studies on the parameter α. Finally, we analyzed data streams with bi-dynamic data distributions under different feature missing ratios and verified the generalization. The results show that OLBD significantly outperforms its rivals. Additionally, a practical case study on movie review classification was conducted to illustrate the effectiveness of OLBD in real-world scenarios.

Deepfake Video Detection Using Res-Next CNN and LSTM

Abstract: Deepfake videos are becoming more and more common in the digital age, which has led to major worries about how they can undermine the trustworthiness of visual media. With deep learning algorithms' rising processing power, producing lifelike human-synthesized films or deepfakes has never been easier. Disinformation and political unrest can be caused by these videos. A novel deep learning-based method has been created to distinguish between actual and AI-generated fraudulent videos to address this problem. The suggested technique uses Attention-based networks (Res-Next CNN), a kind of deep learning architecture that can selectively focus on significant features in a video, to fine-tune the transformer module to search for new sets of feature space to detect false images.

Spatiotemporal evolution and driving mechanisms of desertification on the Mongolian Plateau

Desertification poses a severe ecological and environmental challenge in the Mongolian Plateau (MP). It is difficult to quantify desertification distribution using unified indicators in the entire MP, because of its complex physical geographic conditions and various climatic zones covered. To accurately address this challenge, the spatial distribution of desertification at a 30-m resolution from 1990 to 2020 were mapped in this study. The desertification potential occurrence zone was identified by using a moisture index on the MP firstly. The feature space model and five machine learning models were constructed to make the map based on Google Earth Engine and Landsat data. The spatiotemporal distribution of desertification were further analyzed, and the dominant drivers of desertification distribution and evolution were identified using Geodetector model. The results indicate that the potential occurrence area of desertification accounted for 83.88 % of the total land area. The gradient boosted tree model for desertification assessment has the best performance with the highest overall accuracy of 88.18 %. The year 2010 marked a pivotal transition from land degradation to land restoration in the MP. Between 2010 and 2020, desertified land continued to deteriorate extensively in the southern Mongolia, while Inner Mongolia, China, essentially entered a full recovery phase. Precipitation and land use emerged as the primary drivers of the spatial distribution of desertification on the Mongolian Plateau and Mongolia, with potential evapotranspiration and precipitation influencing the distribution of desertification in Inner Mongolia, China. Land use change was the primary driver of desertification evolution on the MP and Mongolia. This study constructs an indicator system and methodology suitable for desertification monitoring on the MP, addresses the lack of refined desertification data over a long time series, and provides scientific reference for decision-making support in combating desertification in this region, and other large arid and semi-arid areas in the world.

HD-Former: A hierarchical dependency Transformer for medical image segmentation

Medical image segmentation is a compelling fundamental problem and an important auxiliary tool for clinical applications. Recently, the Transformer model has emerged as a valuable tool for addressing the limitations of convolutional neural networks by effectively capturing global relationships and numerous hybrid architectures combining convolutional neural networks (CNNs) and Transformer have been devised to enhance segmentation performance. However, they suffer from multilevel semantic feature gaps and fail to account for multilevel dependencies between space and channel. In this paper, we propose a hierarchical dependency Transformer for medical image segmentation, named HD-Former. First, we utilize a Compressed Bottleneck (CB) module to enrich shallow features and localize the target region. We then introduce the Dual Cross Attention Transformer (DCAT) module to fuse multilevel features and bridge the feature gap. In addition, we design the broad exploration network (BEN) that cascades convolution and self-attention from different percepts to capture hierarchical dense contextual semantic features locally and globally. Finally, we exploit uncertain multitask edge loss to adaptively map predictions to a consistent feature space, which can optimize segmentation edges. The extensive experiments conducted on medical image segmentation from ISIC, LiTS, Kvasir-SEG, and CVC-ClinicDB datasets demonstrate that our HD-Former surpasses the state-of-the-art methods in terms of both subjective visual performance and objective evaluation. Code: https://github.com/barcelonacontrol/HD-Former.

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Speech is produced by a nonlinear, dynamical Vocal Tract (VT) system, and is transmitted through multiple (air, bone and skin conduction) modes, as captured by the air, bone and throat microphones respectively. Speaker specific characteristics that capture this nonlinearity are rarely used as stand-alone features for speaker modeling, and at best have been used in tandem with well known linear spectral features to produce tangible results. This paper proposes Recurrent Plot (RP) embeddings as stand-alone, non-linear speaker-discriminating features. Two datasets, the continuous multimodal TIMIT speech corpus and the consonant-vowel unimodal syllable dataset, are used in this study for conducting closed-set speaker identification experiments. Experiments with unimodal speaker recognition systems show that RP embeddings capture the nonlinear dynamics of the VT system which are unique to every speaker, in all the modes of speech. The Air (A), Bone (B) and Throat (T) microphone systems, trained purely on RP embeddings perform with an accuracy of 95.81%, 98.18% and 99.74%, respectively. Experiments using the joint feature space of combined RP embeddings for bimodal (A–T, A–B, B–T) and trimodal (A–B–T) systems show that the best trimodal system (99.84% accuracy) performs on par with trimodal systems using spectrogram (99.45%) and MFCC (99.98%). The 98.84% performance of the B–T bimodal system shows the efficacy of a speaker recognition system based entirely on alternate (bone and throat) speech, in the absence of the standard (air) speech. The results underscore the significance of the RP embedding, as a nonlinear feature representation of the dynamical VT system that can act independently for speaker recognition. It is envisaged that speech recognition too will benefit from this nonlinear feature.