Subspace Projection Approach Research Articles

FEDA: A Nonlinear Subspace Projection Approach for Electronic Nose Data Classification

The electronic nose (e-nose) is susceptible to sensor drift and instrumental variation, which may result in distribution discrepancy in data collected, hence leading to classification performance degradation. It is necessary to apply domain adaptation to solve this problem. A nonlinear subspace projection approach named feature entropy domain adaption (FEDA) is proposed for domain adaptation for e-nose data classification. One important aspect of FEDA is that adversarial training is introduced to minimize the distribution discrepancy between source and target domains. No projection matrix and parameter fine-tuning are needed anymore in comparison with the popular linear subspace projection approaches. In addition, feature norm and conditional entropy are introduced into adversarial training in FEDA to reduce the decision boundary uncertainty and the overlap between classes, respectively. Experimental results show that the FEDA can deal with the distribution discrepancy of e-nose effectively, and can achieve satisfactory classification accuracy on various datasets. Source code can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/threedteam/DA_FEDA</uri> .

Oblique Projection-Based Covariance Matrix Reconstruction and Steering Vector Estimation for Robust Adaptive Beamforming

Adaptive beamforming can efficiently contract interference and noise. Due to high sensitivity of the beamformer to model mismatch, the capability of interference reduction will critically degrade when the signal model mismatch occurs, particularly when the sampling sequence contains the desired signal. For the purpose of enhancing the robustness of beamformers to signal model mismatch, we propose a new robust adaptive beamforming (RAB) method. Firstly, the precise steering vector (SV) associating with the desired signal is estimated by employing the minimum norm of subspace projection (MNSP) approach. Secondly, the nominal interference SVs are estimated via the maximum entropy power spectrum. Subsequently, the corrected interference SVs and powers are obtained by oblique projection. Finally, the interference-plus-noise covariance matrix (INCM) is reconstructed, and the proposed RAB is obtained. Multiple simulations are carried out and demonstrate the robustness of the proposed RAB method.

A subspace projection approach to quantify respiratory variations in the f-wave frequency trend.

Background: The autonomic nervous system (ANS) is known as a potent modulator of the initiation and perpetuation of atrial fibrillation (AF), hence information about ANS activity during AF may improve treatment strategy. Respiratory induced ANS variation in the f-waves of the ECG may provide such information. Objective: This paper proposes a novel approach for improved estimation of such respiratory induced variations and investigates the impact of deep breathing on the f-wave frequency in AF patients. Methods: A harmonic model is fitted to the f-wave signal to estimate a high-resolution f-wave frequency trend, and an orthogonal subspace projection approach is employed to quantify variations in the frequency trend that are linearly related to respiration using an ECG-derived respiration signal. The performance of the proposed approach is evaluated and compared to that of a previously proposed bandpass filtering approach using simulated f-wave signals. Further, the proposed approach is applied to analyze ECG data recorded for 5 min during baseline and 1 min deep breathing from 28 AF patients from the Swedish cardiopulmonary bioimage study (SCAPIS). Results: The simulation results show that the estimates of respiratory variations obtained using the proposed approach are more accurate than estimates obtained using the previous approach. Results from the analysis of SCAPIS data show no significant differences between baseline and deep breathing in heart rate (75.5 ± 22.9 vs. 74 ± 22.3) bpm, atrial fibrillation rate (6.93 ± 1.18 vs. 6.94 ± 0.66) Hz and respiratory f-wave frequency variations (0.130 ± 0.042 vs. 0.130 ± 0.034) Hz. However, individual variations are large with changes in heart rate and atrial fibrillatory rate in response to deep breathing ranging from −9% to +5% and −8% to +6%, respectively and there is a weak correlation between changes in heart rate and changes in atrial fibrillatory rate (r = 0.38, p < 0.03). Conclusion: Respiratory induced f-wave frequency variations were observed at baseline and during deep breathing. No significant changes in the magnitude of these variations in response to deep breathing was observed in the present study population.

Learning-Based Clutter Mitigation with Subspace Projection and Sparse Representation in Holographic Subsurface Radar Imaging

The holographic subsurface radar (HSR) is an effective remote sensing modality for surveying shallowly buried objects with high resolution images in plan-view. However, strong reflections from the rough surface and inhomogeneities obscure the detection of stationary targets response. In this paper, a learning-based method is proposed to mitigate the clutter in HSR applications. The proposed method first decomposes the HSR image into raw clutter and target data using an adaptive subspace projection approach. Then, the autoencoder is applied to carry out unsupervised learning to extract the target features and mitigate the clutter. The sparse representation is also combined to further optimize the model and the alternating direction multiplier method (ADMM) is used to solve the optimization problem for precision and efficiency. Experiments using real data were conducted to demonstrate that the proposed method can effectively mitigate the strong clutter with the target preserved. The visual and quantitative results show that the proposed method achieves superior performance on suppressing clutter in HSR images compared with the widely used state-of-the-art clutter mitigation approaches.

A Subspace Projection Approach for Clutter Mitigation in Holographic Subsurface Imaging

The holographic subsurface radar (HSR) is recognized as an effective remote sensing modality for the detection of shallowly buried objects with a high-resolution image in plain view. However, subsurface detection with HSR is prone to be impaired by clutter contamination, which often obscures the target response. In this letter, a novel clutter mitigation method combining singular value decomposition (SVD) and response cross correlation analysis is presented. The proposed method first applies SVD to decompose the radar data matrix to a number of singular components. Furthermore, the signal cross correlation characteristics are analyzed to demonstrate that the variance of left singular vectors is directly proportional to the target proportion in radar data. Then, target and clutter subspaces can be identified by maximizing the defined weighted target-to-clutter ratio (WTCR). Results of numerical simulation and laboratory experiments corroborate the effectiveness of the proposed method in reducing clutter while preserving the target image.

Face Recognition by Using 2D Orthogonal Subspace Projections

In this paper, the implementations and comparison of some classifiers along with 2D subspace projection approaches have been carried out for the face recognition problem. For this purpose, the well-known classifiers such as K-Nearest Neighbor (K-NN), Common Matrix Approach (CMA), Support Vector Machine (SVM) and Convolutional Neural Network (CNN) are conducted on low dimensional face representations that are determined from 2DPCA-, 2DSVD- and 2DFDA approaches. CMA, which is a 2D version of the Common Vector Approach (CVA), finds a common matrix for each face class. From the experimental results, we have observed that the SVM presents a dominant performance in general. When overall results of all datasets are considered, CMA is slightly superior to others in case of 2DPCA- and 2DSVD-based features matrices of the AR dataset. On the other side, CNN is better than other classifiers when it comes to develop a face recognition system based on original face samples and 2DPCA-based feature matrices of the Yale dataset. The experimental results indicate that use of these feature matrices with CMA, SVM, and CNN in classification problems is more advantageous than the use of original pixel matrices in the sense of both processing time and memory requirement.

Discriminative Layer Pruning for Convolutional Neural Networks

The predictive ability of convolutional neural networks (CNNs) can be improved by increasing their depth. However, increasing depth also increases computational cost significantly, in terms of both floating point operations and memory consumption, hindering applicability on resource-constrained systems such as mobile and internet of things (IoT) devices. Fortunately, most networks have spare capacity, that is, they require fewer parameters than they actually have to perform accurately. This motivates network compression methods, which remove or quantize parameters to improve resource-efficiency. In this work, we consider a straightforward strategy for removing entire convolutional layers to reduce network depth. Since it focuses on depth, this approach not only reduces memory usage, but also reduces prediction time significantly by mitigating the serialization overhead incurred by forwarding through consecutive layers. We show that a simple subspace projection approach can be employed to estimate the importance of network layers, enabling the pruning of CNNs to a resource-efficient depth within a given network size constraint. We estimate importance on a subspace computed using Partial Least Squares , a feature projection approach that preserves discriminative information. Consequently, this importance estimation is correlated to the contribution of the layer to the classification ability of the model. We show that cascading discriminative layer pruning with filter-oriented pruning improves the resource-efficiency of the resulting network compared to using any of them alone, and that it outperforms state-of-the-art methods. Moreover, we show that discriminative layer pruning alone, without cascading, achieves competitive resource-efficiency compared to methods that prune filters from all layers.

Regularized Local Multivariate Reduced-Order Models With Nonaffine Parameter Dependence

This paper addresses a singular problem, not yet discussed in the literature, which occurs when parametric reduced-order models are created using a subspace projection approach with multiple concatenated projection bases. We show that this technique may lead to the appearance of localized artifacts in the frequency characteristics of a system, even when the reduced-order projection basis is rich enough to describe the original system. These artifacts are found to be related to nonphysical poles of the transfer function that emerge whenever more than one projection basis is used, each spanning is a similar space, and these bases are directly put together to build multivariate reduced-order models. These unwanted poles are identified using the Bauer–Fike theorem and then the parametrized reduced-order model is regularized with a simple deflation procedure that completely removes the artifacts due to nonphysical resonances from the circuit characteristics. Finally, real-life numerical examples illustrate the accuracy and abilities of the proposed approach.

Anti-Drift in Electronic Nose via Dimensionality Reduction: A Discriminative Subspace Projection Approach

Sensor drift is a well-known issue in the field of sensors and measurement and has plagued the sensor community for many years. In this paper, we propose a sensor drift correction method to deal with the sensor drift problem. Specifically, we propose a discriminative subspace projection approach for sensor drift reduction in electronic noses. The proposed method inherits the merits of the subspace projection method called domain regularized component analysis. Moreover, the proposed method takes the source data label information into consideration, which minimizes the within-class variance of the projected source samples and at the same time maximizes the between-class variance. The label information is exploited to avoid overlapping of samples with different labels in the subspace. Experiments on two sensor drift datasets have shown the effectiveness of the proposed approach.

Band-Specified Virtual Dimensionality for Band Selection: An Orthogonal Subspace Projection Approach

This paper develops a new Neyman–Pearson detection approach, to be called band-specified virtual dimensionality (BSVD), to estimating the number of bands required by band selection (BS), $n_{\mathrm {BS}}$ , as well as finding desired bands at the same time. Its idea is derived from target-specified virtual dimensionality (TSVD) where targets under hypotheses as signal sources in TSVD are replaced with bands as signal sources and the test statistics derived for a Neyman–Pearson detector (NPD) is signal-to-noise ratio (SNR) that is used to derive orthogonal subspace projection (OSP) approach for hyperspectral image classification and dimensionality reduction. Accordingly, the resulting virtual dimensionality is referred to as OSP-based BSVD. Several benefits resulting from BSVD cannot be offered by the traditional BS methods. One is its direct approach to dealing with $n_{\mathrm {BS}}$ . Another is no-search strategy needed for finding optimal bands. Instead, it uses NPD to determine and rank desired bands for band prioritization. Most importantly, it determines $n_{\mathrm {BS}}$ and finds desired bands simultaneously and progressively.

Anti-drift in E-nose: A subspace projection approach with drift reduction

Anti-drift is an emergent and challenging issue in sensor-related subjects. In this paper, we propose to address the time-varying drift (e.g. electronic nose drift), which is sometimes an ill-posed problem due to its uncertainty and unpredictability. Considering that drift is with different probability distribution from the regular data, a machine learning based subspace projection approach is proposed. The main idea behind is that given two data clusters with different probability distribution, we tend to find a latent projection P (i.e. a group of basis), such that the newly projected subspace of the two clusters is with similar distribution. In other words, drift is automatically removed or reduced by projecting the data onto a new common subspace. The merits are threefold: 1) the proposed subspace projection is unsupervised; without using any data label information; 2) a simple but effective domain distance is proposed to represent the mean distribution discrepancy metric; 3) the proposed anti-drift method can be easily solved by Eigen decomposition; and anti-drift is manifested with a well solved projection matrix in real application. Experiments on synthetic data and real datasets demonstrate the effectiveness and efficiency of the proposed anti-drift method in comparison to state-of-the-art methods.

Interference DOA estimation and suppression for GNSS receivers using fully augmentable arrays

Multiple antenna receivers have been proposed for interference nulling in Global Navigation Satellite Systems (GNSS). An open-loop anti-jam approach is considered and an interference direction-of-arrival (DOA) estimation technique using fully augmentable non-uniform linear arrays is introduced. The DOAs of incoming jammers are estimated utilising the minimum output power method applied to the GNSS receiver coarray and through two different approaches of spectrum sensing and polynomial rooting. Subsequently, the strong interferers are cancelled utilising a subspace projection approach, whereas the 30 dB despreading gain of GNSS receivers protects the satellite signals against weak jammers. Supporting simulation results demonstrate that DOA estimation based on polynomial rooting outperforms spectrum sensing and validate the effectiveness of the proposed anti-jam strategy.

Underdetermined blind separation of overlapped speech mixtures in time-frequency domain with estimated number of sources

Noise suppression and the estimation of the number of sources are two practical issues in applications of underdetermined blind source separation (UBSS). This paper proposes a noise-robust instantaneous UBSS algorithm for highly overlapped speech sources in the short-time Fourier transform (STFT) domain. The proposed algorithm firstly estimates the unknown complex-valued mixing matrix and the number of sources, which are then used to compute the STFT coefficients of corresponding sources at each auto-source time-frequency (TF) point. After that, the original sources are recovered by the inverse STFT. To mitigate the noise effect on the detection of auto-source TF points, we propose a method to effectively detect the auto-term location of the sources by using the principal component analysis (PCA) of the STFTs of noisy mixtures. The PCA-based detection method can achieve similar UBSS outcome as some filtering-based methods. More importantly, an efficient method to estimate the mixing matrix is proposed based on subspace projection and clustering approaches. The number of sources is obtained by counting the number of the resultant clusters. Evaluations have been carried out by using the speech corpus NOIZEUS and the experimental results have shown improved robustness and efficiency of the proposed algorithm.

Fuzzy Signature-Based Discriminative Subspace Projection for Hyperspectral Data Classification

Mixed pixels in the hyperspectral image (HSI) are often misclassified under a strict clustering assumption. In this paper, we relax the assumption and assign a fuzzy signature for each pixel in HSI, whose element indicates the probability it belongs to some class. A fuzzy signature-based discriminative subspace projection (FS-DSP) approach is then developed for simultaneous dimensionality reduction and classification of HSI. In FS-DSP, a signature Laplacian regularizer is derived from both labeled and unlabeled pixels to pull the neighbors with similar fuzzy signatures together. A discriminant term is constructed to further pull different classes away and push the same classes toward after the projection. The two terms are combined to define a subspace projection optimization problem, and an alternating direction method of multipliers (ADMM) algorithm is employed to iteratively calculate fuzzy signatures. Effectiveness of FS-DSP is evaluated by five datasets, and the results show that it exhibits state-of-the-art performance as to the numerical guidelines, such as overall accuracy (OA), average accuracy (AA), and Kappa coefficients (KC), when there are only very few labeled pixels.

A Subspace Projection Approach for Analysis of Speech Under Stressed Condition

In this paper, a novel subspace projection approach is proposed for analysis of speech signal under stressed condition. The subspace projection method is based on the assumption of orthogonality between speech subspace and stress subspace. Speech and stress subspaces contain speech and stress information, respectively. The projection of stressed speech vectors onto the speech subspace will separate speech-specific information. In this work, the speech subspace consists of neutral speech vectors. Speech and stress recognition techniques are used to verify the orthogonal relation between speech and stress subspaces. The evaluation database consists of 119 word vocabulary under neutral, angry, sad and Lombard conditions. Hidden Markov models for speech and stress recognition are used with mel-frequency cepstral coefficient features for evaluation of estimated speech and stress information.

Robust Ambiguous Clutter Suppression for the Near-Shore Water Areas With Spaceborne Multichannel SAR Systems

In practice, the ambiguous image of the strong ground clutter may dominate in the adjacent water areas in the case of high-azimuth ambiguity level for the spaceborne synthetic aperture radar. In this letter, a robust ambiguous clutter suppression method is proposed for the ground moving target indication. The subspace projection approach is used to suppress the ambiguous clutter, and the notch broadening technique is used to widen the ambiguous clutter notches for the robustness. The width of the notch is determined by the probability density function of the interferogram's phase. The validity and robustness of the proposed method are verified by the simulation results.

A Subspace Projection Approach for Wall Clutter Mitigation in Through-the-Wall Radar Imaging

One of the main challenges in through-the-wall radar imaging (TWRI) is the strong exterior wall returns, which tend to obscure indoor stationary targets, rendering target detection and classification difficult, if not impossible. In this paper, an effective wall clutter mitigation approach is proposed for TWRI that does not require knowledge of the background scene nor does it rely on accurate modeling and estimation of wall parameters. The proposed approach is based on the relative strength of the exterior wall returns compared to behind-wall targets. It applies singular value decomposition to the data matrix constructed from the space-frequency measurements to identify the wall subspace. Orthogonal subspace projection is performed to remove the wall electromagnetic signature from the radar signals. Furthermore, this paper provides an analysis of the wall and target subspace characteristics, demonstrating that both wall and target subspaces can be multidimensional. While the wall subspace depends on the wall type and building material, the target subspace depends on the location of the target, the number of targets in the scene, and the size of the target. Experimental results using simulated and real data demonstrate the effectiveness of the subspace projection method in mitigating wall clutter while preserving the target image. It is shown that the performance of the proposed approach, in terms of the improvement factor of the target-to-clutter ratio, is better than existing approaches and is comparable to that of background subtraction, which requires knowledge of a reference background scene.

Semisupervised Dual-Geometric Subspace Projection for Dimensionality Reduction of Hyperspectral Image Data

Exploring the geometric prior in the dimensionality reduction (DR) of hyperspectral image data (HID) is an important issue because it can overcome the possible overclassification of spectrally homogeneous areas in the HID classification. In this paper, the local geometric similarity of hyperspectral vectors is explored in both the manifold domain and image domain, and a semisupervised dual-geometric subspace projection (DGSP) approach is proposed for the DR of HID, by utilizing both labeled and unlabeled samples. First, the geometric information in the manifold domain is captured by a sparse coding-based geometric graph, and then, a local-consistency-constrained geometric matrix is defined to reveal the geometric structure in the image domain. Second, unlabeled samples are used to refine the geometric structure by defining a pairwise similarity matrix. Third, three scatter matrices are then derived from these similarity matrices to find the optimal subspace projection that captures the most important properties of the subspaces with respect to classification. Some experiments are taken on the airborne visible infrared imaging spectrometer (AVIRIS) HID to prove the efficiency of the proposed method.

OTHR Spectrum Reconstruction of Maneuvering Target with Compressive Sensing

High-frequency (HF) over-the-horizon radar (OTHR) works in a very complicated electromagnetic environment. It usually suffers performance degradation caused by transient interference. In this paper, we study the transient interference excision and full spectrum reconstruction of maneuvering targets. The segmental subspace projection (SP) approach is applied to suppress the clutter and locate the transient interference. After interference excision, the spectrum is reconstructed from incomplete measurements via compressive sensing (CS) by using a redundant Fourier-chirp dictionary. An improved orthogonal matching pursuit (IOMP) algorithm is developed to solve the sparse decomposition optimization. Experimental results demonstrate the effectiveness of the proposed methods.

Quality relevant nonlinear batch process performance monitoring using a kernel based multiway non-Gaussian latent subspace projection approach

Multiway kernel partial least squares method (MKPLS) has recently been developed for monitoring the operational performance of nonlinear batch or semi-batch processes. It has strong capability to handle batch trajectories and nonlinear process dynamics, which cannot be effectively dealt with by traditional multiway partial least squares (MPLS) technique. However, MKPLS method may not be effective in capturing significant non-Gaussian features of batch processes because only the second-order statistics instead of higher-order statistics are taken into account in the underlying model. On the other hand, multiway kernel independent component analysis (MKICA) has been proposed for nonlinear batch process monitoring and fault detection. Different from MKPLS, MKICA can extract not only nonlinear but also non-Gaussian features through maximizing the higher-order statistic of negentropy instead of second-order statistic of covariance within the high-dimensional kernel space. Nevertheless, MKICA based process monitoring approaches may not be well suited in many batch processes because only process measurement variables are utilized while quality variables are not considered in the multivariate models. In this paper, a novel multiway kernel based quality relevant non-Gaussian latent subspace projection (MKQNGLSP) approach is proposed in order to monitor the operational performance of batch processes with nonlinear and non-Gaussian dynamics by combining measurement and quality variables. First, both process measurement and quality variables are projected onto high-dimensional nonlinear kernel feature spaces, respectively. Then, the multidimensional latent directions within kernel feature subspaces corresponding to measurement and quality variables are concurrently searched for so that the maximized mutual information between the measurement and quality spaces is obtained. The I2 and SPE monitoring indices within the extracted latent subspaces are further defined to capture batch process faults resulting in abnormal product quality. The proposed MKQNGLSP method is applied to a fed-batch penicillin fermentation process and the operational performance monitoring results demonstrate the superiority of the developed method as apposed to the MKPLS based process monitoring approach.