Dimensionality Reduction Stage Research Articles

A new real-time image semantic segmentation framework based on a lightweight deep convolutional encoder-decoder architecture for robotic environment sensing

Background: Image semantic segmentation can be understood as the allocation of a predefined category label to each pixel in the image to achieve the region segmentation of the image. Different categories in the image are identified with different colors. While achieving pixel classification, the position information of pixel points of different categories in the image is retained. Purpose: Due to the influence of background and complex environment, the traditional semantic segmentation methods have low accuracy. To alleviate the above problems, this paper proposes a new real-time image semantic segmentation framework based on a lightweight deep convolutional encoder-decoder architecture for robotic environment sensing. Methodology: This new framework is divided into three stages: encoding stage, decoding stage and dimension reduction stage. In the coding stage, a cross-layer feature map fusion (CLFMF) method is proposed to improve the effect of feature extraction. In the decoding stage, a new lightweight decoder (LD) structure is designed to reduce the number of convolutional layers to speed up model training and prediction. In the dimension reduction stage, the convolution dimension reduction method (CDR) is presented to connect the encoder and decoder layer by layer to enhance the decoder effect. Results: Compared with other state-of-the-art image semantic segmentation methods, we conduct comparison experiments on datasets Cityscapes, SUN RGB-D, CamVid, KITTI. The Category iIoU combined with the proposed method is more than 70%, and the Category IoU is as high as 89.7%. Conclusion: The results reflect that the new method can achieve the better semantic segmentation effect.

Data-driven forecasting of nonequilibrium solid-state dynamics

We present a data-driven approach to efficiently approximate nonlinear transient dynamics in solid-state systems. Our proposed machine-learning model combines a dimensionality reduction stage with a nonlinear vector autoregression scheme. We report an outstanding time-series forecasting performance combined with an easy-to-deploy model and an inexpensive training routine. Our results are of great relevance as they have the potential to massively accelerate multiphysics simulation software and thereby guide the future development of solid-state-based technologies.

Fault diagnosis of complex hydraulic system based on fast Mahalanobis classification system with high-dimensional imbalanced data

Due to the complexity of hydraulic systems, the data generated is prone to be high-dimensional and imbalanced, posing a challenge for fault diagnosis. This paper proposes a fast Mahalanobis classification system (FMCS). At the dimensionality reduction stage, smymetrical uncertainty and Mahalanobis kernel principal component analysis are used to construct a two-stage fast dimensionality reduction method. At the weight distribution stage, Mahalanobis distance is used to measure the similarity between the main feature and other features to determine the feature weights. At the fault diagnosis stage, Mahalanobis-Taguchi system is improved in threshold determination by atom search optimization to achieve fault diagnosis. Experiments show that FMCS has better classification performance (accuracy = 0.92, F1 = 0.94, G-mean = 0.91) and shorter computing time (4.9045 s) compared with 24 baseline algorithms. FMCS promotes the development of intelligent fault diagnosis for complex hydraulic systems.

Machine learning discovery of new phases in programmable quantum simulator snapshots

Machine learning has recently emerged as a promising approach for studying complex phenomena characterized by rich datasets. In particular, data-centric approaches lead to the possibility of automatically discovering structures in experimental datasets that manual inspection may miss. Here, we introduce an interpretable unsupervised-supervised hybrid machine learning approach, the hybrid-correlation convolutional neural network (hybrid-CCNN), and apply it to experimental data generated using a programmable quantum simulator based on Rydberg atom arrays. Specifically, we apply hybrid-CCNN to discover and identify new quantum phases on square lattices with programmable interactions. The initial unsupervised dimensionality reduction and clustering stage first reveals five distinct quantum phase regions. In a second supervised stage, we refine these phase boundaries and seek insights into the phases by training multiple CCNN classifiers. A learned spatial weighting, introduced to the CCNNs in this work, enables discovery of spatial structure at scales beyond the filter size. The characteristic spatial weightings and snippets of correlations specifically recognized in each phase capture quantum fluctuations in the striated phase and identify a previously undetected boundary-ordered phase as well as motifs of more exotic ordered phases. These observations demonstrate that a combination of programmable quantum simulators with machine learning can be used as a powerful tool for detailed exploration of correlated quantum states of matter.

A Method to Classify Steel Plate Faults Based on Ensemble Learning

With the industrial revolution 4.0, machine learning methods are widely used in all aspects of manufacturing to perform quality prediction, fault diagnosis, or maintenance. In the steel industry, it is important to precisely detect faults/defects in order to produce high-quality steel plates. However, determining the exact first-principal model between process parameters and mechanical properties is a challenging process. In addition, steel plate defects are detected through manual, costly, and less productive offline inspection in the traditional manufacturing process of steel. Therefore, it is a great necessity to enable the automatic detection of steel plate faults. To this end, this study explores the capabilities of the following three machine learning models Adaboost, Bagging, and Random Forest in detecting steel plate faults. The well-known steel plate failure dataset provided by Communication Sciences Research Centre Semeion was used in this study. The aim of many studies using this dataset is to correctly classify defects in steel plates using traditional machine learning models, ignoring the applicability of the developed models to real-world problems. Manufacturing is a dynamic process with constant adjustments and improvements. For this reason, it is necessary to establish a learning process that determines the best model based on the arrival of new information. Contrary to previous studies on the steel plate failure dataset, this article presents a systematic modelling approach that includes the normalization step in the data preparation stage to reduce the effects of outliers, the feature selection step in the dimension reduction stage to develop a machine learning model with fewer inputs, and hyperparameter optimization step in the model development stage to increase the accuracy of the machine learning model. The performances of the developed machine learning models were compared according to statistical metrics in terms of precision, recall, sensitivity, and accuracy. The results revealed that AdaBoost performed well on this dataset, achieving accuracy scores of 93.15% and 91.90% for the training and test datasets, respectively.

Yogurt classification using an electronic tongue system and machine learning techniques

• The development of a pattern recognition methodology for the classification of yogurts. • The use of a dataset of yogurts from the region of Boyacá-Colombia. • The development of an electronic tongue system for the classification of yogurt Electronic tongue sensor arrays allow the analysis of liquid substances and provide information for classification tasks to artificially emulate a taste recognition system. These kinds of devices are needed for the alimentary industry because they ensure their products’ quality; however, they are still in the development phase because of the components required for their robustness. This paper presents an electronic tongue system for yogurt classification to contribute to developing automatic systems for this industry. The developed system uses a sensor array with Screen-Printed Electrode sensors, a potentiostat, and a multiplexer to handle eight different sensors. A pattern recognition methodology for the processing of multi-step amperometry signals is defined. It comprises the following stages: First, a data unfolding process to build a two-dimensional matrix with the information from the sensors is applied. Next, raw data is normalized using the mean-centered group scaling method. Then, a dimensionality reduction stage is applied. Two methods are evaluated, the t-distributed stochastic neighbor embedding ( t -SNE) and Principal Component Analysis (PCA). Subsequently, a supervised machine learning classification algorithm is used. Five methods were evaluated to determine the best classifier: LDA, classification trees(C4.5), naive Bayes, k -NN, and SVM. A dataset of 151 samples of 8 different yogurts is used for the methodology validation. This validation is executed through a Leave One Out Cross Validation (LOOCV) method. Final classification accuracy of 100% was obtained When PCA and LDA or t -SNE and k -NN were used; however, from the computational point of view, a shorter execution time was evidenced for PCA and LDA, showing the convenience in its use in the methodology.

Integrated random projection and dimensionality reduction by propagating light in photonic lattices.

It is proposed that the propagation of light in disordered photonic lattices can be harnessed as a random projection that preserves distances between a set of projected vectors. This mapping is enabled by the complex evolution matrix of a photonic lattice with diagonal disorder, which turns out to be a random complex Gaussian matrix. Thus, by collecting the output light from a random subset of the waveguide channels, one can perform an embedding from a higher- to a lower-dimensional space that respects the Johnson-Lindenstrauss lemma and nearly preserves the Euclidean distances. The distance-preserving random projection through photonic lattices requires intermediate disorder levels that allow diffusive propagation of light. The proposed scheme can be utilized as a simple and powerful integrated dimension reduction stage that can greatly reduce the burden of a subsequent neural computation stage.

Hybrid Inception Recurrent Residual Convolutional Neural Network (HIRResCNN) with Harmony Search Optimization (HSO) for Early Breast Cancer Detection System

For women, most common cause of death is Breast tumour and in worldwide, it is the second leading reason for cancer deaths. Due the requirement of breast cancer’s early detection and false diagnosis impact on patients, made researchers to investigate Deep Learning (DL) techniques for mammograms. There are four stages in this proposed HIRResCNN framework, namely, Pre-processing, reduction of dimensionality, segmentation and classification. From images, noises are removed using two filtering algorithms called Median and mean filtering in pre-processing stage. Then canny edge detector is used for detecting edges. Gaussian filtering is used in canny edge detector to smoothen the images. In the next dimensionality reduction stage, attributes are correlated using Principal Component Analysis (PCA) inclusive of related features. So, this huge dataset is minimized and only few variables are used for expressing it. In order to detect the breast cancer accurately, foreground and background subtraction is done in the third stage called segmentation stage. At last, for detecting and classifying breast cancer, a Hybrid Inception Recurrent Residual Convolutional Neural Network (HIRResCNN) is introduced, which integrates Harmony Search Optimization (HSO) to tune bias and weight parameters and classification accuracy is enhanced using HIRResCNN-HSO model. Strength of Recurrent Convolutional Neural Network (RCNN), Residual Network (ResNet) and Inception Network (Inception-v4), are combined in a powerful Deep Convolutional Neural Network (DCNN) model called HIRResCNN. using Mammographic Image Analysis Society (MIAS) dataset, various experiments are conduced and results are compared with other available techniques. Around 92.6% accuracy rate is produced using this proposed HIRResCNN classifier in finding breast cancer.

Reduced order modeling for parameterized time-dependent PDEs using spatially and memory aware deep learning

We present a novel reduced order model (ROM) approach for parameterized time-dependent PDEs based on modern learning. The ROM is suitable for multi-query problems and is nonintrusive. It is divided into two distinct stages: a nonlinear dimensionality reduction stage that handles the spatially distributed degrees of freedom based on convolutional autoencoders, and a parameterized time-stepping stage based on memory aware neural networks (NNs), specifically causal convolutional and long short-term memory NNs. Strategies to ensure generalization and stability are discussed. To show the variety of problems the ROM can handle, the methodology is demonstrated on the advection equation, and the flow past a cylinder problem modeled by the incompressible Navier–Stokes equations.

T-Distributed Stochastic Neighbor Embedding to Improve the Discrimination of Yogurt Using a Multistep Amperometry Electronic Tongue

In this work, a classification system of eight different yogurts produced in the Boyacá region of Colombia is developed. The system is based on a sensor array used as an electronic tongue. Following the concept of the IUPAC technical report [1], an “electronic tongue” is an analytical instrument including an array of non-selective chemical sensors with partial specificity to different solution components and an appropriate pattern recognition instrument, capable of recognizing quantitative and qualitative compositions of simple and complex solutions. First, the electronic tongue developed is described with its parts, its characteristics, and operation. Subsequently, the electrochemical techniques used in the developed experiments, that is, multi-step amperometry, are detailed, as well as the configuration parameters. The electronic tongue described in this work is composed of an array of Sensors of the Screen-Printed Electrode (SPE) type from the BVT technologies brand. The selected potentiostat is the Palmsens4 offered by the Palmsens company, in addition to a MUX8R2 multiplexer equipment also from the Palmsens company. This multiplexer allows handling up to 8 sensors. The schematic diagram in Figure 1 illustrates the components of the electronic tongue system. The different signals coming from the 8 sensors in each experiment require a signal processing and pattern recognition methodology for the correct classification of the yogurts. Our team developed a new pattern recognition methodology composed of the following stages. First, the data is arranged to build a two-dimensional matrix following a data unfolding process. Next, the raw data is normalized using the mean-centered group scaling method, in order to consider the different magnitudes of the signals coming from the sensors. Then, a dimensionality reduction and feature extraction stage is performed using the t-distributed stochastic neighbor embedding t-SNE method [2]. The feature matrix serves as input of a supervised machine learning classification algorithm, in this case a k-NN method is used [3]. The total number of experiments yields few samples per yogurt, therefore, in total, an imbalanced dataset of 151 samples of 8 different types of yogurt is used for the validation of the proposed methodology. This validation is executed through a Leave One Out Cross Validation method [4]. A final classification accuracy of 99. 33% was obtained by applying the developed methodology. Only one sample mistake in the 151 samples was obtained.The authors thank FONDO DE CIENCIA TECNOLOGÍA E INNOVACION FCTeI DEL SISTEMA GENERAL DE REGALÍAS SGR. The authors express their gratitude to the Administrative Department of Science, Technology and Innovation—Colciencias with the grant 779—“Convocatoria para la Formación de Capital Humano de Alto Nivel para el Departamento de Boyacá 2017” for sponsoring the research presented herein. Jersson X. Leon-Medina is grateful with Colciencias and Gobernación de Boyacá for the PhD fellowship.

Enhanced K-Nearest Neighbor for Intelligent Fault Diagnosis of Rotating Machinery

Case-based intelligent fault diagnosis methods of rotating machinery can deal with new faults effectively by adding them into the case library. However, case-based methods scarcely refer to automatic feature extraction, and k-nearest neighbor (KNN) commonly required by case-based methods is unable to determine the nearest neighbors for different testing samples adaptively. To solve these problems, a new intelligent fault diagnosis method of rotating machinery is proposed based on enhanced KNN (EKNN), which can take advantage of both parameter-based and case-based methods. First, EKNN is embedded with a dimension-reduction stage, which extracts the discriminative features of samples via sparse filtering (SF). Second, to locate the nearest neighbors for various testing samples adaptively, a case-based reconstruction algorithm is designed to obtain the correlation vectors between training samples and testing samples. Finally, according to the optimized correlation vector of each testing sample, its nearest neighbors can be adaptively selected to obtain its corresponding health condition label. Extensive experiments on vibration signal datasets of bearings are also conducted to verify the effectiveness of the proposed method.

A novel dimension reduction and dictionary learning framework for high-dimensional data classification

High-dimensional problem poses significant challenges for dictionary learning based classification architecture. Joint Dimension Reduction and Dictionary Learning (JDRDL) framework shows great potential for overcoming the challenges caused by high dimensionality. However, most of the existing JDRDL approaches do not consider the complex nonlinear relationships within high-dimensional data, which limits their classification performance. To overcome this problem, a novel joint dimension reduction and dictionary learning framework is proposed in this paper for high-dimensional data classification. Firstly, at dimension reduction stage, an autoencoder is employed to learn a nonlinear mapping that reduces dimensionality and preserves nonlinear structure of the high-dimensional data. Then, at dictionary learning stage, the locality constraint with label embedding, which takes the locality and label information into account together, is incorporated into the learning process to preserve desirable nonlinear local structure and enhance class discrimination. Moreover, the mapping function and dictionary are optimized simultaneously to enhance the performance. Encouraging experimental results on multiple benchmark datasets confirm that the proposed framework is effective and efficient for high-dimensional data classification.

Statistical Methodology for the Definition of Standard Model for Energy Analysis of Residential Buildings in Korea

This study was conducted to propose an optimal methodology for deriving a standard model from existing residential buildings. To strategically improve existing residential buildings, it is necessary to identify standard models that can be used as quantitative standards. In this study, a total of six methods were established for different algorithms in the dimensionality reduction and clustering stage of the data preprocessing stage. In addition, a total of 22,342 households’ data were analyzed, and a total of 26 variables were used to perform cluster analysis. The process of method 6 (data pre-processing, principal components analysis, clustering [K-medoids], verification) was proposed as a way to derive the standard model from the existing Korean housing. The method proposed in this study is capable of deriving a number of standard models considering all variables (n) in a single analysis. The representative building derived in this study contains a lot of building data, so it can be effectively used for planning and research related to buildings on a regional and national scale. In addition, this process can be applied to various buildings to derive representative buildings.

Dimensionality reduction of hyperspectral images of vegetation and crops based on self-organized maps

Hyperspectral images are multidimensional massive sets of information that have shown a great potential for different kind of applications as urban mapping, environmental management, vegetation and crops supervision and mineral detection. However, due to its high dimensional nature and the high variability of the spectral information, the dimensionality reduction process is one of the main challenges in processing hyperspectral images. The aim of dimensionality reduction is to eliminate redundant information and simplify the subsequent processes of classification and the search of information. In this context, several dimensionality reduction methods have been proposed, but most of them are not flexible enough to deal with the particular features of the hyperspectral images. In this way, the use of intelligent methods as neural networks and specially an unsupervised approach as self-organized maps, may improve the dimensionality reduction stage and the final classification process. This paper proposes an unsupervised method for the dimensionality reduction of hyperspectral images based on Kohonen self-organized maps, which, compared with other traditional methods such as principal component analysis (PCA) and wavelet decomposition, provides better classification results. The results provided in this paper use an RBF (radial basis function) classifier. On average, the proposed method provides a 64% dimensionality reduction and an 88.5% classification accuracy. These results suggest that the dimensionality reduction algorithm based on self-organized maps is an efficient approach compared with other popular algorithms. This is due to the ability of self-organized maps to automatically detect (self-organizing) relationships within the set of input patterns, which provides flexibility to deal with the special features of the hyperspectral images.

Human activity recognition based on smartphone using fast feature dimensionality reduction technique

Human activity recognition aims to identify the activities carried out by a person. Recognition is possible by using information that is retrieved from numerous physiological signals by attaching sensors to the subject’s body. Lately, sensors like accelerometer and gyroscope are built-in inside the Smartphone itself, which makes activity recognition very simple. To make the activity recognition system work properly in smartphone which has power constraint, it is very essential to use an optimization technique which can reduce the number of features used in the dataset with less time consumption. In this paper, we have proposed a dimensionality reduction technique called fast feature dimensionality reduction technique (FFDRT). A dataset (UCI HAR repository) available in the public domain is used in this work. Results from this study shows that the fast feature dimensionality reduction technique applied for the dataset has reduced the number of features from 561 to 66, while maintaining the activity recognition accuracy at 98.72% using random forest classifier and time consumption in dimensionality reduction stage using FFDRT is much below the state of the art techniques.

Deep machine learning for structural health monitoring on ship hulls using acoustic emission method

ABSTRACT Corrosion, fatigue and corrosion-fatigue cracking are the most pervasive types of structural problems experienced by ship structures. These damage modes, can potentially lead to unanticipated out of service time or catastrophic failure. Acoustic Emission is gaining ground as a complementary Structural Health Monitoring (SHM) technique, since it can offer real-time damage detection. Deep learning, on the other hand, has shown great success over the last years for a large number of applications. In this paper, the SHM on ship hulls is treated as a classification problem. Firstly, the AE signals are transformed, using the Discrete Cosine Transform, followed by a dimensionality reduction stage. Afterwards, a Deep Neural Network is employed by the classification module. The proposed approach was validated and the results indicate that our proposed method can be very effective and efficient, selecting the optimum AE sensor positions and providing almost perfect localisation results.

Novel Methodology for Condition Monitoring of Gear Wear Using Supervised Learning and Infrared Thermography

In gearboxes, the occurrence of unexpected failures such as wear in the gears may occur, causing unwanted downtime with significant financial losses and human efforts. Nowadays, noninvasive sensing represents a suitable tool for carrying out the condition monitoring and fault assessment of industrial equipment in continuous operating conditions. Infrared thermography has the characteristic of being installed outside the machinery or the industrial process under assessment. Also, the amount of information that sensors can provide has become a challenge for data processing. Additionally, with the development of condition monitoring strategies based on supervised learning and artificial intelligence, the processing of signals with significant improvements during the classification of information has been facilitated. Thus, this paper proposes a novel noninvasive methodology for the diagnosis and classification of different levels of uniform wear in gears through thermal analysis with infrared imaging. The novelty of the proposed method includes the calculation of statistical time-domain features from infrared imaging, the consideration of a dimensionality reduction stage by means of Linear Discriminant Analysis, and automatic fault diagnosis performed by an artificial neural network. The proposed method is evaluated under an experimental laboratory data set, which is composed of the following conditions: healthy, and three severity degrees of uniform wear in gears, namely, 25%, 50%, and 75% of uniform wear. Finally, the obtained results are compared with classical condition monitoring approaches based on vibration analysis.

User-Guided Clustering for Video Segmentation on Coarse-Grained Feature Extraction

Video segmentation is the task of temporally dividing a video into semantic sections, which are typically based on a specific concept or a theme that is usually defined by the user’s intention. However, previous studies of video segmentation have that far not taken a user’s intention into consideration. In this paper, a two-stage user-guided video segmentation framework has been presented, including dimension reduction and temporal clustering. During the dimension reduction stage, a coarse granularity feature extraction is conducted by a deep convolutional neural network pre-trained on ImageNet. In the temporal clustering stage, the information of the user’s intention is utilized to segment videos on time domain with a proposed operator, which calculates the similarity distance between dimension reduced frames. To provide more insight into the videos, a hierarchical clustering method that allows users to segment videos at different granularities is proposed. Evaluation on Open Video Scene Detection(OVSD) dataset shows that the average F-score achieved by the proposed method is 0.72, even coarse-grained feature extraction is adopted. The evaluation also demonstrated that the proposed method can not only produce different segmentation results according to the user’s intention, but it also produces hierarchical segmentation results from a low level to a higher abstraction level.

Multiple Feature Kernel Sparse Representation Classifier for Hyperspectral Imagery

Multiple types of features, e.g., spectral, filtering, texture, and shape features, are helpful for hyperspectral image (HSI) classification tasks. Combining multiple features can describe the characteristics of pixels from different perspectives, and always results in better classification performance. Recently, multifeature combination learning has been widely employed to the multitask-learning-based representation-based model to obtain a multifeature representation vector. However, the linear sparse representation-based classifier (SRC) cannot handle the HSI with highly nonlinear distribution, and kernel sparse representation-based classifier (KSRC) can remedy the drawback of linear SRC. By adopting nonlinear mapping, the samples in kernel space are often of high or even infinite dimensionality. In this paper, we integrate kernel principal component analysis into multifeature-based KSRC and propose a novel multiple feature kernel sparse representation-based classifier (namely, MFKSRC) for hyperspectral imagery. More specifically, spatial features, Gabor textures, local binary patterns, and difference morphological profiles are adopted and then each kind of feature is transformed nonlinearly into a new low-dimensional kernel space. The proposed framework can handle data with nonlinear distribution and add a dimensionality reduction stage in kernel space before optimizing the corresponding cost function. Experimental results on different HSIs demonstrate that the proposed MFKSRC algorithm outperforms the state-of-the-art classifiers.

Structured Ordinary Least Squares: A Sufficient Dimension Reduction approach for regressions with partitioned predictors and heterogeneous units.

In many scientific and engineering fields, advanced experimental and computing technologies are producing data that are not just high dimensional, but also internally structured. For instance, statistical units may have heterogeneous origins from distinct studies or subpopulations, and features may be naturally partitioned based on experimental platforms generating them, or on information available about their roles in a given phenomenon. In a regression analysis, exploiting this known structure in the predictor dimension reduction stage that precedes modeling can be an effective way to integrate diverse data. To pursue this, we propose a novel Sufficient Dimension Reduction (SDR) approach that we call structured Ordinary Least Squares (sOLS). This combines ideas from existing SDR literature to merge reductions performed within groups of samples and/or predictors. In particular, it leads to a version of OLS for grouped predictors that requires far less computation than recently proposed groupwise SDR procedures, and provides an informal yet effective variable selection tool in these settings. We demonstrate the performance of sOLS by simulation and present a first application to genomic data. The R package "sSDR," publicly available on CRAN, includes all procedures necessary to implement the sOLS approach.