Principal Component Analysis Learning Algorithms Research Articles

Identification of Target Chicken Populations by Machine Learning Models Using the Minimum Number of SNPs.

Simple SummaryClassifying a target population at the genetic level can provide important information for the preservation and commercial use of a breed. In this study, the minimum number of markers was used in combination, to distinguish target populations based on high-density single nucleotide polymorphism (SNP) array data. Subsequently, a genome-wide association study for filtering target-population-specific SNPs between the case and control groups and principal component analysis with machine learning algorithms could be used to explore various combinations with the minimum number of markers. In addition, the optimal combination of SNP markers was able to produce stable results for the target population in verification studies, in which samples were analyzed.A marker combination capable of classifying a specific chicken population could improve commercial value by increasing consumer confidence with respect to the origin of the population. This would facilitate the protection of native genetic resources in the market of each country. In this study, a total of 283 samples from 20 lines, which consisted of Korean native chickens, commercial native chickens, and commercial broilers with a layer population, were analyzed to determine the optimal marker combination comprising the minimum number of markers, using a 600 k high-density single nucleotide polymorphism (SNP) array. Machine learning algorithms, a genome-wide association study (GWAS), linkage disequilibrium (LD) analysis, and principal component analysis (PCA) were used to distinguish a target (case) group for comparison with control chicken groups. In the processing of marker selection, a total of 47,303 SNPs were used for classifying chicken populations; 96 LD-pruned SNPs (50 SNPs per LD block) served as the best marker combination for target chicken classification. Moreover, 36, 44, and 8 SNPs were selected as the minimum numbers of markers by the AdaBoost (AB), Random Forest (RF), and Decision Tree (DT) machine learning classification models, which had accuracy rates of 99.6%, 98.0%, and 97.9%, respectively. The selected marker combinations increased the genetic distance and fixation index (Fst) values between the case and control groups, and they reduced the number of genetic components required, confirming that efficient classification of the groups was possible by using a small number of marker sets. In a verification study including additional chicken breeds and samples (12 lines and 182 samples), the accuracy did not significantly change, and the target chicken group could be clearly distinguished from the other populations. The GWAS, PCA, and machine learning algorithms used in this study can be applied efficiently, to determine the optimal marker combination with the minimum number of markers that can distinguish the target population among a large number of SNP markers.

Evaluation of rice varieties using LIBS and FTIR techniques associated with PCA and machine learning algorithms.

Laser-induced breakdown spectroscopy (LIBS) for atomic multi-elementary analyses, and Fourier transform infrared spectroscopy (FTIR) for molecular identification, are often suggested as the most versatile spectroscopic techniques. The present work aimed to evaluate the performance of both techniques, LIBS and FTIR, combined with principal component analysis (PCA) and machine learning (ML) algorithms in the detection of the composition analysis and differentiation of four different types of rice, white, brown, black, and red. The two techniques were primarily used to obtain the elemental and molecular qualitative characterization of rice samples. Then, LIBS and FTIR data sets were subjected to PCA and supervised ML analysis to investigate which main chemical features were responsible for nutritional differences for the white (milled) and colored rice samples. In particular, PCA data analysis suggested that protein, fatty acids, and magnesium were the highest contributors to the sample's differentiation. The ML analysis based on this information yielded a 100% level of accuracy, sensitivity, and specificity on sample classification. In conclusion, LIBS and FTIR coupled with multivariate analysis were confirmed as promising tools alternative to traditional analytical techniques for composition analysis and differentiation when subtle chemical variations were observed.

Nanowire-Based Sensor Array for Detection of Cross-Sensitive Gases Using PCA and Machine Learning Algorithms

In this work, a gas sensor array has been designed and developed comprising of Pt, Cu and Ag decorated TiO2 and ZnO functionalized GaN nanowires using industry standard top-down fabrication approach. The receptor metal/metal-oxide combinations within the array have been determined from our prior molecular simulation results using first principle calculations based on density functional theory (DFT). The gas sensing data was collected for both singular and mixture of NO2, ethanol, SO2 and H2 in presence of H2O and O2 gases under UV light at room temperature. Each gas produced a unique response pattern across the sensors within the array by which precise identification of cross-sensitive gases is possible. After pre-processing of raw data, unsupervised principal component analysis (PCA) technique was applied on the array response. It is found that, each analyte gas forms a separate cluster in the score plot for all the target gases and their mixtures, indicating a clear discrimination among them. Then, four supervised machine learning algorithms such as- Decision Tree, Support Vector Machine (SVM), Naive Bayes (kernel) and k-Nearest Neighbor (k-NN) were trained and optimized using their significant parameters with our array dataset for the classification of gas type. Results indicate that the optimized SVM and NB classifier models exhibited 100% classification accuracy on test dataset. Practical applicability of the considered algorithms has been discussed as well. Moreover, this array device works at room-temperature using very low power and low-cost UV light-emitting diode (LED) as compared to high power consuming commercially available metal-oxide sensors.

Dynamics of Generalized PCA and MCA Learning Algorithms

Principal component analysis (PCA) and minor component analysis (MCA) are two important statistical tools which have many applications in the fields of signal processing and data analysis. PCA and MCA neural networks (NNs) can be used to online extract principal component and minor component from input data. It is interesting to develop generalized learning algorithms of PCA and MCA NNs. Some novel generalized PCA and MCA learning algorithms are proposed in this paper. Convergence of PCA and MCA learning algorithms is an essential issue in practical applications. Traditionally, the convergence is studied via deterministic continuous-time (DCT) method. The DCT method requires the learning rate of the algorithms to approach to zero, which is not realistic in many practical applications. In this paper, deterministic discrete-time (DDT) method is used to study the dynamical behaviors of the proposed algorithms. The DDT method is more reasonable for the convergence analysis since it does not require constraints as that of the DCT method. It is proven that under some mild conditions, the weight vector in these proposed algorithms will converge exponentially to principal or minor component. Simulation results are further used to illustrate the theoretical results.

A ROBUST AND GLOBALLY CONVERGENT PCA LEARNING ALGORITHM

Principal component analysis (PCA) using neural networks is an active research field with many applications to signal processing and data analysis. This paper presents a PCA neural network endowed with a novel learning algorithm, and an analysis of its features. As the basic discrete-time Oja's PCA neural network does not converge globally, it is important to derive a robust and globally convergent PCA learning algorithm. Based on previous works on the globally convergent PCA learning algorithm, a robust and globally convergent PCA learning algorithm is proposed in this paper. The behavior of this discrete-time learning algorithm is directly studied in this paper. We show that the algorithm is robust and globally convergent. The selection of the parameters of this algorithm will also be discussed in details. Finally, simulation results are provided to verify the theoretical results presented. Compared to other PCA learning algorithms, the proposed algorithm performs favorably in terms of robust stability, global convergence, speed and accuracy.

Global convergence of Oja's PCA learning algorithm with a non-zero-approaching adaptive learning rate

A non-zero-approaching adaptive learning rate is proposed to guarantee the global convergence of Oja's principal component analysis (PCA) learning algorithm. Most of the existing adaptive learning rates for Oja's PCA learning algorithm are required to approach zero as the learning step increases. However, this is not practical in many applications due to the computational round-off limitations and tracking requirements. The proposed adaptive learning rate overcomes this shortcoming. The learning rate converges to a positive constant, thus it increases the evolution rate as the learning step increases. This is different from learning rates which approach zero which slow the convergence considerably and increasingly with time. Rigorous mathematical proofs for global convergence of Oja's algorithm with the proposed learning rate are given in detail via studying the convergence of an equivalent deterministic discrete time (DDT) system. Extensive simulations are carried out to illustrate and verify the theory derived. Simulation results show that this adaptive learning rate is more suitable for Oja's PCA algorithm to be used in an online learning situation.

Convergence Analysis of a Deterministic Discrete Time System of Oja's PCA Learning Algorithm

The convergence of Oja's principal component analysis (PCA) learning algorithms is a difficult topic for direct study and analysis. Traditionally, the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time (DCT) systems. Such a method will require the learning rate to converge to zero, which is not a reasonable requirement to impose in many practical applications. Recently, deterministic discrete time (DDT) systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms. Unlike DCT systems, DDT systems allow learning rates to be constant (which can be a nonzero). This paper will provide some important results relating to the convergence of a DDT system of Oja's PCA learning algorithm. It has the following contributions: 1) A number of invariant sets are obtained, based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever. Thus, the nondivergence of the trajectories is guaranteed. 2) The convergence of the DDT system is analyzed rigorously. It is proven, in the paper, that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix. In addition, exponential convergence rate are obtained, providing useful guidelines for the selection of fast convergence learning rate. 3) Since the trajectories may diverge, the careful choice of initial vectors is an important issue. This paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence. 4) Simulation results will be furnished to illustrate the theoretical results achieved.

A globally convergent learning algorithm for PCA neural networks

Principal component analysis (PCA) by neural networks is one of the most frequently used feature extracting methods. To process huge data sets, many learning algorithms based on neural networks for PCA have been proposed. However, traditional algorithms are not globally convergent. In this paper, a new PCA learning algorithm based on cascade recursive least square (CRLS) neural network is proposed. This algorithm can guarantee the network weight vector converges to an eigenvector associated with the largest eigenvalue of the input covariance matrix globally. A rigorous mathematical proof is given. Simulation results show the effectiveness of the algorithm.

TIME-ORIENTED HIERARCHICAL METHOD FOR COMPUTATION OF PRINCIPAL COMPONENTS USING SUBSPACE LEARNING ALGORITHM

Principal Component Analysis (PCA) and Principal Subspace Analysis (PSA) are classic techniques in statistical data analysis, feature extraction and data compression. Given a set of multivariate measurements, PCA and PSA provide a smaller set of "basis vectors" with less redundancy, and a subspace spanned by them, respectively. Artificial neurons and neural networks have been shown to perform PSA and PCA when gradient ascent (descent) learning rules are used, which is related to the constrained maximization (minimization) of statistical objective functions. Due to their low complexity, such algorithms and their implementation in neural networks are potentially useful in cases of tracking slow changes of correlations in the input data or in updating eigenvectors with new samples. In this paper we propose PCA learning algorithm that is fully homogeneous with respect to neurons. The algorithm is obtained by modification of one of the most famous PSA learning algorithms--Subspace Learning Algorithm (SLA). Modification of the algorithm is based on Time-Oriented Hierarchical Method (TOHM). The method uses two distinct time scales. On a faster time scale PSA algorithm is responsible for the "behavior" of all output neurons. On a slower scale, output neurons will compete for fulfillment of their "own interests". On this scale, basis vectors in the principal subspace are rotated toward the principal eigenvectors. At the end of the paper it will be briefly analyzed how (or why) time-oriented hierarchical method can be used for transformation of any of the existing neural network PSA method, into PCA method.

A class of learning algorithms for principal component analysis and minor component analysis

Principal component analysis (PCA) and minor component analysis (MCA) are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing. In this paper, we first propose a differential equation for the generalized eigenvalue problem.We prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue. Based on this generalized differential equation, a class of PCA and MCA learning algorithms can be obtained. We demonstrate that many existing PCA and MCA learning algorithms are special cases of this class, and this class includes some new and simpler MCA learning algorithms. Our results show that all the learning algorithms of this class have the same order of convergence speed, and they are robust to implementation error.

A class of learning algorithms for principal component analysis and minor component analysis

Principal component analysis (PCA) and minor component analysis (MCA) are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing. In this paper, we first propose a differential equation for the generalized eigenvalue problem.We prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue. Based on this generalized differential equation, a class of PCA and MCA learning algorithms can be obtained. We demonstrate that many existing PCA and MCA learning algorithms are special cases of this class, and this class includes some new and simpler MCA learning algorithms. Our results show that all the learning algorithms of this class have the same order of convergence speed, and they are robust to implementation error.

A general theory of a class of linear neural nets for principal and minor component analysis

This paper presents a unified theory of a class of learning neural nets for principal component analysis (PCA) and minor component analysis (MCA). First, some fundamental properties are addressed which all neural nets in the class have in common. Second, a subclass called the generalized asymmetric learning algorithm is investigated, and the kind of asymmetric structure which is required in general to obtain the individual eigenvectors of the correlation matrix of a data sequence is clarified. Third, focusing on a single-neuron model, a systematic way of deriving both PCA and MCA learning algorithms is shown, through which a relation between the normalization in PCA algorithms and that in MCA algorithms is revealed.

Handwritten Digit Recognition by a Mixture of Local PrincipalComponent Analysis

Mixture of local principal component analysis (PCA) has attracted attention due to a number of benefits over global PCA. The performance of a mixture model usually depends on the data partition and local linear fitting. In this paper, we propose a mixture model which has the properties of optimal data partition and robust local fitting. Data partition is realized by a soft competition algorithm called neural ’gas‘ and robust local linear fitting is approached by a nonlinear extension of PCA learning algorithm. Based on this mixture model, we describe a modular classification scheme for handwritten digit recognition, in which each module or network models the manifold of one of ten digit classes. Experiments demonstrate a very high recognition rate.

An adaptive learning algorithm for principal component analysis

Principal component analysis (PCA) is one of the most general purpose feature extraction methods. A variety of learning algorithms for PCA has been proposed. Many conventional algorithms, however, will either diverge or converge very slowly if learning rate parameters are not properly chosen. In this paper, an adaptive learning algorithm (ALA) for PCA is proposed. By adaptively selecting the learning rate parameters, we show that the m weight vectors in the ALA converge to the first m principle component vectors with almost the same rates. Comparing with the Sanger's generalized Hebbian algorithm (GHA), the ALA can quickly find the desired principal component vectors while the GHA fails to do so. Finally, simulation results are also included to illustrate the effectiveness of the ALA.

Principal component analysis learning algorithms: a neurobiological analysis

The biological relevance of principal component analysis (PCA) learning algorithms is addressed by: (i) describing a plausible biological mechanism which accounts for the changes in synaptic efficacy implicit in Oja's 'Subspace' algorithm (Int. J. neural Syst. 1, 61 (1989)); and (ii) establishing a potential role for PCA-like mechanisms in the development of functional segregation. PCA learning algorithms comprise an associative Hebbian term and a decay term which interact to find the principal patterns of correlations in the inputs shared by a group of units. We propose that the presynaptic component of this decay could be regulated by retrograde signals that are translocated from the terminal arbors of presynaptic neurons to their cell bodies. This proposal is based on reported studies of structural plasticity in the nervous system. By using simulations we demonstrate that PCA-like mechanisms can eliminate afferent connections whose signals are unrelated to the prevalent pattern of afferent activity. This elimination may be instrumental in refining extrinsic cortico-cortical connections that underlie functional segregation.