Dimensionality Reduction Process Research Articles

A Mathematical Interpretation of Autoregressive Generative Pre-Trained Transformer and Self-Supervised Learning

In this paper, we present a rigorous mathematical examination of generative pre-trained transformer (GPT) models and their autoregressive self-supervised learning mechanisms. We begin by defining natural language space and knowledge space, which are two key concepts for understanding the dimensionality reduction process in GPT-based large language models (LLMs). By exploring projection functions and their inverses, we establish a framework for analyzing the language generation capabilities of these models. We then investigate the GPT representation space, examining its implications for the models’ approximation properties. Finally, we discuss the limitations and challenges of GPT models and their learning mechanisms, considering trade-offs between complexity and generalization, as well as the implications of incomplete inverse projection functions. Our findings demonstrate that GPT models possess the capability to encode knowledge into low-dimensional vectors through their autoregressive self-supervised learning mechanism. This comprehensive analysis provides a solid mathematical foundation for future advancements in GPT-based LLMs, promising advancements in natural language processing tasks such as language translation, text summarization, and question answering due to improved understanding and optimization of model training and performance.

TopoDimRed: a novel dimension reduction technique for topological data analysis

Topological data analysis (TDA) has emerged as a powerful approach for analyzing complex datasets, capturing the underlying shape and structure inherent in the data. However, TDA often encounters challenges when dealing with high-dimensional data due to the curse of dimensionality. To address this issue, we propose a novel dimension reduction technique called TopoDimRed that integrates topological analysis with advanced dimension reduction algorithms. TopoDimRed aims to reduce the dimensionality of topological data while preserving important topological features, enabling efficient visualization and analysis. In this paper, we present the methodology of TopoDimRed, highlighting its ability to capture and preserve relevant topological structures during the dimension reduction process. We conduct extensive experimental evaluations on diverse datasets from different domains, comparing TopoDimRed with traditional dimension reduction techniques. The results demonstrate that TopoDimRed outperforms or achieves comparable performance in terms of preserving topological features, visualization quality, and computational efficiency. Furthermore, we showcase the application of TopoDimRed in various domains, including biological networks, social networks, materials science, and neuroscience, illustrating its utility in gaining insights from high-dimensional topological data. We discuss the strengths and limitations of TopoDimRed and propose potential future directions for its development and application. Overall, TopoDimRed offers a valuable tool for researchers and practitioners to explore, visualize, and analyze high-dimensional topological data, facilitating the discovery of hidden structures and meaningful insights in complex datasets.

Two-scale analysis of spaceframes with complex additive manufactured nodes

The advancements in additive manufacturing (AM) technology have allowed for the production of geometrically complex parts with customizable designs. This versatility benefits large-scale space-frame structures, as the individual design of each structural node can be tailored to meet specific mechanical and other functional requirements. To this end, however, the design and analysis of such space-frames with distinct structural nodes needs to be highly automated. A critical aspect in this context is automated integration of the local 3D features into the 1D large-scale models. In the present work, a two-scale modeling approach is developed to improve the design and linear-elastic analysis of space frames with complex additively manufactured nodes. The mechanical characteristics of the 3D nodes are numerically reduced through an automated dimensional reduction process based on the Finite Cell Method (FCM) and substructuring. The reduced stiffness quantities are assembled in the large-scale 1D model which, in turn, enables efficient structural analysis. The response of the 1D model is passed on to the local model, enabling fully resolved 3D linear-elastic analysis. The proposed approach is numerically verified on a simplified beam example. Furthermore, the workflow is demonstrated on a tree canopy structure with additively manufactured nodes with bolted connections. The form of the large-scale structure is found based on the Combinatorial Equilibrium Modeling framework, and the different designs of the local structural nodes are based on generative exploration of the design space. It is demonstrated that the proposed methodology effectively automates the design and analysis of space-frame structures with complex, distinct structural nodes.

Wheeled Robot Visual Odometer Based on Two-dimensional Iterative Closest Point Algorithm

According to the two-dimensional motion characteristics of planar motion wheeled robot, the visual odometer was dimensionally reduced in this study. In the feature point matching part of visual odometer, the contour constraint was used to filter out the mismatched feature point pairs (abbreviated as FPP). This method could also filter out the matched FPP, and the feature of FPP was correct color image matches, however, their depth image error was large. This offered higher quality matched FPP for the subsequent interframe motion estimation. Dimension reduction was performed in the interframe motion estimation part, and the two-dimensional Iterative Closest Point (ICP) algorithm was used for camera motion estimation. The experiments indicated that the proposed algorithm effectively improved the computational speed and precision of planar motion wheeled robot visual odometer. This research indicates that the dimension reduction processing of ICP algorithm can effectively improve the operation speed and calculation accuracy of planar motion wheeled robot visual odometry, which provides a good reference and data support for the subsequent research of wheeled robot visual odometry in the future.

Neural Network-Based Multiplicatively Gait Feature Eradication and Detection

This research proposes an innovative method based on machine learnings for extracting and identifying gait features from multiple sources. The method aims to enhance the accuracy of gait identification by minimizing interferences caused by complex backgrounds and shelters, thereby capturing more precise information that reflects the walking characteristics of moving individuals. The technical approach involves the acquisition of gait data using a video recorder and a pyroelectric IR sensor. The image source information obtained from the video recorder is utilized to extract skeleton feature variables and Radon difference peak characteristic variables. In addition, the pyroelectric IR source information is transformed from a voltage signal to frequency domain characteristic variables. These variables are then merged after undergoing dimension reduction and signal processing. Finally, a backpropagation neural network is employed as the classifier to perform classified identification based on the merged characteristics, and the identification accuracy is evaluated. The primary application of this method is in the field of identification.

Structural optimization of multistage centrifugal pump via computational fluid dynamics and machine learning method

Abstract To implement energy savings in multistage centrifugal pumps, a return channel is utilized to replace the origin inter-stage flow channel structure, and then a single-objective optimization work containing high-precision numerical simulation, design variable dimensionality reduction, and machine learning is conducted to obtain the optimal geometric parameters. The variable dimensionality reduction process is based on the Spearman correlation analysis method. The influence of 15 design variables of the impeller and return channel is investigated, and seven of them with high-impact factors are selected as the final optimization variables. Thereafter, a genetic algorithm-backpropagation neural network (GA-BPNN) model is used to create a surrogate model with a high-fitting performance by employing a GA to optimize the initial thresholds and weights of a BPNN. Finally, a multi-island genetic algorithm (MIGA) is employed to maximize hydraulic efficiency under the nominal condition. The findings demonstrate that the optimized model’s efficiency is increased by 4.29% at 1.0Qd, and the deterioration of the pump performance under overload conditions is effectively eliminated (the maximum efficiency increase is 14.72% at 1.3Qd). Furthermore, the internal flow analysis indicates that the optimization scheme can improve the turbulence kinetic energy distribution and reduce unstable flow structures in the multistage centrifugal pump.

Differential Privacy High-Dimensional Data Publishing Based on Feature Selection and Clustering

As a social information product, the privacy and usability of high-dimensional data are the core issues in the field of privacy protection. Feature selection is a commonly used dimensionality reduction processing technique for high-dimensional data. Some feature selection methods only process some of the features selected by the algorithm and do not take into account the information associated with the selected features, resulting in the usability of the final experimental results not being high. This paper proposes a hybrid method based on feature selection and a cluster analysis to solve the data utility and privacy problems of high-dimensional data in the actual publishing process. The proposed method is divided into three stages: (1) screening features; (2) analyzing the clustering of features; and (3) adaptive noise. This paper uses the Wisconsin Breast Cancer Diagnostic (WDBC) database from UCI’s Machine Learning Library. Using classification accuracy to evaluate the performance of the proposed method, the experiments show that the original data are processed by the algorithm in this paper while protecting the sensitive data information while retaining the contribution of the data to the diagnostic results.

Quantitative Two-Stage Classification of Gas Mixtures Using 2-D TMDC and PGM Chalcogenides

Accurate and quantitative classification of gas mixtures is an important issue in various fields, including the healthcare and food industries. However, traditional classification approaches, such as gas chromatography, mass spectroscopy, and chemical analysis, not only require specialized skills but are also time-consuming, inaccurate, and expensive. For these reasons, we used a chemiresistive sensor based on 2-D transition metal dichalcogenides (2-D TMDCs) and platinum group material-based chalcogenides, which have high responsivity, selectivity, and stability toward target gases. Raman spectroscopy, scanning electron microscopy, and X-ray photoelectron spectroscopy were used to characterize the WS2 and RuS2 sensing channels. Moreover, the gas-sensing properties toward NO2, NH3, and their mixtures (1:1 and 2:1) were analyzed, and the classification of these gases was carried out via our proposed two-stage classification model consisting of dimensionality reduction and classification processes. The proposed model achieved more than 90% accuracy in all cases when classifying single gases and their mixtures, which could be industrially applicable in the future.

Quantum-PSO based unsupervised clustering of users in social networks using attributes.

Unsupervised cluster detection in social network analysis involves grouping social actors into distinct groups, each distinct from the others. Users in the clusters are semantically very similar to those in the same cluster and dissimilar to those in different clusters. Social network clustering reveals a wide range of useful information about users and has many applications in daily life. Various approaches are developed to find social network users' clusters, using only links or attributes and links. This work proposes a method for detecting social network users' clusters based solely on their attributes. In this case, users' attributes are considered categorical values. The most popular clustering algorithm used for categorical data is the K-mode algorithm. However, it may suffer from local optimum due to its random initialization of centroids. To overcome this issue, this manuscript proposes a methodology named the Quantum PSO approach based on user similarity maximization. In the proposed approach, firstly, dimensionality reduction is conducted by performing the relevant attribute set selection followed by redundant attribute removal. Secondly, the QPSO technique is used to maximize the similarity score between users to get clusters. Three different similarity measures are used separately to perform the dimensionality reduction and similarity maximization processes. Experiments are conducted on two popular social network datasets; ego-Twitter, and ego-Facebook. The results show that the proposed approach performs better clustering results in terms of three different performance metrics than K-Mode and K-Mean algorithms.

A communication-efficient and privacy-aware distributed algorithm for sparse PCA

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

Dimensionality Reduction and Visualization in Public Management Research from the Perspective of Algorithm Recommendation

With the development of computer technology, algorithm recommendation systems have been widely used in intelligent transportation, urban planning, and other fields. Aiming at the current problems in public management in China, this paper proposes a dimensionality reduction calculation method, and uses the improved particle swarm optimization algorithm to analyze and evaluate the dimensionality reduction process. With the expectation of providing some reference value and ideas for solving practical application problems, a corresponding relationship equation is established with the minimum expected value as the objective function to solve the corresponding optimal solution, the evaluation results of system performance indicators and comprehensive scoring criteria under relevant parameters and constraints are given. The test results show that the visualization effect of the dimensionality reduction visualization model for public management research based on the algorithm recommendation perspective is over 90%.

Linear feature projective geometric damped convolutional deep belief network for indoor floor planning

Wireless localization or positioning is essential for delivering location-based services for designing location tracking systems. Traditional indoor floor planning system employs wireless signals for accurate position estimation. But these positioning schemes failed to perform position estimation effectively and accurately through many obstacles or objects. The novel technique called Linear Features Projective Geometric Damped Convolutional Deep Belief Network (LFPGDCDBN) is introduced to improve the position estimation accuracy with minimum error. The proposed LFPGDCDBN technique includes two major processes namely dimensionality reduction and position estimation. First, the dimensionality reduction process is performed by projecting the principle features using Linear Helmert–Wolf blocked Sammon projection. After the feature selection, Geometric Levenberg–Marquardt Convolutional deep belief network is employed to estimate the position of the devices with higher accuracy and minimum error. The Convolutional deep belief network uses the triangulation geometric method to identify the position of the device in an indoor positioning system. Then the Levenberg–Marquardt function is a Damped least square method to minimize the squares of the deviations between the expected and observed results at the output unit. As a result, the LFPGDCDBN increases the positioning accuracy and minimizes the error rate. Experimental MATLAB assessment is carried out with various factors such as computational time, Computational space, positioning accuracy, and positing error. The experimental results and discussion indicate that the proposed LFPGDCDBN provides improved performance in terms of achieving higher positioning accuracy and minimum error as well as computational time when compared to the existing methods. The experimental results and discussion indicate that the proposed LFPGDCDBN increases the positioning accuracy by 47% and computational time, computational complexity, and reduces the positioning error by 45%, 29%, and 74% as compared to state-of-the-art works.

Casualty on the Titanic based on Machine Learning Methods

The Titanic sank on April 15, 1914, with 2224 people on board, and only 32% survived. The survivors are somewhat random, but they are somewhat the same. Studying the types of people who are more likely to survive in a disaster will promote an understanding of the values and ideology of the society at the time. Therefore, this study collected data on some of the passengers on the Titanic and the survivors through Kaggle. This study predicts the survival of passengers through data cleaning, feature engineering, dimensionality reduction processing, and different models such as Random Forest, Decision Tree, KNN, and Logistic regression. In this study, it was found that Age, gender, representation of socioeconomic status, and whether they traveled alone were associated with survival rates. Among them, the decision tree and random forest models scored 98.65, which performed the best and had the highest prediction accuracy. After visualizing the data, this study concluded that survival rates were higher for females, children younger than ten years old first-class passengers, and non-solitary travelers.

Prediction of return on equity of the energy industry based on equity characteristics

We take the return on equity of energy enterprises as the research object to predict it. Our research adopts a new framework to solve multivariable time series problems. Compared to a single regression model, this model focuses more on the results of the regression equation rather than the coefficients of each indicator. Compared to the single machine learning regression method, this model can use the two-way encoder representation of the Transformers model to embed text data into the data, and then use the XGBoost model for regression model processing after PCA dimensionality reduction processing, thereby improving the accuracy of model prediction. Comparative experiments have verified that the method we use has advantages in terms of prediction accuracy.

Hyperspectral video target tracking based on pixel-wise spectral matching reduction and deep spectral cascading texture features

Illumination variation (IV) is a challenging issue in hyperspectral video target tracking (HVT). In order to resolve this issue, this paper proposes a novel HVT algorithm based on Pixel-wise Spectral Matching Reduction (PSMR) and Deep Spectral Cascading Texture (Deep-SCT) features. The PSMR is a novel dimensionality reduction method that approximately segments the target and background while compressing the hyperspectral image data. The Deep-SCT features are composed of spectral cascading texture (SCT) features and deep features. The local binary pattern operator and illumination invariance along with the Deep-SCT features overcome the interference caused by IV. In addition, we propose a feature fusion method, called group-pixel joint convolution, which fuses SCT features and deep features. Moreover, the segmentation results, produced by the dimensionality reduction process, are used to generate a coarse location mask. This mask is then used to predict the location of target and also to suppress the texture features of the background. Finally, after localizing the target using the Deep-SCT features, the proposed algorithm uses a step-by-step estimation strategy to adjust the size of the target bounding box. Experiments, over benchmark datasets, illustrate the superior performance of the proposed tracker in comparison with the state-of-the-art approaches. It is evident that the proposed strategy is highly robust against IV.

Application of Multidimensional Structural Equations in the Emergency Management of Coal Mine Accidents

The use of coal as a source of energy is crucial for the growth of the national economy, but mining poses numerous risks and a potential for significant disasters. Coal mine safety is the prerequisite and guarantee for coal industry to achieve new industrialization and sustainable development. Therefore, it is crucial to predict a safety accident in the coal mine in advance. In order to facilitate the early warning of coal mine safety accidents, this study seeks to present a prediction model based on emergency management of safety accidents, which is a fusion model of principal component analysis (PCA) and long short-term memory neural network. According to the results, the correlation coefficients of risk identification and monitoring (a11), safety inspection and warning (a12), emergency planning and training (a13), material and technical support (a15), and macroenvironmental management (a21) were 0.718, 0.653, 0.628, 0.444, and 0.553, respectively, after the PCA dimensionality reduction process, demonstrating that the previous principal component analysis had a better effect. The absolute relative errors of each evaluation index of safety accident emergency management did not exceed the limit of 5%, including a15 and a21, whose values were 4.5% and −3.8%, while the relative errors of the remaining indicators were kept at a relatively low level. In conclusion, it is clear that the algorithm model suggested in this research improved the warning capabilities of safety accident emergency risk.

Improved Locality Preserving Projections Based on Heat-Kernel and Cosine Weights for Fault Classification in Complex Industrial Processes

Data-driven fault diagnosis techniques have been widely used in industrial processes. However, facing a large amount of high-dimensional, nonlinear, and strongly coupled process data, traditional data-driven methods achieve low diagnostic accuracy due to ignoring the structural features inside data. To overcome this problem, this article proposes an improved locality preserving projections based on the heat-kernel and cosine weight matrix named Heat-Kernel and Cosine Weights Locality Preserving Projections (HC-LPP). In HC-LPP, a novel weight matrix construction strategy is employed, where a heat-kernel function is combined with a cosine function to optimize the weight matrix between data in terms of shortening distance and angle correlation, respectively. With the new weight matrix, the proposed HC-LPP considers both the distance and the correlation of samples (the shorter the distance is, the closer the neighbors are; the smaller the angle is, the more similar the neighbors are). The dimensionality reduction process of HC-LPP can well maintain the spatial geometric structure of data. Finally, the proposed HC-LPP integrated with the AdaBoost. M2 classifier is applied to the Tennessee Eastman process and the PROcess NeTwork Optimization process for fault diagnosis performance verification. Simulation results show, the proposed HC-LPP achieves better performance in diagnostic accuracy compared with other related methods.

Unsupervised Deep Embedded Clustering for High-Dimensional Visual Features of Fashion Images

Fashion image clustering is the key to fashion retrieval, forecasting, and recommendation applications. Manual labeling-based clustering is both time-consuming and less accurate. Currently, popular methods for extracting features from data use deep learning techniques, such as a Convolutional Neural Network (CNN). These methods can generate high-dimensional feature vectors, which are effective for image clustering. However, high dimensions can lead to the curse of dimensionality, which makes subsequent clustering difficult. The fashion images-oriented deep clustering method (FIDC) is proposed in this paper. This method uses CNN to generate a 4096-dimensional feature vector for each fashion image through migration learning, then performs dimensionality reduction through a deep-stacked auto-encoder model, and finally performs clustering on these low-dimensional vectors. High-dimensional vectors can represent images, and dimensionality reduction avoids the curse of dimensionality during clustering tasks. A particular point in the method is the joint learning and optimization of the dimensionality reduction process and the clustering task. The optimization process is performed using two algorithms: back-propagation and stochastic gradient descent. The experimental findings show that the proposed method, called FIDC, has achieved state-of-the-art performance.

Uncertainty Quantification for Multiphase Computational Fluid Dynamics Closure Relations with a Physics-Informed Bayesian Approach

Multiphase Computational Fluid Dynamics (MCFD) based on the two-fluid model is considered a promising tool to model complex two-phase flow systems. MCFD simulation can predict local flow features without resolving interfacial information. As a result, the MCFD solver relies on closure relations to describe the interaction between the two phases. Those empirical or semi-mechanistic closure relations constitute a major source of uncertainty for MCFD predictions. In this paper, we leverage a physics-informed uncertainty quantification (UQ) approach to inversely quantify the closure relations’ model form uncertainty in a physically consistent manner. This proposed approach considers the model form uncertainty terms as stochastic fields that are additive to the closure relation outputs. Combining dimensionality reduction and Gaussian processes, the posterior distribution of the stochastic fields can be effectively quantified within the Bayesian framework with the support of experimental measurements. As this UQ approach is fully integrated into the MCFD solving process, the physical constraints of the system can be naturally preserved in the UQ results. In a case study of adiabatic bubbly flow, we demonstrate that this UQ approach can quantify the model form uncertainty of the MCFD interfacial force closure relations, thus effectively improving the simulation results with relatively sparse data support.

Asymptotic derivation of a higher-order one-dimensional model for tape springs.

We derive a one-dimensional model for tape springs. The derivation starts from nonlinear thin-shell theory and uses a dimension reduction technique that combines a centreline-based parametrization of the tape-spring midsurface with the assumption that the strain varies slowly along the length of the tape spring. The one-dimensional model is effectively a higher-order rod model: at leading order, the strain energy depends on the extensional, bending and twisting strains and is consistent with classical results from the literature; the two following orders are novel and capture the dependence of the strain energy on the strain gradients. The cross-sectional displacements are solved as part of the dimension reduction process, making the one-dimensional model asymptotically exact. We expect that the model will accurately and efficiently capture the deformations and instabilities in tape springs, including those involving highly localized deformations. This article is part of the theme issue 'Probing and dynamics of shock sensitive shells'.