Boltzmann Machine Research Articles

A deep belief network-based energy consumption prediction model for water source heat pump system

To achieve optimal energy efficiency in buildings, accurately forecasting the energy consumption of air conditioning systems is crucial. This study develops an energy consumption prediction model based on a deep belief network, which is constructed according to the principles of a restricted Boltzmann machine. Actual experimental data from a water source heat pump system are collected, and feature variables are selected. The study discusses the impact of model parameters and training set sizes on the performance of energy consumption prediction model. Additionally, the trend in model prediction performance is analyzed through parameter adjustments. The results show that the coefficient of determination (R2) for the optimized model has increased to 0.585. The mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) have been reduced to 6.311, 2.512, and 1.625, respectively. The deep belief network energy consumption prediction model outperforms other common machine learning models for water source heat pump systems.

Enhancing wind power prediction with self-attentive variational autoencoders: A comparative study

Accurate wind power prediction is critical for efficient grid management and the integration of renewable energy sources into the power grid. This study presents an effective deep-learning approach that improves short-term wind power forecasting accuracy. The method incorporates a Variational Autoencoder (VAE) with a self-attention mechanism applied in both the encoder and decoder. This empowers the model to leverage VAE's strengths in time-series modeling and nonlinear approximation while focusing on the most relevant features within the wind power data. The effectiveness of this approach is evaluated through a comprehensive comparison with eight established deep learning methods, including Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Bidirectional LSTMs (BiLSTMs), Convolutional LSTMs (ConvLSTMs), Gated Recurrent Units (GRUs), Stacked Autoencoders (SAEs), Restricted Boltzmann Machines (RBMs), and vanilla VAEs. Real-world data from five wind turbines in France and Turkey is used for the evaluation. Five statistical metrics are employed to quantitatively assess the prediction performance of each method. The results indicate that the SA-VAE model consistently outperformed other models, achieving the highest average R2 value of 0.992, demonstrating its superior predictive capability compared to existing techniques.

Optimizing electric vehicle paths to charging stations using parallel greylag goose algorithm and Restricted Boltzmann Machines

Optimizing electric vehicle paths to charging stations using parallel greylag goose algorithm and Restricted Boltzmann Machines

Generative Modeling of RNA Sequence Families with Restricted Boltzmann Machines.

In this chapter, we discuss the potential application of Restricted Boltzmann machines (RBM) to model sequence families of structured RNA molecules. RBMs are a simple two-layer machine learning model able to capture intricate sequence dependencies induced by secondary and tertiary structure, as well as mechanisms of structural flexibility, resulting in a model that can be successfully used for the design of allosteric RNA such as riboswitches. They have recently been experimentally validated as generative models for the SAM-I riboswitch aptamer domain sequence family. We introduce RBM mathematically and practically, providing self-contained code examples to download the necessary training sequence data, train the RBM, and sample novel sequences. We present in detail the implementation of algorithms necessary to use RBMs, focusing on applications in biological sequence modeling.

Postoperative facial prediction for mandibular defect based on surface mesh deformation

Objectives: This study aims to introduce a novel predictive model for the post-operative facial contours of patients with mandibular defect, addressing limitations in current methodologies that fail to preserve geometric features and lack interpretability.Methods: Utilizing surface mesh theory and deep learning, our model diverges from traditional point cloud approaches by employing surface triangular mesh grids. We extract latent variables using a Mesh Convolutional Restricted Boltzmann Machines (MCRBM) model to generate a three-dimensional deformation field, aiming to enhance geometric information preservation and interpretability.Results: Experimental evaluations of our model demonstrate a prediction accuracy of 91.2 %, which represents a significant improvement over traditional machine learning-based methods.Conclusions: The proposed model offers a promising new tool for pre-operative planning in oral and maxillofacial surgery. It significantly enhances the accuracy of post-operative facial contour predictions for mandibular defect reconstructions, providing substantial advancements over previous approaches.

Localization of Speaker using Fusion Techniques and Neural Network Algorithms

ABSTACT Sound source localization especially speech and speaker is sole of the most significant techniques recently because used in various applications like smart environments, industry, robots, and audio conferences. So, the usage of these techniques needs more accuracy. In this paper, a speaker localization proposed it depends on the speech signals in closed spaces by employing fusion techniques and neural networks (NN) algorithms to get more accuracy. The proposed work included finding the classification of the speaker signals, which included three phases: the preprocessing phase, the phase of the feature extraction and classification phase. Data Fusion technique used to generate the dataset of speakers. In feature extraction phase features fusion technique was used for constructing a feature vector by using Generalized Cross Correlation (GCC) for time delay estimation, Root_MUSIC, and Minimum Variance Distortion Less (MVDR) for a direction of arrival for the signal source. In the classification stage two NN algorithms used, Restricted Boltzmann Machine (RBM), which implemented using Tensor flow library and Long Short-Term Memory (LSTM), which implemented using Keras library. The experiments results shows that the accuracy of the two methods was 99.84%, 99.15% for RBM, and LSTM respectively.

A surrogate-assisted evolutionary algorithm with dual restricted Boltzmann machines and reinforcement learning-based adaptive strategy selection

To improve the effectiveness of surrogate-assisted evolutionary algorithms (SAEAs) in solving high-dimensional expensive optimization problems with multi-polar and multi-variable coupling properties, a new approach called DRBM-ASRL is proposed. This approach leverages restricted Boltzmann machines (RBMs) for feature learning and reinforcement learning for adaptive strategy selection. DRBM-ASRL integrates four search strategies based on three heterogeneous surrogate modeling approaches, each catering to different preferences. Two of these strategies focus on generative sampling in the subspaces with varying dimensions, while the other two aim to explore the local and global landscapes in the high-dimensional source space. This allows for more effective tradeoffs between exploration and exploitation in the solution space. Reinforcement learning is employed to adaptively prioritize the search strategies during optimization , based on the online feedback information from the optimal solution. In addition, to enhance the representation of potentially optimal samples in the solution space, two task-driven RBMs are separately trained to construct a feature subspace and reconstruct the features of the source space. DRBM-ASRL has been evaluated on various high-dimensional benchmarks ranging from 50 to 200 dimensions, as well as 14 CEC 2013 complex benchmark problems with 100 dimensions and a power system problem with 118 dimensions. Experimental results demonstrate its superior convergence performance and optimization efficiency compared to eight state-of-the-art SAEAs.

Machine learning in biological physics: From biomolecular prediction to design

Machine learning has been proposed as an alternative to theoretical modeling when dealing with complex problems in biological physics. However, in this perspective, we argue that a more successful approach is a proper combination of these two methodologies. We discuss how ideas coming from physical modeling neuronal processing led to early formulations of computational neural networks, e.g., Hopfield networks. We then show how modern learning approaches like Potts models, Boltzmann machines, and the transformer architecture are related to each other, specifically, through a shared energy representation. We summarize recent efforts to establish these connections and provide examples on how each of these formulations integrating physical modeling and machine learning have been successful in tackling recent problems in biomolecular structure, dynamics, function, evolution, and design. Instances include protein structure prediction; improvement in computational complexity and accuracy of molecular dynamics simulations; better inference of the effects of mutations in proteins leading to improved evolutionary modeling and finally how machine learning is revolutionizing protein engineering and design. Going beyond naturally existing protein sequences, a connection to protein design is discussed where synthetic sequences are able to fold to naturally occurring motifs driven by a model rooted in physical principles. We show that this model is “learnable” and propose its future use in the generation of unique sequences that can fold into a target structure.

Application of Deep Learning Algorithm in Visual Optimization of Industrial Design

In order to understand the application of degree learning algorithms in industrial design visual optimization, the author proposes an application research based on deep learning algorithms in industrial design visual optimization. The author first starts with the basic principles of deep learning algorithms and provides a detailed explanation of the basic structure of the single-layer network of deep learning algorithms, as well as the restricted Boltzmann machine and its training process. Finally, an example of the performance improvement brought by the application of deep learning technology in handwritten digit recognition was given through an automatic encoder, and a simple summary of deep learning technology was made. Secondly, the NCI matching algorithm is used to match industrial design products and reconstruct the point cloud of industrial products to accurately detect their features. On this basis, deep learning algorithms are applied to construct a visual optimization model for industrial design, determine the output format of the model scene and the output situation of industrial design, and process the model according to the changing characteristics of the industrial design model to comprehensively edit the data of the model. Finally, targeted optimization of the industrial design visual optimization model is carried out based on the technical characteristics of deep learning algorithms, in order to complete the industrial design visual optimization. The experimental comparison results show that the industrial design visual optimization method optimized in this design has higher visual clarity than the traditional industrial design method, reaching 98%. Greatly improves the visual clarity of industrial design.

Liouville-space neural network representation of density matrices

Neural network quantum states such as ansatz wave functions have shown a great deal of promise for finding the ground state of spin models. Recently, work has focused on extending this idea to mixed states for simulating the dynamics of open systems. Most approaches so far have used a purification ansatz where a copy of the system Hilbert space is added, which when traced out gives the correct density matrix. Here we instead present an extension of the restricted Boltzmann machine which directly represents the density matrix in Liouville space. This allows the compact representation of states which appear in mean-field theory. We benchmark our approach on two different versions of the dissipative transverse-field Ising model, which show our ansatz is able to compete with other state-of-the-art approaches. Published by the American Physical Society 2024

Self-supervised Gaussian Restricted Boltzmann Machine via joint contrastive representation and contrastive divergence

In this paper, we propose a novel self-supervised Gaussian Restricted Boltzmann Machine with contrastive learning (CL-GRBM), which fuses contrastive representation learning and contrastive divergence to optimize and enhance the representation of GRBM. Built upon the concept of contrastive representation learning, CL-GRBM aims to enhance the representation capacity of the GRBM. Firstly, a pair of positive samples are constructed by one times Gibbs sampling with the original data. Then, the contrastive loss is used to pull the positive samples closer and push other samples further apart, making similar representations closer and different representations farther apart. In summary, during the training process of CL-GRBM using the CD algorithm, the objective is to align the sampling probability distribution of the visible layer in CL-GRBM as closely as possible with the empirical distribution of the original data. In the feature space of the sampling probabilities in the hidden layer, the distance between positive samples is minimized to capture the intrinsic structure of the data. According to the experimental verification, the proposed CL-GRBM shows better performance than contrastive models. As a shallow self-supervised model, it has even better performance than some excellent deep self-supervised models.

Quantum Boltzmann machines: Clifford algebras and asymptotic analysis

We analyze mathematical aspects related to general quantum Boltzmann machine models through representations of algebras and functional analytic techniques. In this context, based on algebraic formulation we show the existence of a symmetry group for a class of Hamiltonians based on generators of Clifford algebras and discuss its implications for the learning scheme. Also, we have developed a rigorous mathematical analysis that allows us to investigate issues related to the asymptotic behavior of quantum Boltzmann machines. Particularly, we obtain a lower bound of fidelity in terms of the upper limit of quantum relative entropy.

Neural-network quantum-state study of the long-range antiferromagnetic Ising chain.

We investigate quantum phase transitions in the transverse field Ising chain with algebraically decaying long-range (LR) antiferromagnetic interactions using the variational Monte Carlo method with the restricted Boltzmann machine employed as a trial wave function ansatz. First, we measure the critical exponents and the central charge through the finite-size scaling analysis, verifying the contrasting observations in the previous tensor network studies. The correlation function exponent and the central charge deviate from the short-range (SR) Ising values at a small decay exponent α_{LR}, while the other critical exponents examined are very close to the SR Ising exponents regardless of α_{LR} examined. However, in the further test of the critical Binder ratio, we find that the universal ratio of the SR limit does not hold for α_{LR}<2, implying a deviation in the criticality. On the other hand, we find evidence of the conformal invariance breakdown in the conformal field theory (CFT) test of the correlation function. The deviation from the CFT description becomes more pronounced as α_{LR} decreases, although a precise breakdown threshold is yet to be determined.

Combined localization and clustering approach for reduced energy presumption in heterogeneous IoT

The field of H-IoT is emerging with enormous potential to empower various technologies. Smart cities and advanced manufacturing are a few of the fields where H-IoT is currently used. The issue with H-IoT is its heavy energy consumption while transmitting data, which makes scaling difficult. To overcome such issues, a hybrid approach of Crayfish Optimization (CFO) with FCM and Restricted Boltzmann Machine (RBM) with Soft Sign Activation (SSA) has been proposed. Initially, Node initialization lays the foundation by configuring individual sensor nodes for network participation. After initialization, Fuzzy C Means clustering optimizes data aggregation by categorizing nodes into clusters based on similarity. Gathering Neighbor Node Traffic Data (NNTD) provides insights into communication patterns. Based on the threshold of NNTD, node localization is performed that enhances network accuracy by pinpointing sensor node locations. Integration of CFO into clustering, along with localization further improves cluster head selection for optimal data routing. Classification through the RBM with SSA function enhances anomaly detection, combining data analysis for optimizing energy utilization in heterogeneous IoT environments. The ‘combined CFO-FCM and SSA-RBM’ has been implemented in MATLAB and achieved an accuracy of 94.50%. As a result, the overall performance of the system is improved.

A policy configured resource management scheme for AHNS using link reliability K‐means clustering algorithm and Weibull distribution‐based blue monkey optimization

SummaryCognitive Radio Ad Hoc Networks (CRAHNs) are an essential method for resolving conflicts between extreme spectrum scarcity and rapid traffic increase while maintaining high‐quality service for consumers. However, the coexistence of primary and secondary users represents a critical challenge for reasonable resource allocation in order to provide a sustaining system performance. Many approaches have been developed to efficiently allocate resources; however, these methods are currently limited by things like user collision, strange traffic networks, and high data transmission error rates. To address these constraints, this paper proposes a policy‐configured reinforcement learning‐based ad hoc network (AHN) model. To obtain the ideal policy configuration for the network, the system first models the cognitive radio (CR) network, in which nodes are initialised and grouped employing the Link Reliability K‐Means clustering Algorithm (LR‐KMA). The available spectrum was then detected and separated into multiple bands utilizing coherent‐based detection (CBD) and signal source identification employing the Parzen–Rosenblatt Window‐based Restricted Boltzmann Machine (PRW‐RBM). Next, the learning model for the resource allocation process employs the Weibull Distribution‐based Blue Monkey Optimization (WD‐BMO) approach to pick the relevant bands. Finally, the experimental results were analyzed in order to evaluate the proposed resource allocation model's performance in CRAHNs. When compared with previous findings, the proposed method improves resource utilization by 5%, the proposed model achieves a 7% higher throughput, and the PRW‐RBM's accuracy improves classification accuracy by 1.07%.

Adoption of deep learning-based magnetic resonance image information diagnosis in brain function network analysis of Parkinson’s disease patients with end-of-dose wearing-off

Objectivethis study was to analyze the brain functional network of end-of-dose wearing-off (EODWO) in patients with Parkinson’s disease (PD) using a convolutional neural network (CNN)-based functional magnetic resonance imaging (fMRI) data classification model. Methodsone hundred PD patients were recruited and assigned to control (Ctrl) group (39 cases without EODWO) and experimental (Exp) group (61 cases with EODWO). The data classification model based on a CNN was employed to assist the analysis of the changes in brain functional network structure in the two groups. The CNN-based fMRI data classification model was primarily based on a CNN architecture, with improvements made to the initialization of convolutional kernel parameters. Firstly, a structure based on restricted Boltzmann machine (RBM) was constructed, followed by the initialization of convolutional kernel parameters. Subsequently, the model underwent training. Utilizing the data analysis module within the GRETNA toolbox, extracted feature sets were analyzed, including local measures such as betweenness centrality (BC) and degree centrality (DC), as well as global measures such as global efficiency (Eg) and local efficiency (Eloc). Resultsas sparsity increased, there was a gradual upward trend observed in Eg; however, the values of Eg in both brain functional networks remained relatively stable within the range of 0.2–0.5. The Eg value of the Ctrl group’s whole-brain functional network was 0.17 ± 0.02, while that of the Exp group’s whole-brain functional network was 0.17 ± 0.03, with no significant difference between them (P>0.05). The functional DC value of the superior frontal gyrus in the Exp group (8.71 ± 2.56) was significantly lower than that of the Ctrl group (13.32 ± 3.22), whereas the functional DC value of the anterior cingulate gyrus in the Exp group (19.33 ± 4.78) was significantly higher than that of the Ctrl group (15.21 ± 4.02) (P<0.05). There was no significant correlation observed between the functional DC value and levodopa or dopamine agonist therapy (DDT) in the Exp group, whereas the Ctrl group exhibited a significant positive correlation. Conclusionanalysis conducted via a CNN-based fMRI data classification model revealed a correlation between the occurrence of EODWO in PD patients and functional impairments in the left precuneus. Additionally, the occurrence of EODWO may potentially diminish the plasticity of the central prefrontal dopamine.

Unlearning regularization for Boltzmann machines

Boltzmann machines (BMs) are graphical models with interconnected binary units, employed for the unsupervised modeling of data distributions. When trained on real data, BMs show the tendency to behave like critical systems, displaying a high susceptibility of the model under a small rescaling of the inferred parameters. This behavior is not convenient for the purpose of generating data, because it slows down the sampling process, and induces the model to overfit the training-data. In this study, we introduce a regularization method for BMs to improve the robustness of the model under rescaling of the parameters. The new technique shares formal similarities with the unlearning algorithm, an iterative procedure used to improve memory associativity in Hopfield-like neural networks. We test our unlearning regularization on synthetic data generated by two simple models, the Curie–Weiss ferromagnetic model and the Sherrington–Kirkpatrick spin glass model. We show that it outperforms Lp -norm schemes and discuss the role of parameter initialization. Eventually, the method is applied to learn the activity of real neuronal cells, confirming its efficacy at shifting the inferred model away from criticality and coming out as a powerful candidate for actual scientific implementations.

Embedded Restricted Boltzmann Machine Approach for Adjustments of Repetitive Physical Activities Using IMU Data

Embedded Restricted Boltzmann Machine Approach for Adjustments of Repetitive Physical Activities Using IMU Data

Violence detection in crowd videos using nuanced facial expression analysis

Video analysis for violence detection is crucial, especially when dealing with crowd data, where the potential for severe mob attacks in sensitive areas is high. This paper proposes a solution utilizing Convolutional Restricted Boltzmann Machine (CRBM) for video analysis, integrating the strengths of Convolutional Neural Network (CNN) and Restricted Boltzmann Machine (RBM). By focusing on image patches rather than entire frames, the method addresses the challenge of object detection in crowded scenes. The CRBM combines deep-level image analysis from CNN with unsupervised feature extraction in RBM, facilitated by image convolution using Gabor filters in the hidden layer. Dropout regularization mitigates overfitting, enhancing model generality. Extracted features are inputted into an SVM classifier for face detection and a custom VGG16 model for emotion identification. Event probability is then determined through logistic regression based on facial expressions. Despite existing approaches for smart crowd behaviour identification, there remains a tradeoff between accuracy and processing time. Our proposed solution addresses this by employing proper frame preprocessing techniques for feature extraction. Validation using quantitative and qualitative metrics confirms the effectiveness of the approach.

A Comprehensive Survey of Predictive Maintenance Techniques for Aircraft Engines Utilizing the C-MAPSS Dataset

The application of deep learning and sophisticated machine learning techniques is driving the rapid advancement of aircraft engine prognostics and predictive maintenance. Remain ing Useful Life (RUL) of aviation engines has been the subject of numerous studies aimed at improving prediction accuracy and efficacy to improve aviation safety and maintenance plans. Innovative approaches and technologies are demonstrated by these projects, which use a variety of methodologies and datasets, including C-MAPSS and N-CMAPSS. Combining feature engi neering, ensemble learning, and deep learning models such as Restricted Boltzmann Machines (RBMs), Long Short-Term Mem ory (LSTM) networks, Convolutional Neural Networks (CNNs), and Deep Bidirectional Recurrent Neural Networks (DBRNNs) is one prominent method. Features are chosen and models are optimized using a variety of methods, including Genetic Algorithms, Recursive Feature Elimination, Lasso, and Feature Importances. In prognostic modeling, the research emphasize the need of interpretability, model adaptability, and measuring uncertainty. Additionally, in order to pinpoint important features and improve model transparency, the study investigates the use of explainable AI techniques such aggregated feature importances with cross-validation (AFICv) and Shapley additive explanation (SHAP). In order to capture prediction uncertainties, the inte gration of Gaussian Processes (GPs) and Bayesian Deep Neural Networks (DNNs) is also investigated. This provides insights into uncertainty-aware prognosis and predictive analytics for industrial assets. The development and publication of datasets such as the N-CMAPSS dataset also makes it possible to conduct more comprehensive and realistic assessments of prognostic models under real-world flight conditions, providing useful tools for benchmarking and improving machine learning algorithms in predictive maintenance. All things considered, these research projects highlight current developments and the possibility of combining cutting edge technology to improve system reliability, improve predictive maintenance techniques, and guarantee safer airline operations. Key Words: Aircraft engine, Prognostics, Predictive main tenance, Machine learning, Deep learning, Remaining Useful Life (RUL), Aviation safety, Feature engineering, Ensemble learning, Uncertainty quantification, Explainable AI, Bayesian Deep Neural Networks (DNNs), Gaussian Processes (GPs), N CMAPSS dataset, Real flight conditions, Industrial assets, System reliability, Benchmarking