Conditional Restricted Boltzmann Machine Research Articles

Deep convolutional and conditional neural networks for large-scale genomic data generation.

Applications of generative models for genomic data have gained significant momentum in the past few years, with scopes ranging from data characterization to generation of genomic segments and functional sequences. In our previous study, we demonstrated that generative adversarial networks (GANs) and restricted Boltzmann machines (RBMs) can be used to create novel high-quality artificial genomes (AGs) which can preserve the complex characteristics of real genomes such as population structure, linkage disequilibrium and selection signals. However, a major drawback of these models is scalability, since the large feature space of genome-wide data increases computational complexity vastly. To address this issue, we implemented a novel convolutional Wasserstein GAN (WGAN) model along with a novel conditional RBM (CRBM) framework for generating AGs with high SNP number. These networks implicitly learn the varying landscape of haplotypic structure in order to capture complex correlation patterns along the genome and generate a wide diversity of plausible haplotypes. We performed comparative analyses to assess both the quality of these generated haplotypes and the amount of possible privacy leakage from the training data. As the importance of genetic privacy becomes more prevalent, the need for effective privacy protection measures for genomic data increases. We used generative neural networks to create large artificial genome segments which possess many characteristics of real genomes without substantial privacy leakage from the training dataset. In the near future, with further improvements in haplotype quality and privacy preservation, large-scale artificial genome databases can be assembled to provide easily accessible surrogates of real databases, allowing researchers to conduct studies with diverse genomic data within a safe ethical framework in terms of donor privacy.

Optimization of Vehicle Powertrain Mounting System Based on Generalized Inverse Cascade Method under Uncertainty

This paper presents a summary of the optimization design process for a multi-objective, two-level engineering problem, utilizing the generalized inverse cascade method under uncertainty. The primary objective is to enhance the vibration isolation performance of a mounting system, considering the influence of uncertain factors on its stiffness. The focus is on determining the value range of the design variables at the bottom layer, ensuring that the design goal is met with a specified confidence level. To illustrate the application of this methodology, the optimization design of a powertrain mount is used as a case study. A data-driven approach is adopted, establishing a quantitative mapping relationship between mount stiffness, force transmission rate, modal decoupling rate, and other design indicators. This is achieved through the development of a CRBM-DBN approximate model, which combines Conditional Restricted Boltzmann Machines (CRBMs) and a Deep Belief Network (DBN). Additionally, an intelligent optimization algorithm and interval search technology are employed to determine the optimal design interval for the mount stiffness. Simulation and experimental verification are conducted using selected parameter combinations. The results demonstrate notable improvements in the vibration isolation performance, modal decoupling rate, and vehicle NVH performance when compared to the original state. These findings provide valuable insights for the interval optimization design of similar multi-objective, as well as two-level engineering problems, serving as useful references for future research and applications.

ANALISIS CONDITIONAL RESTRICTED BOLTZMAN MACHINE UNTUK MEMPREDIKSI HARGA SAHAM BANK SYARIAH INDONESIA

This study aims to predict the movement of Bank Syariah Indonesia shares (BRIS.JK) prices using the Conditional Restricted Boltzmann Machine (CRBM) method. Prediction is needed in conducting share transactions, because the increase or decrease in share price movements is very difficult to predict. The CRBM method is a machine learning algorithm used to model the probability distribution of data associated with variables and inputs. CRBM is a type of Restricted Boltzman Machine (RBM) that consists of two layers, namely the input layer and the hidden layer. CRBM is a type of Boltzmann machine model equipped with a conditioned unit that is used to perform analysis and learning on data that has conditional properties. In this research, the first step is to divide several research scenarios. Then conduct CRBM tests to get prediction results. The data used is daily close data. Based on the research that has been done, it is obtained that the best prediction accuracy is in July - August with MAPE below 5%.

Refinements of Approximation Results of Conditional Restricted Boltzmann Machines.

Conditional restricted Boltzmann machines (CRBMs) are the conditional variant of restricted Boltzmann machines (RBMs), which are used to simulate conditional probability distributions. While promising for practical applications, there is a lack of theoretical studies on the approximation ability of CRBMs. In this article, by contributing analysis tools, especially designed for the conditional models, we improve the results of the representational power of CRBMs based on existing work on RBMs. We first study the properties of CRBMs to approximate conditional Markov random fields. On this basis, a universal approximation result is obtained by deriving upper bounds on the minimal number of hidden units, which improves the previous result. Furthermore, the result about maximal approximation errors of CRBMs is also improved by reducing the number of hidden units that can guarantee approximations within a given error tolerance. Furthermore, the representational efficiency of CRBMs in computing deterministic conditional distributions is investigated. Specifically, we show that there are exponentially many deterministic conditional distributions that cannot be computed by CRBMs whose size does not exponentially grow with the number of input units. Some specific examples of these hard-to-present deterministic conditional distributions are provided. Finally, some numerical experiments are carried out to verify the theoretical results of the properties of CRBMs to approximate conditional Markov random fields and the universal approximation of CRBMs.

Modified Conditional Restricted Boltzmann Machines for Query Recommendation in Digital Archives

Digital archives (DAs) usually store diverse expert-level materials. Nowadays, access to DAs is increasing for non-expert users, However, they might have difficulties formulating appropriate search queries to find the necessary information. In response to this problem, we propose a query log-based query recommendation algorithm that provides expert knowledge to non-expert users, thus supporting their information seeking in DAs. The use case considered is one where after users enter some general queries, they will be recommended semantically similar expert-level queries in the query logs. The proposed modified conditional restricted Boltzmann machines (M-CRBMs) are capable of utilizing the rich metadata in DAs, thereby alleviating the sparsity problem that conventional restricted Boltzmann machines (RBMs) will face. Additionally, compared with other CRBM models, we drop a large number of model weights. In the experiments, the M-CRBMs outperform the conventional RBMs when using appropriate metadata, and we find that the recommendation results are relevant to the metadata fields that are used in M-CRBMs. Through experiments on the Europeana dataset, we also demonstrate the versatility and scalability of our proposed model.

Digital Twin simulation for deep learning framework for predicting solar energy market load in Trade-By-Trade data

Because users behave randomly and in a nonlinear way, predicting electrical loads proves to be a difficult procedure. Due to the development of the smart grid (SG) and advanced metering infrastructure (AMI), humans will be capable of recording, monitoring, and analyzing these non-linear behaviors. The use of electric load projection layouts is a necessity in order to make decisions, plan, and evaluate contracts in electrical systems. Consequently, there have been several load prediction methods in the research that demonstrate trade-offs among prediction precision and runtime (convergence rate). The current paper presents a method for short-term load prediction of market in trade-by-trade data that would be quick and precise. Modified mutual information (MMI) is used to extract abstractive characteristics from historic information. Learning empowers the factored conditional restricted Boltzmann machine (FCRBM) for predicting the electrical loads. Ultimately, the efficiency has been optimized using the suggested modified teaching–learning algorithm (MTLA). The suggested architecture has the advantage of improving prediction precision and convergence rate. The MMI method and FCRBM layout improve prediction precision. In addition, MTLA has been used to enhance the convergence rates. Based on simulation outcomes, the suggested quick and precise layout performs better than conventional layouts when it comes to forecasting precision and convergence rates, including Bi-level, MI-artificial neural network (MI-ANN), and accurate fast converging short-term load forecast (AFC-STLF).

DDIT: An Online Predictor for Multiple Clinical Phenotypic Drug-Disease Associations.

Background: Drug repurposing provides an effective method for high-speed, low-risk drug development. Clinical phenotype-based screening exceeded target-based approaches in discovering first-in-class small-molecule drugs. However, most of these approaches predict only binary phenotypic associations between drugs and diseases; the types of drug and diseases have not been well exploited. Principally, the clinical phenotypes of a known drug can be divided into indications (Is), side effects (SEs), and contraindications (CIs). Incorporating these different clinical phenotypes of drug–disease associations (DDAs) can improve the prediction accuracy of the DDAs. Methods: We develop Drug Disease Interaction Type (DDIT), a user-friendly online predictor that supports drug repositioning by submitting known Is, SEs, and CIs for a target drug of interest. The dataset for Is, SEs, and CIs was extracted from PREDICT, SIDER, and MED-RT, respectively. To unify the names of the drugs and diseases, we mapped their names to the Unified Medical Language System (UMLS) ontology using Rest API. We then integrated multiple clinical phenotypes into a conditional restricted Boltzmann machine (RBM) enabling the identification of different phenotypes of drug–disease associations, including the prediction of as yet unknown DDAs in the input. Results: By 10-fold cross-validation, we demonstrate that DDIT can effectively capture the latent features of the drug–disease association network and represents over 0.217 and over 0.072 improvement in AUC and AUPR, respectively, for predicting the clinical phenotypes of DDAs compared with the classic K-nearest neighbors method (KNN, including drug-based KNN and disease-based KNN), Random Forest, and XGBoost. By conducting leave-one-drug-class-out cross-validation, the AUC and AUPR of DDIT demonstrated an improvement of 0.135 in AUC and 0.075 in AUPR compared to any of the other four methods. Within the top 10 predicted indications, side effects, and contraindications, 7/10, 9/10, and 9/10 hit known drug–disease associations. Overall, DDIT is a useful tool for predicting multiple clinical phenotypic types of drug–disease associations.

Forecasting progression of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) with digital twins

AbstractBackgroundMachine learning models can leverage historical data to forecast disease progression. These predictions can be integrated in clinical trial design to reduce sample size or increase power, speeding up the evaluation of new drugs. This is especially critical in AD where trials face challenges with enrollment and where no new therapies have emerged in 18 years. Recently, there has been a shift towards evaluating drugs in earlier stages of the disease using Clinical Dementia Rating Sum‐of‐Boxes (CDR‐SB) as the primary endpoint. We present a model that includes CDR‐SB and spans a broad range of baseline disease severity from MCI to mild‐to‐moderate AD.MethodWe used nearly 7,000 clinical records from placebo arms of AD clinical trials (in the C‐Path Online Data Repository for AD) and from observational studies (in the AD Neuroimaging Initiative) to train Conditional Restricted Boltzmann Machines (CRBMs). A CRBM is a generative machine learning model that learns a multivariate distribution over the relevant variables, and is particularly suited to model clinical data. A CRBM generates Digital Twins ‐ synthetic clinical records with baseline characteristics matched to those of actual trial subjects describing their likely progression under standard‐of‐care (SOC) with/without placebo. These forecasts include all components of CDR‐SB, Alzheimer’s Disease Assessment Scale ‐ Cognitive Subscale (ADAS‐Cog11) and Mini Mental State Examination (MMSE), along with other variables including labs, vitals, and biomarker status. We evaluated Digital Twins on a held‐out portion of the dataset (not included in the training dataset), stratified by baseline ADAS‐Cog11. For each variable, the mean change from baseline +/‐ 95% confidence interval error at 3‐month intervals was estimated by drawing 100 Digital Twins for each subject and averaging over the population.ResultCRBM‐generated predictions of ADAS‐Cog11, MMSE, and CDR‐SB progression up to 18 months were consistent within 95% confidence intervals with actual data across all scores, timepoints, and cohorts.ConclusionDigital Twins accurately model MCI and AD subjects’ progression on SOC with/without placebo across a broad range of baseline disease severity, and can be integrated into clinical trials as prognostic scores to increase power or reduce sample size required to achieve desired power.

Quantum computing based hybrid deep learning for fault diagnosis in electrical power systems

Quantum computing (QC) and deep learning have shown promise of supporting transformative advances and have recently gained popularity in a wide range of areas. This paper proposes a hybrid QC-based deep learning framework for fault diagnosis of electrical power systems that combine the feature extraction capabilities of conditional restricted Boltzmann machine with an efficient classification of deep networks. Computational challenges stemming from the complexities of such deep learning models are overcome by QC-based training methodologies that effectively leverage the complementary strengths of quantum assisted learning and classical training techniques. The proposed hybrid QC-based deep learning framework is tested on a simulated electrical power system with 30 buses and wide variations of substation and transmission line faults, to demonstrate the framework’s applicability, efficiency, and generalization capabilities. High computational efficiency is enjoyed by the proposed hybrid approach in terms of computational effort required and quality of diagnosis performance over classical training methods. In addition, superior and reliable fault diagnosis performance with faster response time is achieved over state-of-the-art pattern recognition methods based on artificial neural networks (ANN) and decision trees (DT).

CRBM-DBN-based prediction effects inter-comparison for significant wave height with different patterns

Based on the Conditional Restricted Boltzmann Machine - Deep Belief Network (CRBM-DBN), we select four patterns and compare their prediction effects for the significant wave height in the Gulf of Mexico (GoM). Historical datasets of all 12 buoys managed by the National Data Buoy Center are employed to train and construct models. Root-mean-square error (RMSE) and coefficient of efficiency (CE) between the observed and predicted wave heights are investigated. We find that for the short-term prediction (i.e., lead time≤12 h), the best results (RMSE<0.24 m and CE > 0.92) are achieved with the univariate significant wave height as the input in most cases of the whole gulf. When the lead time is equal to 24 h or 48 h, the multivariate pattern of “significant wave height + dominant wave direction + wind speed + wind direction” has the optimal effects (0.18 m < RMSE<0.40 m and 0.72 < CE < 0.93) in the vicinity of 26。N oceans. The superiority is very obvious and gradually diminishes as the latitude increases to the north and decreases to the south. Regarding the wave height predictions in different oceans of GoM, the findings provide evidence that it may be contributed to select optimal prediction patterns and obtain the best applications.

Personalized news recommendation based on an improved conditional restricted Boltzmann machine

PurposeBecause of the extensive user coverage of news sites and apps, greater social and commercial value can be realized if users can access their favourite news as easily as possible. However, news has a timeliness factor; there are serious cold start and data sparsity in news recommendation, and news users are more susceptible to recent topical news. Therefore, this study aims to propose a personalized news recommendation approach based on topic model and restricted Boltzmann machine (RBM).Design/methodology/approachFirstly, the model extracts the news topic information based on the LDA2vec topic model. Then, the implicit behaviour data are analysed and converted into explicit rating data according to the rules. The highest weight is assigned to recent hot news stories. Finally, the topic information and the rating data are regarded as the conditional layer and visual layer of the conditional RBM (CRBM) model, respectively, to implement news recommendations.FindingsThe experimental results show that using LDA2vec-based news topic as a conditional layer in the CRBM model provides a higher prediction rating and improves the effectiveness of news recommendations.Originality/valueThis study proposes a personalized news recommendation approach based on an improved CRBM. Topic model is applied to news topic extraction and used as the conditional layer of the CRBM. It not only alleviates the sparseness of rating data to improve the efficient in CRBM but also considers that readers are more susceptible to popular or trending news.

Analyzing joint brand purchases by conditional restricted Boltzmann machines

We introduce the conditional restricted Boltzmann machine as method to analyze brand-level market basket data of individual households. The conditional restricted Boltzmann machine includes marketing variables and household attributes as independent variables. To our knowledge this is the first study comparing the conditional restricted Boltzmann machine to homogeneous and heterogeneous multivariate logit models for brand-level market basket data across several product categories. We explain how to estimate the conditional restricted Boltzmann machine starting from a restricted Boltzmann machine without independent variables. The conditional restricted Boltzmann machine turns out to excel all the other investigated models in terms of log pseudo-likelihood for holdout data. We interpret the selected conditional restricted Boltzmann machine based on coefficients linking purchases to hidden variables, interdependences between brand pairs as well as own and cross effects of marketing variables. The conditional restricted Boltzmann machine indicates pairwise relationships between brands that are more varied than those of the multivariate logit model are. Based on the pairwise interdependences inferred from the restricted Boltzmann machine we determine the competitive structure of brands by means of cluster analysis. Using counterfactual simulations, we investigate what three different models (independent logit, heterogeneous multivariate logit, conditional restricted Boltzmann machine) imply with respect to the retailer’s revenue if each brand is put on display. Finally, we mention possibilities for further research, such as applying the conditional restricted Boltzmann machine to other areas in marketing or retailing.

Conditional restricted Boltzmann machine as a generative model for body‐worn sensor signals

Sensor-based human activity classification requires time and frequency domain feature extraction techniques. The set of choice in time and frequency domain features may have a significant impact on the overall classification accuracy. Another problem is to train deep learning models with sufficient dataset. The use of generative models eliminates the requirement of choosing certain features of the signal. As a generative model, restricted Boltzmann machine (RBM) is an energy-based probabilistic graphical model which factorises the probability distribution of a random variable over a binary probability distribution. Conditional restricted Boltzmann machines (CRBMs) is an extension to RBM, which can capture temporal information in time-series signals and can be deployed as a generative model in classification. In this study, the authors show how CRBMs can be trained to learn signal features. They present four generative model training results, RBM, CRBM, generative adversarial network, Wasserstein generative adversarial network – gradient penalty and compare the models' performances with a performance criterion. They show that the CRBM model can generate signals closest to true signals with a significantly higher success rate as compared to other presented generative models. They present a statistical analysis of the findings and show that the findings significantly hold.

Interdependences of Products in Market Baskets: Comparing the Conditional Restricted Boltzmann Machine to the Multivariate Logit Model

Abstract We analyze market baskets of individual households in two consumer durables categories (music, computer related products) by the multivariate logit (MVL) model, its finite mixture extension (FM-MVL) and the conditional restricted Boltzmann machine (CRBM). The CRBM attains a vastly better out-of-sample performance than MVL and FM-MVL models. Based on simulation-based likelihood ratio tests we prefer the CRBM to the FM-MVL model. To interpret hidden variables of conditional Boltzmann machines we look at their average probability differences between purchase and non-purchases of any sub-category across all baskets. To measure interdependences we compute cross effects between sub-categories for the best performing FM-MVL model and CRBM. In both product categories the CRBM indicates more or higher positive cross effects than the FM-MVL model. Finally, we suggest appropriate future research based on larger and more detailed data sets.

Geometry and expressive power of conditional restricted Boltzmann machines

Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define mo...

Improving maritime traffic emission estimations on missing data with CRBMs

Maritime traffic emissions are a major concern to governments as they heavily impact the Air Quality in coastal cities. Ships use the Automatic Identification System (AIS) to continuously report position and speed among other features, and therefore this data is suitable to be used to estimate emissions, if it is combined with engine data. However, important ship features are often inaccurate or missing. State-of-the-art complex systems, like CALIOPE at the Barcelona Supercomputing Center, are used to model Air Quality. These systems can benefit from AIS based emission models as they are very precise in positioning the pollution. Unfortunately, these models are sensitive to missing or corrupted data, and therefore they need data curation techniques to significantly improve the estimation accuracy. In this work, we propose a methodology for treating ship data using Conditional Restricted Boltzmann Machines (CRBMs) plus machine learning methods to improve the quality of data passed to emission models that can also be applied to other GPS and time-series problems. Results show that we can improve the default methods proposed to cover missing data. In our results, we observed that using our method the models boosted their accuracy to detect otherwise undetectable emissions. In particular, we used a real data-set of AIS data, provided by the Spanish Port Authority, to estimate that thanks to our method, the model was able to detect 45% of additional emissions, representing 152 tonnes of pollutants per week in Barcelona and propose new features that may enhance emission modeling.

Electric load forecasting based on deep learning and optimized by heuristic algorithm in smart grid

Accurate electric load forecasting is important due to its application in the decision making and operation of the power grid. However, the electric load profile is a complex signal due to the non-linear and stochastic behavior of consumers. Despite much research conducted in this area; still, accurate forecasting models are needed. In this article, a novel hybrid short-term electric load forecasting model is proposed. The proposed model is an integrated framework of data pre-processing and feature selection module, training and forecasting module, and an optimization module. The data pre-processing and feature selection module is based on modified mutual information (MMI) technique, which is an improved version of the mutual information technique, used to select abstractive features from historical data. The training and forecasting module is based on factored conditional restricted Boltzmann machine (FCRBM), which is a deep learning model, empowered via learning to forecast the future electric load. The optimization module is based on our proposed genetic wind-driven (GWDO) optimization algorithm, which is used to fine-tune the adjustable parameters of the model. The accuracy of the proposed framework is evaluated through historical hourly load data of three USA power grids, taken from publicly available PJM electricity market. The proposed model is validated by comparing it with four recent forecasting models like Bi-level, mutual information-based artificial neural network (MI-ANN), ANN-based accurate and fast converging (AFC-ANN), and long short-term memory (LSTM) in terms of accuracy and convergence rate.

A Novel Accurate and Fast Converging Deep Learning-Based Model for Electrical Energy Consumption Forecasting in a Smart Grid

Energy consumption forecasting is of prime importance for the restructured environment of energy management in the electricity market. Accurate energy consumption forecasting is essential for efficient energy management in the smart grid (SG); however, the energy consumption pattern is non-linear with a high level of uncertainty and volatility. Forecasting such complex patterns requires accurate and fast forecasting models. In this paper, a novel hybrid electrical energy consumption forecasting model is proposed based on a deep learning model known as factored conditional restricted Boltzmann machine (FCRBM). The deep learning-based FCRBM model uses a rectified linear unit (ReLU) activation function and a multivariate autoregressive technique for the network training. The proposed model predicts future electrical energy consumption for efficient energy management in the SG. The proposed model is a novel hybrid model comprising four modules: (i) data processing and features selection module, (ii) deep learning-based FCRBM forecasting module, (iii) genetic wind driven optimization (GWDO) algorithm-based optimization module, and (iv) utilization module. The proposed hybrid model, called FS-FCRBM-GWDO, is tested and evaluated on real power grid data of USA in terms of four performance metrics: mean absolute percentage deviation (MAPD), variance, correlation coefficient, and convergence rate. Simulation results validate that the proposed hybrid FS-FCRBM-GWDO model has superior performance than existing models such as accurate fast converging short-term load forecasting (AFC-STLF) model, mutual information-modified enhanced differential evolution algorithm-artificial neural network (MI-mEDE-ANN)-based model, features selection-ANN (FS-ANN)-based model, and Bi-level model, in terms of forecast accuracy and convergence rate.

Towards Modified Entropy Mutual Information Feature Selection to Forecast Medium-Term Load Using a Deep Learning Model in Smart Homes.

Over the last decades, load forecasting is used by power companies to balance energy demand and supply. Among the several load forecasting methods, medium-term load forecasting is necessary for grid’s maintenance planning, settings of electricity prices, and harmonizing energy sharing arrangement. The forecasting of the month ahead electrical loads provides the information required for the interchange of energy among power companies. For accurate load forecasting, this paper proposes a model for medium-term load forecasting that uses hourly electrical load and temperature data to predict month ahead hourly electrical loads. For data preprocessing, modified entropy mutual information-based feature selection is used. It eliminates the redundancy and irrelevancy of features from the data. We employ the conditional restricted Boltzmann machine (CRBM) for the load forecasting. A meta-heuristic optimization algorithm Jaya is used to improve the CRBM’s accuracy rate and convergence. In addition, the consumers’ dynamic consumption behaviors are also investigated using a discrete-time Markov chain and an adaptive k-means is used to group their behaviors into clusters. We evaluated the proposed model using GEFCom2012 US utility dataset. Simulation results confirm that the proposed model achieves better accuracy, fast convergence, and low execution time as compared to other existing models in the literature.

Robust sequential high‐resolution range profile recognition based on the infinite conditional restricted Boltzmann machine

Feature extraction for radar high-resolution range profiles (HRRPs) is an essential procedure for robust target recognition based on radar HRRPs. It is almost always better to utilise multiple HRRPs from the target than a single HRRP. The commonly used recognition techniques are not robust enough to extract features from HRRP sequences, especially when there is not enough training data. In this study, the authors present a compact model named the infinite conditional restricted Boltzmann machine (iCRBM) for jointly feature learning and recognition of radar HRRPs. The model is developed from the conditional restricted Boltzmann machine, which is a powerful model for representing high-dimensional and multi-modal conditional distributions. Unlike ordinary RBMs, iCRBM can automatically select the model complexity according to the data, which especially suits for the case of large data dimensionality and small training dataset. A multi-scale structure is also proposed for learning multiple scales of temporal/spatial dependencies. Two types of radar HRRP recognition experiments were conducted to validate the efficiency and robustness of iCRBM: the multi-view HRRP recognition and the small aperture HRRP recognition. The results indicate that, the proposed model can be more adaptive at learning the feature of HRRPs than other common methods. The multi-scale structure is helpful for better recovering missing HRRP samples from non-consecutive sequences.