Bayesian Learning Method Research Articles

Reduced-Complexity Subarray-Level Sparse Recovery STAP for Multichannel Airborne Radar WGMTI Application

For multichannel airborne radars, wide-area ground-moving target indication (WGMTI) processing can quickly obtain the dynamic distribution of moving targets in a wide area, which holds considerable significance in many fields. Nevertheless, the WGMTI mode suffers from the interference of powerful ground clutter, which frequently submerges slow-moving targets and causes many false alarms in subsequent moving target detection. Space–time adaptive processing (STAP) can successfully suppress clutter, but its performance depends critically on the available training samples. Consequently, an effective STAP method characterized by fast processing and a small sample size for WGMTI application in multichannel airborne radars must be developed. In this paper, a subarray-level sparse recovery STAP (SR-STAP) processing framework is proposed for multichannel airborne radars. First, the characteristics of the subarray-level received clutter are discussed in detail. Second, on the basis of this analysis, we further designed a joint space–time dictionary and developed a separable tensor-based sparse Bayesian learning (STSBL) method. In this method, two-stage decomposition is proposed to ensure that large-scale data can be degraded into small-scale data in processing, which significantly improves computation efficiency. Finally, the effectiveness of the proposed STSBL-STAP method in WGMTI processing was verified using real measurement data obtained from a developed dual-channel Ku-band airborne radar.

Polynomial chaos surrogate and bayesian learning for coupled hydro-mechanical behavior of soil slope

As rainfall infiltrates into soil slopes, the hydraulic and mechanical behaviors of soils are interacted. In this study, an efficient probabilistic parameter estimation method for coupled hydro-mechanical behavior in soil slope is proposed. This method integrates the Polynomial Chaos Expansion (PCE) method, the coupled hydro-mechanical modeling, and the Bayesian learning method. A coupled hydro-mechanical numerical model is established for the simulation of behaviors of unsaturated soil slope under rainfall infiltration, following by training a cheap-to-run PCE surrogate to replace it. Probabilistic estimation of soil parameters is conducted based on the Bayesian learning technique with the Markov Chain Monte Carlo (MCMC) simulation. A numerical example of an unsaturated slope under rainfall infiltration is presented to illustrate the proposed method. The effects of measurement durations and response types on parameter estimation are addressed. The result shows that with the increase of measurement duration, the uncertainties of soil parameters are significantly reduced. The uncertainties of hydraulic properties are reduced significantly using the pore water pressure data, while the uncertainties of soil strength parameters are reduced greatly using the measured displacement data.

A Bayesian Approach for Joint Discriminative Dictionary and Classifier Learning

Sparse representation has been widely applied to image classification, where the key issue is to extract a suitable discriminative dictionary. To this end, we propose a joint dictionary and classifier learning algorithm based on a parameterized Bayesian model. Therein, the Gaussian priors of a dictionary endow it with the capability of discrimination and representation. Moreover, we introduce a multivariate Gaussian prior for the sparse codes to achieve group sparsity, thereby substantially improving the classification performance. Furthermore, the sparse codes are estimated by a group-sparse Bayesian learning (GSBL) method, and the dictionary atoms are updated sequentially by maximizing a posterior. Moreover, to avoid manual parameter adjustment, the hyperparameters are optimized by an evidence maximization method. Accordingly, we develop a classification scheme via GSBL. Finally, extensive experiments are conducted on six benchmark datasets of face classification, object recognition, handwritten recognition, and scene categorization to substantiate the effectiveness and superiority of the proposed method.

Learning-Based Sparse Recovery for Joint Activity Detection and Channel Estimation in Massive Random Access Systems

We consider the problem of joint activity detection and channel estimation in massive random access. When the receiver has multiple antennas, this is a joint sparse recovery problem with multiple measurement vectors (MMV). For the general setting where the channels could be correlated across antennas, we first develop a modified minimum mean squared error (MMSE) shrinkage function to be used in the Trainable Iterative Soft Thresholding Algorithm (TISTA). Then, we learn this MMSE shrinkage function using a model-based neural network. In the simulation results, the proposed learning-based method, L-MMSE-MMV-TISTA, offers a 30-40% reduction in preamble length requirement compared to TISTA. We also compare L-MMSE-MMV-TISTA with the state-of-the-art MMV sparse Bayesian learning (M-SBL) method. While M-SBL can provide better performance at the cost of higher complexity in highly measurement-constrained settings, LMMSE-MMV-TISTA provides a significant complexity advantage when only a slightly larger number of measurements are available.

Learning, disagreement and inflation forecasting

This paper studies inflation forecasting based on the Bayesian learning algorithm which simultaneously learns about parameters and state variables. The Bayesian learning method updates posterior beliefs with accumulating information from inflation and disagreement about expected inflation from the Survey of Professional Forecasters (SPF). The empirical results show that Bayesian learning helps refine inflation forecasts at all horizons over time. Incorporating a Student’s t innovation improves the accuracy of long-term inflation forecasts. Including disagreement has an effect on refining short-term inflation density forecasts. Furthermore, there is strong evidence supporting a positive correlation between disagreement and trend inflation uncertainty. Our findings are helpful for policymakers when they forecast the future and make forward-looking decisions.

Fast grid-free strength mapping of multiple sound sources from microphone array data using a Transformer architecture.

Conventional microphone array methods for the characterization of sound sources that require a focus-grid are, depending on the grid resolution, either computationally demanding or limited in reconstruction accuracy. This paper presents a deep learning method for grid-free source characterization using a Transformer architecture that is exclusively trained with simulated data. Unlike previous grid-free model architectures, the presented approach requires a single model to characterize an unknown number of ground-truth sources. The model predicts a set of source components, spatially arranged in clusters. Integration over the predicted cluster components allows for the determination of the strength for each ground-truth source individually. Fast and accurate source mapping performance of up to ten sources at different frequencies is demonstrated and strategies to reduce the training effort at neighboring frequencies are given. A comparison with the established grid-based CLEAN-SC and a probabilistic sparse Bayesian learning method on experimental data emphasizes the validity of the approach.

Low-frequency sound source localization in enclosed space based on time reversal method

This study investigates sound source localization (SSL) in enclosed space at low frequencies. Reverberation and noise are important factors that affect localization result. Time reversal technology has the characteristics of compensating the multipath effect and adaptive focus, therefore, it can realize (SSL) in enclosed space. In order to improve the spatial resolution of localization, the recovery of the second transmitting signal received after time reversal is completed by the compressive sensing algorithm. We propose time reversal sparse Bayesian learning (TR-SBL) method to solve the problem of SSL. Firstly, time reversal focusing model of sound source propagation in the enclosed space is established. Then compressive sensing algorithm is performed on the second received signal, a sparse Bayesian framework is used to solve the localization problem. The time reversal method can overcome the reverberation interference in the enclosed space, and the compressive sensing algorithm can improve the spatial resolution. A series of simulations, which can be compared with the time reversal method and L1 norm method, are carried out to verify the localization performance of the proposed method, including simulations of sound sources with different frequencies, signal-to-noise ratios (SNRs), absorption coefficients, and positions. The algorithm is also evaluated in a reverberation chamber using sound sources with different frequencies. The numerical simulations and experimental results show that the proposed method achieves good localization performance in a low-SNR and reverberation environment, the spatial resolution is up to 0.1 m for a sound source with a frequency of 125 Hz.

Diversity entropy-based Bayesian deep learning method for uncertainty quantification of remaining useful life prediction in rolling bearings

Remaining useful life (RUL) prediction of rolling bearings plays a critical role in reducing unplanned downtime and improving machine productivity. The existing prediction methods primarily provide point estimates of RUL without quantifying uncertainty. However, uncertainty quantification of RUL is crucial to conduct reliable risk analysis and make maintenance decision, which can significantly decrease the maintenance costs. To solve the uncertainty quantification problem and improve prediction accuracy at the same time, a novel diversity entropy-based Bayesian deep learning (DE-BDL) method is proposed. First, start degradation time (SDT) of bearings is adaptively determined using diversity entropy, which can extract early degradation information. Then, multi-scale diversity entropy (MDE) is developed to extract dynamic characteristics over multiple scales. Third, the obtained features using MDE are fed into the BDL model for degradation tracking and prediction. By doing this, the proposed DE-BDL method has merits in subsequent decision making, which can not only provide point estimation but also offer uncertainty quantification with epistemic uncertainty and aleatoric uncertainty. The superiority of the proposed method is validated using run-to-failure data. The experimental results and comparison with state-of-art prediction methods have demonstrated that the proposed DE-BDL method is promising for RUL of rolling bearings.

A joint DOA and polarization estimation method based on the conformal polarization sensitive array from the sparse reconstruction perspective

The conformal polarization sensitive array (CPSA) is formed by placing some vector sensors on the conformal array and it has a wide range of practical application in direction of arrival (DOA) and polarization parameters estimation. However, due to the diversity of each sensor’s direction, the performance of the conventional parameters estimation methods based on the CPSA would decrease greatly, especially under the low signal to noise ratio (SNR) and limited snapshots. In order to solve this problem, a unified framework and sparse reconstruction perspective for joint DOA and polarization estimation based on CPSA is proposed in this paper. Specifically, the array received signal model of the CPSA is formulated first and the two-dimensional spatial sparsity of the incident signals is then exploited. Subsequently, after employing the singular value decomposition method to reduce the dimension of array output matrix, the variational sparse Bayesian learning and orthogonal matching pursuit methods are utilized to solve the source DOA estimation, respectively. Finally, the polarization parameters are obtained by the minimum eigenvector method. Simulation results demonstrate that the novel approaches can provide improved estimation accuracy and resolution with low SNR and limited snapshots.

Random Access With Massive MIMO-OTFS in LEO Satellite Communications

This paper considers the joint channel estimation and device activity detection in the grant-free random access systems, where a large number of Internet-of-Things devices intend to communicate with a low-earth orbit satellite in a sporadic way. In addition, the massive multiple-input multiple-output (MIMO) with orthogonal time-frequency space (OTFS) modulation is adopted to combat the dynamics of the terrestrial-satellite link. We first analyze the input-output relationship of the single-input single-output OTFS when the large delay and Doppler shift both exist, and then extend it to the grant-free random access with massive MIMO-OTFS. Next, by exploring the sparsity of channel in the delay-Doppler-angle domain, a two-dimensional pattern coupled hierarchical prior with the sparse Bayesian learning and covariance-free method (TDSBL-FM) is developed for the channel estimation. Then, the active devices are detected by computing the energy of the estimated channel. Finally, the generalized approximate message passing algorithm combined with the sparse Bayesian learning and two-dimensional convolution (ConvSBL-GAMP) is proposed to decrease the computations of the TDSBL-FM algorithm. Simulation results demonstrate that the proposed algorithms outperform conventional methods.

Bayesian Network Structure Learning Method Based on Causal Direction Graph for Protein Signaling Networks

Constructing the structure of protein signaling networks by Bayesian network technology is a key issue in the field of bioinformatics. The primitive structure learning algorithms of the Bayesian network take no account of the causal relationships between variables, which is unfortunately important in the application of protein signaling networks. In addition, as a combinatorial optimization problem with a large searching space, the computational complexities of the structure learning algorithms are unsurprisingly high. Therefore, in this paper, the causal directions between any two variables are calculated first and stored in a graph matrix as one of the constraints of structure learning. A continuous optimization problem is constructed next by using the fitting losses of the corresponding structure equations as the target, and the directed acyclic prior is used as another constraint at the same time. Finally, a pruning procedure is developed to keep the result of the continuous optimization problem sparse. Experiments show that the proposed method improves the structure of the Bayesian network compared with the existing methods on both the artificial data and the real data, meanwhile, the computational burdens are also reduced significantly.

Application of a Bayesian Network Learning Model to Predict Longitudinal Trajectories of Executive Function Difficulties in Elementary School Students.

Executive function is the mental ability to modulate behavior or thinking to accomplish a task. This is developmentally important for children’s academic achievements and ability to adjust to school. We classified executive function difficulties (EFDs) in longitudinal trajectories in Korean children from 7 to 10 years old. We found predictors of EFDs using latent class growth analysis and Bayesian network learning methods with Panel Study data. Three types of latent class models of executive function difficulties were identified: low, intermediate, and high EFDs. The modeling performance of the high EFD group was excellent (AUC = .91), and the predictors were the child’s gender, temperamental emotionality, happiness, DSM (Diagnostic and Statistical Manual of Mental Disorders) anxiety problems, and the mother’s depression as well as coparenting conflict recognized by the mother. The results show that using latent class growth analysis and Bayesian network learning are helpful in classifying the longitudinal EFD patterns in elementary school students. Furthermore, school-age EFD is affected by emotional problems in parents and children that continue from early life. These findings can support children’s development and prevent risk by preclassifying children who may experience persistent EFD and tracing causes.

Stochastic one-way carsharing systems with dynamic relocation incentives through preference learning

The development of carsharing services is expected to achieve greater resource efficiency and provide a sustainable solution for future mobility systems. However, operators inevitably face the imbalance between demand and supply in one-way carsharing systems (CSSs). Also, it is challenging for them to make quick and efficient operational decisions when both travel time and trip requests are uncertain. This study leverages historical and online data and proposes a learning-based methodology to quickly make real-time decisions for CSSs, including vehicle assignment, relocation, and user incentive decisions. Compared to the literature in approximate dynamic programming (ADP), which mostly focuses on uncertainty in trip requests, we propose an offline–online ADP approach to consider the temporal and spatial uncertainty in both trip requests and travel time. To tackle our high-dimensional problem, we integrate an online look-ahead policy into the offline value function approximation (VFA) policy to produce a computational tractable high-quality dynamic fleet management policy. Furthermore, a user-based relocation strategy is investigated to rebalance the fleet distribution to meet the demand better. Specifically, we aim to solve the optimal incentives the operator could offer to users to relocate cars, while user preferences towards relocation incentives are generally unknown in practice. We further enhance our anticipatory policy by developing an online module via Bayesian learning that learns the preference model on the fly using users’ revealed preference data collected online. The numerical experiments in Singapore demonstrate our offline–online ADP approach improves the solution quality and significantly compared to offline VFA policy. The results also confirm the importance of incorporating uncertainty in travel time. The benefits of the user-based relocation scheme using the Bayesian learning method using online data to enhance anticipatory decisions and learn unknown user preferences are also illustrated. It quickly and efficiently learns the user preferences, which further reduces the relocation cost and increases the profit.

A Bayesian deep learning method for freeway incident detection with uncertainty quantification

Incident detection is fundamental for freeway management to reduce non-recurrent congestions and secondary incidents. Recently, machine learning technologies have made considerable progress in the incident detection field, but many still face challenges in uncertainty quantification due to the aleatoric uncertainty of traffic data and the epistemic uncertainty of model deviations. In this study, a Bayesian deep learning method was proposed for freeway incident detection with uncertainty quantification. A convolutional neural network variant was designed on a Bayesian framework, and mechanisms of Bayes by backpropagation and local reparameterization technics were used to update the weight of the proposed model. The predictive uncertainty of the proposed method was modeled jointly by integrating the aleatoric and epistemic uncertainty. The proposed model was tested on the PORTAL dataset and compared with four benchmark models: standard normal deviate, wavelet neural network, long-short term memory neural network, and convolutional neural network. The results show that the proposed model outperforms the baseline methods in terms of accuracy, detection rate and false alarm rate. Perturbation experiments were used to test the robustness of the model against noise. The results indicated that the aleatoric uncertainty of the model remained almost constant under different noise levels. The proposed method may benefit future studies on uncertainty quantification while using machine learning method in incident management and other fields in intelligent transportation systems.

Bayesian inductive learning in group recommendations for seen and unseen groups

Group recommendation refers to a recommendation of items to a group of users (i.e., members). When predicting relevant items, a model commonly faces unseen groups that do not appear in the training step. Recently, deep neural networks and an attention mechanism were applied to group recommendations by aggregating user preferences. However, current methods are insufficient to handle unseen groups (i.e., transductive models) or struggle to compute cost-effective attention networks and regularizations. In this study, we propose the novel Bayesian inductive learning method, called IndiG, for making recommendations to seen and unseen groups. To learn inductively, a function distribution consisting of efficient attention-based aggregation is used as shared information across groups. By incorporating a transductive model as a posterior into the proposed Bayesian method, an inductive model as a prior can learn robustly. We adopt cost-effective regularization to prevent degenerated solutions by maximizing a correlation between group representations of a transductive model and an inductive model, while decorrelating dimensions of group representations. Through experiments, we demonstrated that the proposed method outperformed other existing methods. The experiments also showed that the utilization of uncertainty on the predicted ratings of items worked effectively to improve the performance.

A fast sparse Bayesian learning method with adaptive Laplace prior for space‐time adaptive processing

Space-time adaptive processing with finite samples is supposed to be a crucial technique for airborne radar systems. Inspired by the application of Gaussian prior in sparse Bayesian learning algorithm and the adaptive least absolute shrinkage and selection operator algorithm, a hierarchical Bayesian framework with adaptive Laplace priors is proposed. In this paper, a novel method is applied to avoid the high-dimension matrix inverse operation in the proposed algorithm. Moreover, in order to apply the method in the complex-valued domain, the complex-valued signal is split into two independent variables. Then, the sparse recovery problem in the complex-valued domain can be transformed into the real-value domain. Simulation experiments show that the proposed algorithm can achieve great clutter suppression performance and also ensure high computational efficiency.

Multiagent Bayesian Deep Reinforcement Learning for Microgrid Energy Management Under Communication Failures

Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this article, we consider the MG as a multiagent environment with IoT devices in which AI agents exchange information with their peers for collaboration. However, the collaboration information may be lost due to communication failures or packet loss. Such events may affect the operation of the whole MG. To this end, we propose a multiagent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures. We first define a multiagent partially observable Markov decision process (MA-POMDP) to describe agents under communication failures, in which each agent can update its beliefs on the actions of its peers. Then, we apply a double deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (DDQN) architecture for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -value estimation in BA-DRL, and propose a belief-based correlated equilibrium for the joint-action selection of multiagent BA-DRL. Finally, the simulation results show that BA-DRL is robust to both power supply uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3% higher reward than Nash deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (Nash-DQN) and alternating direction method of multipliers (ADMM), respectively, under 1% communication failure probability.

Investigating Machine Learning Techniques for Predicting the Process Characteristics of Stencil Printing.

Stencil printing is the most crucial process in reflow soldering for the mass assembly of electronic circuits. This paper investigates different machine learning-based methods to predict the essential process characteristics of stencil printing: the area, thickness, and volume of deposited solder paste. The training dataset was obtained experimentally by varying the printing speed (from 20 to 120 mm/s), the size (area ratio from 0.35 to 1.7) of stencil apertures, and the particle size (characterized by a log-normal distribution) in the solder paste. Various machine learning-based methods were assessed; ANFIS–adaptive neuro-fuzzy inference systems; ANN artificial neural networks (with different learning methods); boosted trees, regression trees, SVM–support vector machines. Each method was optimized and fine-tuned with hyperparameter optimization, and the overfitting phenomenon was also prevented with cross-validation. The regression tree was the best performing approach for modelling the stencil printing, while ANN with the Bayesian regularization learning method was only slightly worse. The presented methodology for fine-tuning, parameter optimization, and the comparison of different machine learning-based methods can easily be adapted to any application field in electronics manufacturing.

An intelligent web-based spatial group decision support system to investigate the role of the opponents’ modeling in urban land use planning

Urban land-use planning decisions generally require negotiation between multiple stakeholders to reach an agreement on a specific plan. Computer-aided tools such as group decision support systems can facilitate the actors in this complicated process. In the context of these systems, using software agents enhance the effectiveness and efficiency of group decision support. The software agents can perform some computational and analytical tasks on behalf of the stakeholders. In more advanced cases, the agents can also learn stakeholders’ preferences and behavior to help them make good decisions. This paper proposes an intelligent web-based spatial group decision support system to investigate the role of opponents modeling in urban land use planning by using a multi-agent system approach. For this purpose, two successive meetings are held in which the system is used: in the first meeting, the stakeholders revise the existing plans and respond to other stakeholders’ requests. During the meeting, software agents attempt to model the behavior of the stakeholders they are associated with, based on a Bayesian learning method in combination with social value orientation theory to describe stakeholders’ decision behavior in a group context. In the second meeting, the software agents help the stakeholders in the step of plan revision by providing the information obtained to the stakeholders. In an application, a comparison of the results of the meetings showed that the provided information about the opponents reduced the negotiation time and contributed to reaching a better spatial configuration of land-uses based on a criterion provided by social value orientation theory.

Long-Term Forecasting of Strong Earthquakes in North America, South America, Japan, Southern China and Northern India With Machine Learning

Strong earthquakes (magnitude ≥7) occur worldwide affecting different cities and countries while causing great human, ecological and economic losses. The ability to forecast strong earthquakes on the long-term basis is essential to minimize the risks and vulnerabilities of people living in highly active seismic areas. We have studied seismic activities in North America, South America, Japan, Southern China and Northern India in search for patterns in strong earthquakes on each of these active seismic zones between 1900 and 2021 with the powerful mathematical tool of wavelet transform. We found that the primary seismic activity patterns for M ≥ 7 earthquakes are 55, 3.7, 7.7, and 8.6 years, for seismic zones of the southwestern United States and northern Mexico, southwestern Mexico, South American, and Southern China-Northern India, respectively. In the case of Japan, the most important seismic pattern for earthquakes with magnitude 7 ≤ M&lt; 8 is 4.1 years and for strong earthquakes with M ≥ 8, it is 40 years. Every seismic pattern obtained clusters the earthquakes in historical intervals/episodes with and without strong earthquakes in the individually analyzed seismic zones. We want to clarify that the intervals where no strong earthquakes do not imply the total absence of seismic activity because earthquakes can occur with lesser magnitude within this same interval. From the information and pattern we obtained from the wavelet analyses, we created a probabilistic, long-term earthquake prediction model for each seismic zone using the Bayesian Machine Learning method. We propose that the periods of occurrence of earthquakes in each seismic zone analyzed could be interpreted as the period in which the stress builds up on different planes of a fault, until this energy releases through the rupture along faults and fractures near the plate tectonic boundaries. Then a series of earthquakes can occur along the fault until the stress subsides and a new cycle begins. Our machine learning models predict a new period of strong earthquakes between 2040 ± 5 and 2057 ± 5, 2024 ± 1 and 2026 ± 1, 2026 ± 2 and 2031 ± 2, 2024 ± 2 and 2029 ± 2, and 2022 ± 1 and 2028 ± 2 for the five active seismic zones of United States, Mexico, South America, Japan, and Southern China and Northern India, respectively. In additon, our methodology can be applied in areas where moderate earthquakes occur, as for the case of the Parkfield section of the San Andreas fault (California, United States). Our methodology explains why a moderate earthquake could never occur in 1988 ± 5 as proposed and why the long-awaited Parkfield earthquake event occurred in 2004. Furthermore, our model predicts that possible seismic events may occur between 2019 and 2031, with a high probability of earthquake events at Parkfield around 2025 ± 2 years.