Parallel Learning Research Articles

PLIC-FSR-SYS: System reliability analysis based on parallel learning of influential components with filtered sample region

In practical engineering, system reliability analysis is highly concerned since many structures or products have multiple failure modes. Accordingly, this paper develops an innovative method for system reliability analysis by parallel learning of influential component limit-state functions with filtered sample region (PLIC-FSR-SYS) based on Kriging modeling. Different from the traditional adaptive learning methods that train only one component in each iteration when constructing the surrogate of the composite limit-state function, a new strategy is explored to adaptively identify several important components in one iteration so as to train them simultaneously. In the meanwhile, a filtering formula is explored to determine the fatal region so that the unimportant samples can be removed to further accelerate the training process. Based on the join forces of parallel learning of influential components and avoiding the training at unimportant samples, PLIC-FSR-SYS can achieve a fairly efficient system reliability analysis with multiple failure modes. Finally, four different case studies, including an engineering application to the ultra-voltage on-load tap-changer, are conducted to prove the effectiveness of the proposed method. The results indicate that compared to traditional adaptive learning methods, the proposed method makes a significant efficiency improvement for system reliability analysis with multiple failure modes.

Multivariate quality prediction of thin-walled parts machining using multi-task parallel deep transfer learning

Multivariate machining quality prediction of thin-walled parts with multiple machining features is a complex problem due to different data distribution between training and unlabelled test samples. Traditional quality prediction methods ignore the correlation of multiple quality labels and do not consider changes in data distribution, resulting in low accuracy of multivariate quality prediction. Therefore, a multivariate quality prediction method using multi-task parallel deep transfer learning is proposed to solve this problem. Specifically, a multi-output quality prediction model of cross-machining features is constructed through the joint design of multi-task parallel learning and deep transfer learning. Furthermore, a domain matcher is designed to form multiple transfer strategies, which can be used for dynamic matching of multi-source and multi-target machining features with multiple quality labels. The domain invariant data features through dynamic domain adaptation are extracted to deal with data distribution discrepancy between the source and target domains. Finally, the results of multiple comparison experiments show that the proposed method can effectively achieve the accurate quality prediction of the target domain with unlabelled labels and different distributions. Compared with the traditional methods, the proposed method has improved by 8.34%, 7.14%, and 9.09%, respectively, in MAE, RMSE, and score on average.

MRSAPose: Multi-level routing sparse attention for multi-person pose estimation

Multi-person Pose Estimation (MPPE) aims to reconstruct human poses by locating and connecting keypoints of individuals in input images. The variability of human poses and the complexity of scenes make MPPE reliant on both local details and global structures, and the absence of either can lead to the generation of deformed poses. With the emergence of Transformer, the performance of MPPE has been significantly improved. However, due to self-attention computing attention scores between each pair of positions, the current Transformer-based MPPE exhibits high quadratic complexity. To address these issues, this paper proposes a novel pose estimation model, MRSAPose. MRSAPose utilizes Multi-level Routing Sparse Attention (MRSA) to dynamically select relevant regions for attention, reducing computational complexity and mitigating the impact of irrelevant regions. Furthermore, MRSAPose constructs a Transformer-CNN Parallel Interaction Block (T-CP block) through MRSA and Recursive Residual Gated Convolution (Res-gnConv), facilitating parallel learning of global and local information. By relying on multi-level routing algorithms and high-order spatial interactions conducted by recursive processing of adjacent features, T-CP block helps MRSAPose effectively alleviates the issues of occlusion and misalignment in pose estimation. On multiple challenging keypoint datasets, MRSAPose outperforms current state-of-the-art algorithms, particularly excelling in crowded and occluded scenes.

Leader, Lead Thyself: The Benefit of Contract Grading in Social Work Leadership Education

ABSTRACT Contract grading provides graduate-level students with greater control and autonomy within the learning process. The contract grading approach requires students to take greater responsibility for achieving course learning outcomes as well as allowing learners to adjust the style and focus of assignments to meet their own unique needs and desires. Within this article, contract grading is applied to social work leadership education and explored as a parallel learning process designed to reflect some of the fundamentals of leadership in action while also teaching about leadership. A review of the scholarship of contract grading, an example of the application of the method, and a reflection on potential challenges that may result are also included in the article.

A bio-inspired reinforcement learning model that accounts for fast adaptation after punishment

Humans and animals can quickly learn a new strategy when a previously-rewarding strategy is punished. It is difficult to model this with reinforcement learning methods, because they tend to perseverate on previously-learned strategies − a hallmark of impaired response to punishment. Past work has addressed this by augmenting conventional reinforcement learning equations with ad hoc parameters or parallel learning systems. This produces reinforcement learning models that account for reversal learning, but are more abstract, complex, and somewhat detached from neural substrates. Here we use a different approach: we generalize a recently-discovered neuron-level learning rule, on the assumption that it captures a basic principle of learning that may occur at the whole-brain-level. Surprisingly, this gives a new reinforcement learning rule that accounts for adaptation and lose-shift behavior, and uses only the same parameters as conventional reinforcement learning equations. In the new rule, the normal reward prediction errors that drive reinforcement learning are scaled by the likelihood the agent assigns to the action that triggered a reward or punishment. The new rule demonstrates quick adaptation in card sorting and variable Iowa gambling tasks, and also exhibits a human-like paradox-of-choice effect. It will be useful for experimental researchers modeling learning and behavior.

Dopamine-mediated interactions between short- and long-term memory dynamics.

In dynamic environments, animals make behavioural decisions on the basis of the innate valences of sensory cues and information learnt about these cues across multiple timescales1-3. However, it remains unclear how the innate valence of a sensory stimulus affects the acquisition of learnt valence information and subsequent memory dynamics. Here we show that in the Drosophila brain, interconnected short- and long-term memory units of the mushroom body jointly regulate memory through dopamine signals that encode innate and learnt sensory valences. By performing time-lapse in vivo voltage-imaging studies of neural spiking in more than 500 flies undergoing olfactory associative conditioning, we found that protocerebral posterior lateral 1 dopamine neurons (PPL1-DANs)4 heterogeneously and bidirectionally encode innate and learnt valences of punishment, reward and odour cues. During learning, these valence signals regulate memory storage and extinction in mushroom body output neurons (MBONs)5. During initial conditioning bouts, PPL1-γ1pedc and PPL1-γ2α'1 neurons control short-term memory formation, which weakens inhibitory feedback from MBON-γ1pedc>α/β to PPL1-α'2α2 and PPL1-α3. During further conditioning, this diminished feedback allows these two PPL1-DANs to encode the net innate plus learnt valence of the conditioned odour cue, which gates long-term memory formation. A computational model constrained by the fly connectome6,7 and our spiking data explains how dopamine signals mediate the circuit interactions between short- and long-term memory traces, yielding predictions that our experiments confirmed. Overall, the mushroom body achieves flexible learning through the integration of innate and learnt valences in parallel learning units sharing feedback interconnections. This hybrid physiological-anatomical mechanism may be a general means by which dopamine regulates memory dynamics in other species and brain structures, including the vertebrate basal ganglia.

Parallel diffusion models promote high detail-fidelity photoacoustic microscopy in sparse sampling

Reconstructing sparsely sampled data is fundamental for achieving high spatiotemporal resolution photoacoustic microscopy (PAM) of microvascular morphology in vivo. Convolutional networks (CNN) and generative adversarial networks (GAN) have been introduced to high-speed PAM, but due to the use of upsampling in CNN-based networks to restore details and the instability in GAN training, they struggle to learn the entangled microvascular network structure and vascular texture features, resulting in only achieving low detail-fidelity imaging of microvascular. The diffusion models is richly sampled and can generate high-quality images, which is very helpful for the complex vascular features in PAM. Here, we propose an approach named parallel diffusion models (PDM) with parallel learning of Noise task and Image task, where the Noise task optimizes through variational lower bounds to generate microvascular structures that are visually realistic, and the Image task improves the fidelity of the generated microvascular details through image-based loss. With only 1.56% of fully sampled pixels from photoacoustic human oral data, PDM achieves an LPIPS of 0.199. Additionally, using PDM in high-speed 16x PAM prevents breathing artifacts and image distortion issues caused by low-speed sampling, reduces the standard deviation of the Row-wise Self-Correlation Coefficient, and maintains high image quality. It achieves high confidence in reconstructing detailed information from sparsely sampled data and will promote the application of reconstructed sparsely sampled data in realizing high spatiotemporal resolution PAM.

A hybrid spatiotemporal distribution forecast methodology for IES vulnerabilities under uncertain and imprecise space-air-ground monitoring data scenarios

The weak spots in an integrated energy system that may jeopardize the overall reliability call for timely and efficient Inspection and Maintenance (I&M). One core step is the reasonable allocation and deployment of limited I&M personnel or apparatus to the regions or periods with higher event risks, which requires a pinpoint spatiotemporal distribution forecast of future vulnerabilities. This paper presents a hybrid forecast methodology, the Saliency-Rough Fuzzy Utility Pattern recognition ensemble, in light of space-air-ground multi-source-heterogeneous input data. A parallel learning architecture is established and identifies the critical components with higher yields to enhance efficiency. Accordingly, more reasonable quantitative and qualitative evaluations can be carried out concurrently. Potential imprecise and uncertain data scenes are handled in quantitative assessments, both the failure hazard path sets and survival function likelihood boxes are incorporated in the designed relative path-Fussell Vesely Saliency (rp-FVS) model; and in qualitative analyses, the underlying perilous components can be distinguished via a combination of the variable precision-rough model. The rp-FVS-based fuzzy inference logic configures all membership functions identically according to components’ impacts. These two parts are integrated into the rough-fuzzy Utility Measure to discover concealed component-vulnerability interconnection patterns. Finally, an empirical case study is conducted for validation.

Element-wise parallel deep learning for structural distributed damage diagnosis by leveraging physical properties of long-gauge static strain transmissibility under moving loads

The Transmissibility Function (TF) has gained considerable interest in structural damage detection because of its relatively high sensitivity to damage and robustness to excitation. This study proposed a distributed damage diagnosis method for beam-like structures based on a long-gauge static strain TF defined as the ratio of the Fourier transform of static strain response under moving loads from a target long-gauge sensor to that from the reference sensor. It has been discovered that each long-gauge static strain TF is independent of moving loads, making it an ideal damage indicator. It exhibits direct proportionality to the ratio of bending stiffness between the two measured zones covered by the target and reference long-gauge strain sensors while being unaffected by the stiffness of any other zones not covered by these two sensors. Moreover, the TF at zero frequency was proved to be equivalent to the ratio of static strain time history areas, relying solely on the stiffness of the covered zones. Leveraged by the physical properties of long-gauge static strain TF, an element-wise parallel deep neural network architecture was developed to decompose damage diagnosis into independent sub-tasks on the element level, utilizing a variational autoencoder (VAE) in the Bayesian inference framework for extracting features from the long-gauge strain TF and a regressor for mapping the features to elemental stiffness reduction. The divide-and-conquer strategy and element-wise parallel learning architecture allow for a significant reduction in the number of labeled training samples generated based on the updated numerical baseline model as it only requires simulated scenarios involving a single damage element. The efficiency and robustness of the proposed method were demonstrated through a numerical simply supported beam and a laboratory experiment on a two-span bridge.

Interval type-2 fuzzy stochastic configuration networks for soft sensor modeling of industrial processes

Soft sensors have been widely applied to predict key variables that are difficult to measure for industrial process modeling. In this paper, a novel randomized interval type-2 fuzzy neural network with parallel learning, called IT2F-PSCN, is presented for soft sensor modeling of industrial processes. It trains the upper and lower bounds of the interval type-2 fuzzy logic system separately to facilitate type reduction, thereby integrating the fuzzy logic system with stochastic configuration networks. To achieve the appropriate structure and parameters of the model, we develop a two-phase training scheme. In the first phase, a sparse rule interpolation method with stochastic configuration is applied to generate new fuzzy rules. In the second phase, the hidden layer is constructed through parallel stochastic configuration to enhance the nonlinear representational capacity. The validity of IT2F-PSCN is confirmed by a series of experiments, including four benchmark data modelings, simulation on the Tennessee Eastman process, and soft sensor modeling for the slurry grade of the first rougher in a zinc flotation process. The experimental results indicate that the proposed IT2F-PSCN performs favorably compared with other methods.

Expected lifetime prediction for time- and space-dependent structural systems based on active learning surrogate model

Predicting the expected lifetime of structural systems is fundamental for effective design and maintenance. However, existing methods based on physics overlook critical spatial considerations, particularly those related to random field inputs. In this study, we propose a new active learning surrogate-based method to predict the expected lifetime of structural systems affected by both time and space factors. The method starts by training an initial Kriging model with random samples and time-space coordinates. An extended expected feasibility function is proposed to select new training samples and their corresponding time-space coordinates, refining the Kriging model. A novel parallel learning strategy is proposed to expedite computations. By using the constructed Kriging model and the first failure time of random samples, we can predict the expected lifetime. The proposed method is adaptable to structural systems with multiple failure modes, implicit functions, and random field variations. The efficiency and accuracy of the proposed method are validated through four examples.

The Stories Behind Students’ Dropping Out: The Case Of The Philippine Alternative Learning System

Education for all has been a campaign of education advocates over the years. But despite having various education initiatives and programs that promote inclusivity, there are still learners who have left the traditional teaching and learning environment. In the Philippines, the Alternative Learning System (ALS), which is considered to be a parallel learning program, serves as a bridge for those who are unable to attend the formal education system due to various circumstances. Hence, this paper aims to deeply understand the reasons behind students’ dropping out of formal education. Additionally, using a qualitative case study research design, this attempts to examine the authentic experiences of out-of-school youths and adults that led them leave the four corners of classroom. Interestingly, the results of this study revealed that ALS students left the formal education system due to problems with family, personal conflicts, financial concerns, and accessibility concerns. Nevertheless, the narratives shared by the participants served as a way of deeply understanding the root cause that impedes their learning journey as they navigate the path of alternative education. Keywords: Adult education, Alternative Learning System, Education for all, Inclusive education, Out-of-school youths and adults

RQ-OSPTrans: A Semantic Classification Method Based on Transformer That Combines Overall Semantic Perception and “Repeated Questioning” Learning Mechanism

The pre-trained language model based on Transformers possesses exceptional general text-understanding capabilities, empowering it to adeptly manage a variety of tasks. However, the topic classification ability of the pre-trained language model will be seriously affected in the face of long colloquial texts, expressions with similar semantics but completely different expressions, and text errors caused by partial speech recognition. We propose a long-text topic classification method called RQ-OSPTrans to effectively address these challenges. To this end, two parallel learning modules are proposed to learn long texts, namely, the repeat question module and the overall semantic perception module. The overall semantic perception module will conduct average pooling on the semantic embeddings produced by BERT, in addition to multi-layer perceptron learning. The repeat question module will learn the text-embedding matrix, extracting detailed clues for classification based on words as fundamental elements. Comprehensive experiments demonstrate that RQ-OSPTrans can achieve a generalization performance of 98.5% on the Chinese dataset THUCNews. Moreover, RQ-OSPTrans can achieve state-of-the-art performance on the arXiv-10 dataset (84.4%) and has a comparable performance with other state-of-the-art pre-trained models on the AG’s News dataset. Finally, the results indicate that our method exhibits a superior performance compared with the baseline methods on small-scale domain-specific datasets by validating RQ-OSPTrans on a specific task scenario by using our custom-built dataset CCIPC.

A homotopy-based reinforcement learning scheme to optimal control for Markov switched interconnected systems

ABSTRACT In this paper, a homotopy-based reinforcement learning optimal control method is developed for Markov switched interconnected systems with unknown system dynamics. By utilising the subsystem decomposition method and parallel learning control method, the solution of the game coupled algebraic Riccati equations with jumping parameters is approximated. To dispense with the requirement of initial stability, a homotopy-based policy iteration is introduced, which can place unstable poles into a stable plane. In this regard, a model-free reinforcement learning method is presented to design the optimal controller for Markov switched interconnected systems. Finally, the effectiveness of the proposed method is verified by a numerical example and a practical example.

Stock Trend Prediction and Analysis Based on Machine Learning

Stock trend prediction is a critically important activity in the financial sector. Investors aim to gain a better understanding of market dynamics, make wiser investment decisions, and manage investment risks more effectively by forecasting stock price trends. This is the focus of our upcoming research. This paper aims to utilize machine learning methods such as Support Vector Machines (SVM), based on spatial interval maximization strategies, Random Forest, based on multi-decision tree parallel learning strategies, and eXtreme Gradient Boosting (XGBoost), based on classification functions, to devise accurate classification function strategies for predicting and analyzing fluctuations in Dow Jones stock prices. Principal Component Analysis (PCA) will then be used to select some suitable features, identifying key factors with the greatest impact and minimal inter-correlation related to stock prices. Through a combination of various classification models, the research aims to forecast the future rise and fall of Dow Jones stocks. It has been observed that performing feature curation through PCA in advance enhances the predictive accuracy of different machine learning methods. Notably, the combination of PCA and XGBoost achieves the highest predictive rate. This approach provides a robust and scientifically grounded method for future stock price predictions and the formulation of trading strategies.

Parallel Learning for Legal Intelligence: A HANOI Approach Based on Unified Prompting

Parallel Learning for Legal Intelligence: A HANOI Approach Based on Unified Prompting

Simulated operant reflex conditioning environment reveals effects of feedback parameters.

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

A Siamese ResNeXt network for predicting carotid intimal thickness of patients with T2DM from fundus images.

To develop and validate an artificial intelligence diagnostic model based on fundus images for predicting Carotid Intima-Media Thickness (CIMT) in individuals with Type 2 Diabetes Mellitus (T2DM). In total, 1236 patients with T2DM who had both retinal fundus images and CIMT ultrasound records within a single hospital stay were enrolled. Data were divided into normal and thickened groups and sent to eight deep learning models: convolutional neural networks of the eight models were all based on ResNet or ResNeXt. Their encoder and decoder modes are different, including the standard mode, the Parallel learning mode, and the Siamese mode. Except for the six unimodal networks, two multimodal networks based on ResNeXt under the Parallel learning mode or the Siamese mode were embedded with ages. Performance of eight models were compared via the confusion matrix, precision, recall, specificity, F1 value, and ROC curve, and recall was regarded as the main indicator. Besides, Grad-CAM was used to visualize the decisions made by Siamese ResNeXt network, which is the best performance. Performance of various models demonstrated the following points: 1) the RexNeXt showed a notable improvement over the ResNet; 2) the structural Siamese networks, which extracted features parallelly and independently, exhibited slight performance enhancements compared to the traditional networks. Notably, the Siamese networks resulted in significant improvements; 3) the performance of classification declined if the age factor was embedded in the network. Taken together, the Siamese ResNeXt unimodal model performed best for its superior efficacy and robustness. This model achieved a recall rate of 88.0% and an AUC value of 90.88% in the validation subset. Additionally, heatmaps calculated by the Grad-CAM algorithm presented concentrated and orderly mappings around the optic disc vascular area in normal CIMT groups and dispersed, irregular patterns in thickened CIMT groups. We provided a Siamese ResNeXt neural network for predicting the carotid intimal thickness of patients with T2DM from fundus images and confirmed the correlation between fundus microvascular lesions and CIMT.

An Efficient Checkpoint Strategy for Federated Learning on Heterogeneous Fault-Prone Nodes

Federated learning (FL) is a distributed machine learning method in which client nodes train deep neural network models locally using their own training data and then send that trained model to a server, which then aggregates all of the trained models into a globally trained model. This protects personal information while enabling machine learning with vast amounts of data through parallel learning. Nodes that train local models are typically mobile or edge devices from which data can be easily obtained. These devices typically run on batteries and use wireless communication, which limits their power, making their computing performance and reliability significantly lower than that of high-performance computing servers. Therefore, training takes a long time, and if something goes wrong, the client may have to start training again from the beginning. If this happens frequently, the training of the global model may slow down and the final performance may deteriorate. In a general computing system, a checkpointing method can be used to solve this problem, but applying an existing checkpointing method to FL may result in excessive overheads. This paper proposes a new FL method for situations with many fault-prone nodes that efficiently utilizes checkpoints.

Data-Driven Compositional Optimization in Misspecified Regimes

With a manifold growth in the scale and intricacy of systems, the challenges of parametric misspecification become pronounced. These concerns are further exacerbated in compositional settings, which emerge in problems complicated by modeling risk and robustness. In “Data-Driven Compositional Optimization in Misspecified Regimes,” the authors consider the resolution of compositional stochastic optimization problems, plagued by parametric misspecification. In considering settings where such misspecification may be resolved via a parallel learning process, the authors develop schemes that can contend with diverse forms of risk, dynamics, and nonconvexity. They provide asymptotic and rate guarantees for unaccelerated and accelerated schemes for convex, strongly convex, and nonconvex problems in a two-level regime with extensions to the multilevel setting. Surprisingly, the nonasymptotic rate guarantees show no degradation from the rate statements obtained in a correctly specified regime and the schemes achieve optimal (or near-optimal) sample complexities for general T-level strongly convex and nonconvex compositional problems.