Generative Learning Model Research Articles

HAR_Locator: a novel protein subcellular location prediction model of immunohistochemistry images based on hybrid attention modules and residual units.

Introduction: Proteins located in subcellular compartments have played an indispensable role in the physiological function of eukaryotic organisms. The pattern of protein subcellular localization is conducive to understanding the mechanism and function of proteins, contributing to investigating pathological changes of cells, and providing technical support for targeted drug research on human diseases. Automated systems based on featurization or representation learning and classifier design have attracted interest in predicting the subcellular location of proteins due to a considerable rise in proteins. However, large-scale, fine-grained protein microscopic images are prone to trapping and losing feature information in the general deep learning models, and the shallow features derived from statistical methods have weak supervision abilities. Methods: In this work, a novel model called HAR_Locator was developed to predict the subcellular location of proteins by concatenating multi-view abstract features and shallow features, whose advanced advantages are summarized in the following three protocols. Firstly, to get discriminative abstract feature information on protein subcellular location, an abstract feature extractor called HARnet based on Hybrid Attention modules and Residual units was proposed to relieve gradient dispersion and focus on protein-target regions. Secondly, it not only improves the supervision ability of image information but also enhances the generalization ability of the HAR_Locator through concatenating abstract features and shallow features. Finally, a multi-category multi-classifier decision system based on an Artificial Neural Network (ANN) was introduced to obtain the final output results of samples by fitting the most representative result from five subset predictors. Results: To evaluate the model, a collection of 6,778 immunohistochemistry (IHC) images from the Human Protein Atlas (HPA) database was used to present experimental results, and the accuracy, precision, and recall evaluation indicators were significantly increased to 84.73%, 84.77%, and 84.70%, respectively, compared with baseline predictors.

Influence of school location and gender on generative learning model on secondary school students' academic achievement in chemistry

This study investigated the influence of school location and gender on the generative learning model's impact on academic achievement in Chemistry among secondary school students in the Awka Education zone, Anambra State. The study employed a quasi-experimental design and used two research questions and two hypotheses as guidance. The sample consisted of 143 senior secondary school one (SS1) Chemistry students in the zone, comprising 73 males and 110 females. From the 49 co-educational secondary schools in the zone, four were selected using a simple random technique. Two schools were assigned to urban locations (47 males and 67 females), while the other two were assigned to rural locations (26 males and 43 females) through simple random sampling by balloting. Fifty Chemistry achievement tests (CAT) were adapted from West African Examination Council (WAEC) objective past question papers. The instruments were validated by experts in the science education department and education foundation. CAT reliability was established using Kudar Richardson 20 (KR-20), which yielded a reliability coefficient of 0.87. To address the research questions, mean and standard deviation were utilized, while the analysis of covariance was employed to test the hypotheses at a significance level of 0.05. The study found that the generative learning model improved student achievement in both urban and rural schools, with a significant difference in mean achievement scores between the two groups. Specifically, the generative learning model proved more beneficial for students in urban schools, particularly for male students. Based on these findings, the study draws conclusions and provides recommendations.

The effect of the generative learning model on the student critical thinking ability in environmental conservation topic

The low level of students' critical thinking ability makes students need help understanding science concepts. The application of an interactive learning model can be a solution to this problem. This research aims to analyze the effect of the Generative learning model on the critical thinking ability of 7th graders in Environmental conservation Materials. The research method used is the Quasy Experiment method carried out in class VII junior high school SMP Mutiara Insani Tangerang. Students' critical thinking abilities are measured by a written test instrument in the form of essay questions made according to indicators of critical thinking ability that have been adjusted as many as ten questions tested for validity and reliability. The selected experimental class comes from a population that is normally distributed, has a homogeneous variance, and has the same mean. The research results show that the experimental class with the Generative Learning model and the control class with the Discovery Learning model are normally distributed. Hypothesis testing is done by t-test, and the significance level is based on the results of the Posttest; there is a significant influence of the Generative Learning learning model on the critical thinking ability of class VII students. In conclusion, the Generative Learning Learning Model affects the Critical Thinking Ability of 7th graders Students in Environmental Conservation material.

GenCo: A Generative Learning Model for Heterogeneous Text Classification Based on Collaborative Partial Classifications

The use of generative learning models in natural language processing (NLP) has significantly contributed to the advancement of natural language applications, such as sentimental analysis, topic modeling, text classification, chatbots, and spam filtering. With a large amount of text generated each day from different sources, such as web-pages, blogs, emails, social media, and articles, one of the most common tasks in NLP is the classification of a text corpus. This is important in many institutions for planning, decision-making, and creating archives of their projects. Many algorithms exist to automate text classification tasks but the most intriguing of them is that which also learns these tasks automatically. In this study, we present a new model to infer and learn from data using probabilistic logic and apply it to text classification. This model, called GenCo, is a multi-input single-output (MISO) learning model that uses a collaboration of partial classifications to generate the desired output. It provides a heterogeneity measure to explain its classification results and enables a reduction in the curse of dimensionality in text classification. Experiments with the model were carried out on the Twitter US Airline dataset, the Conference Paper dataset, and the SMS Spam dataset, outperforming baseline models with 98.40%, 89.90%, and 99.26% accuracy, respectively.

A deep learning model with spatio-temporal graph convolutional networks for river water quality prediction

Abstract High-precision water quality prediction plays a vital role in preventing and controlling river pollution. However, river water's highly nonlinear and complex spatio-temporal dependencies pose significant challenges to water quality prediction tasks. In order to capture the spatial and temporal characteristics of water quality data simultaneously, this paper combines deep learning algorithms for river water quality prediction in the river network area of Jiangnan Plain, China. A water quality prediction method based on graph convolutional network (GCN) and long short-term memory neural network (LSTM), namely spatio-temporal graph convolutional network model (ST-GCN), is proposed. Specifically, the spatio-temporal graph is constructed based on the spatio-temporal correlation between river stations, the spatial features in the river network are extracted using GCN, and the temporal correlation of water quality data is obtained by integrating LSTM. The model was evaluated using R2, MAE, and RMSE, and the experimental results were 0.977, 0.238, and 0.291, respectively. Compared with traditional water quality prediction models, the ST-GCN model has significantly improved prediction accuracy, better stability, and generalization ability.

Automatic Contrast Generation from Contrastless Computed Tomography.

The use of contrast-enhanced computed tomography (CTCA) for detection of coronary artery disease (CAD) exposes patients to the risks of iodine contrast-agents and excessive radiation, increases scanning time and healthcare costs. Deep learning generative models have the potential to artificially create a pseudo-enhanced image from non-contrast computed tomography (CT) scans.In this work, two specific models of generative adversarial networks (GANs) - the Pix2Pix-GAN and the Cycle-GAN - were tested with paired non-contrasted CT and CTCA scans from a private and public dataset. Furthermore, an exploratory analysis of the trade-off of using 2D and 3D inputs and architectures was performed. Using only the Structural Similarity Index Measure (SSIM) and the Peak Signal-to-Noise Ratio (PSNR), it could be concluded that the Pix2Pix-GAN using 2D data reached better results with 0.492 SSIM and 16.375 dB PSNR. However, visual analysis of the output shows significant blur in the generated images, which is not the case for the Cycle-GAN models. This behavior can be captured by the evaluation of the Fréchet Inception Distance (FID), that represents a fundamental performance metric that is usually not considered by related works in the literature.Clinical relevance- Contrast-enhanced computed tomography is the first line imaging modality to detect CAD resulting in unnecessary exposition to the risk of iodine contrast and radiation in particularly in young patients with no disease. This algorithm has the potential of being translated into clinical practice as a screening method for CAD in asymptomatic subjects or quick rule-out method of CAD in the acute setting or centres with no CTCA service. This strategy can eventually represent a reduction in the need for CTCA reducing its burden and associated costs.

The Effect of Reflective-Impulsive Conceptual Thinking Style Models and Learning Models on Learning Outcomes of Complex Analysis by Controlling Students' Initial Knowledge

Concept understanding of complex analysis is very necessary for students of the Mathematics Education Department. Considering the concepts contained in Mathematics course are needed by students, especially in developing reasoning power in related material and solving problems. This study aims to analyze interactions between the reflective-impulsive conceptual thinking style model and the learning model on learning outcomes of complex analysis by controlling initial knowledge. This research is a quasi-experimental research involving 92 students of the Department of Mathematics Education as a sample. The sampling technique to determine each treatment group is simple random sampling. This study uses a 2x2 factorial design. Data analysis was performed using analysis of covariance (ANCOVA). The results showed that the learning outcomes of complex analysis of groups of students who followed the generative learning model were higher than those who followed conventional learning models. The learning outcomes of complex analysis of groups of students who had a reflective thinking style were higher than those who had an impulsive style. There was an influence of interaction between the learning model and conceptual thinking style on student complex analysis learning outcomes. Generative model learning has implications for increasing mastery of concepts, which are closely related to initial abilities.

A Simple Way to Incorporate Target Structural Information in Molecular Generative Models.

Deep learning generative models are now being applied in various fields including drug discovery. In this work, we propose a novel approach to include target 3D structural information in molecular generative models for structure-based drug design. The method combines a message-passing neural network model that predicts docking scores with a generative neural network model as its reward function to navigate the chemical space searching for molecules that bind favorably with a specific target. A key feature of the method is the construction of target-specific molecular sets for training, designed to overcome potential transferability issues of surrogate docking models through a two-round training process. Consequently, this enables accurate guided exploration of the chemical space without reliance on the collection of prior knowledge about active and inactive compounds for the specific target. Tests on eight target proteins showed a 100-fold increase in hit generation compared to conventional docking calculations and the ability to generate molecules similar to approved drugs or known active ligands for specific targets without prior knowledge. This method provides a general and highly efficient solution for structure-based molecular generation.

AI-Driven De Novo Design and Molecular Modeling for Discovery of Small-Molecule Compounds as Potential Drug Candidates Targeting SARS-CoV-2 Main Protease.

Over the past three years, significant progress has been made in the development of novel promising drug candidates against COVID-19. However, SARS-CoV-2 mutations resulting in the emergence of new viral strains that can be resistant to the drugs used currently in the clinic necessitate the development of novel potent and broad therapeutic agents targeting different vulnerable spots of the viral proteins. In this study, two deep learning generative models were developed and used in combination with molecular modeling tools for de novo design of small molecule compounds that can inhibit the catalytic activity of SARS-CoV-2 main protease (Mpro), an enzyme critically important for mediating viral replication and transcription. As a result, the seven best scoring compounds that exhibited low values of binding free energy comparable with those calculated for two potent inhibitors of Mpro, via the same computational protocol, were selected as the most probable inhibitors of the enzyme catalytic site. In light of the data obtained, the identified compounds are assumed to present promising scaffolds for the development of new potent and broad-spectrum drugs inhibiting SARS-CoV-2 Mpro, an attractive therapeutic target for anti-COVID-19 agents.

Lifelong Learning Architecture of Video Surveillance System

The learning capacity of general deep learning models for object detection would not be large enough to represent real-world scene dynamics, and thus such models would be weak to ‘unseen’ data due to environmental changes. To address this issue, online or active learning methods use data samples obtained in new environments, where the new samples collected from false and/or miss detection cases are used to re-train the original model to enhance detection precision. However, it is inevitably degraded over time due to the catastrophic forgetting problem, that is a well-known intrinsic problem of current deep learning technologies. In this study, we propose a cutting-edge end-to-end system architecture to continuously improve the accuracy of the video analytic algorithms such as object detection with less accuracy degradation, by utilizing a hybrid combination of intelligence both the front-end and back-end systems. We use an iterative process where the current model is self-evolving using new incoming data as part of an ongoing adaptation process. We carried out several experiments of person detection in surveillance videos with various challenging environmental changes and showed the high precision and adaptability of our new architecture while it can be practically implemented at a low cost.

Super Intendo: Semantic Robot Programming from Multiple Demonstrations for taskable robots

When an end-user instructs a taskable robot on a new task, it is important for the robot to learn the user’s intention for the task. Knowing the user’s intention, represented as desired goal conditions, allows the robot to generalize across variations of the learned task seen at execution time. However, it has proven challenging to learn goal conditions due to the large, noisy, and complex space of goal conditions expressed by human users. This paper introduces Semantic Robot Programming with Multiple Demonstrations (SRP-MD) to learn a generative model of latent end-user task goal conditions from multiple end-user demonstrations in a shared workspace. By learning a generative model of the goal conditions, SRP-MD generalizes to task instances even when the quantity of objects to be arranged is not in the training set or novel object instances are included. At test time, a new goal is pulled from the learned generative model given the objects present in the initial scene. The efficacy of SRP-MD as a step toward taskable robots is shown on a Fetch robot learning and executing bin packing tasks in a simulated environment with grocery items.

Conditional Variational AutoEncoder based on Stochastic Attacks

Over the recent years, the cryptanalysis community leveraged the potential of research on Deep Learning to enhance attacks. In particular, several studies have recently highlighted the benefits of Deep Learning based Side-Channel Attacks (DLSCA) to target real-world cryptographic implementations. While this new research area on applied cryptography provides impressive result to recover a secret key even when countermeasures are implemented (e.g. desynchronization, masking schemes), the lack of theoretical results make the construction of appropriate and powerful models a notoriously hard problem. This can be problematic during an evaluation process where a security bound is required. In this work, we propose the first solution that bridges DL and SCA in order to get this security bound. Based on theoretical results, we develop the first Machine Learning generative model, called Conditional Variational AutoEncoder based on Stochastic Attacks (cVAE-SA), designed from the well-known Stochastic Attacks, that have been introduced by Schindler et al. in 2005. This model reduces the black-box property of DL and eases the architecture design for every real-world crypto-system as we define theoretical complexity bounds which only depend on the dimension of the (reduced) trace and the targeting variable over F2n . We validate our theoretical proposition through simulations and public datasets on a wide range of use cases, including multi-task learning, curse of dimensionality and masking scheme.

Interpretable Hybrid Experimental Learning for Trading Behavior Modeling in Electricity Market

A modern electricity market is essentially a complex network, characterized by complicated interactions among cyber communications, physical systems, and social agents. Trading behavior modeling has always been complicated in the physical system-based market. In this paper, trading behavior modeling in the electricity market is solved by a data-driven method combining experimental economics and machine learning, called Hybrid Experimental Learning (HEL). Based on the historical and experiment simulated data, HEL models the trading behavior by a machine learning generative model which will be interpreted by a post hoc interpretation approach. Taking a simulated electricity market based on the trial spot market rule in Guangdong, China as an example, a generative adversarial network (GAN) is employed to generate the offering strategies of a gas generator. Local interpretable model-agnostic explanation (LIME) as a post hoc interpretation approach is applied to explain the relationship between the output of GAN and some of the inputs of HEL, which can be described as offering mechanisms for the gas generator.

Computer-aided diagnosis of schizophrenia based on node2vec and Transformer

Objective:Compared with the healthy control (HC) group, the brain structure and function of schizophrenia (SZ) patients are significantly abnormal, so brain imaging methods can be used to achieve the aided diagnosis of SZ. However, a brain network based on brain imaging data is non-Euclidean, and its intrinsic features cannot be learned effectively by general deep learning models. Furthermore, in the majority of existing studies, brain network features were manually specified as the input of machine learning models. Methods:In this study, brain functional network constructed from the subject’s fMRI data is analyzed, and its small-world value is calculated and t-tested; the node2vec algorithm in graph embedding is introduced to transform the constructed brain network into low-dimensional dense vectors, and the brain network’s non-Euclidean spatial structure characteristics are retained to the greatest extent, so that its intrinsic features can be extracted by deep learning models; GridMask is used to randomly mask part of the information in the vectors to enhance the data; and then features can be extracted using the Transformer model to identify SZ. Results:It is again shown that the small-world value of the brain network in SZ is significantly lower than that in HC by t-test (p=0.014¡0.05). 97.78% classification accuracy is achieved by the proposed methods (node2vec + GridMask + Transformer) in 30 SZ patients and 30 healthy people. Conclusion:The experiment shows that the node2vec used in this paper can effectively solve the problem of brain network features being difficult to learn by general deep learning models. The high-precision computer-aided diagnosis of SZ can be obtained by combining node2vec with Transformer and GridMask. Significance:The proposed methods in the paper are expected to be used for aided diagnosis of SZ.

Unsupervised anomaly detection with generative adversarial networks in mammography

Breast cancer is a common cancer among women, and screening mammography is the primary tool for diagnosing this condition. Recent advancements in deep-learning technologies have triggered the implementation of research studies via mammography. Semi-supervised or unsupervised methods are often used to overcome the limitations of supervised learning, such as manpower and time, for labeling in clinical situations where abnormal data are significantly lacking. Accordingly, we proposed a generative model that uses a state-of-the-art generative network (StyleGAN2) to create high-quality synthetic mammographic images and an anomaly detection method to detect breast cancer on mammograms in unsupervised methods. The generation model was trained via only normal mammograms and breast cancer classification was performed via anomaly detection using 50 breast cancer and 50 normal mammograms that did not overlap with the dataset for generative model learning. Our generative model has shown comparable fidelity to real images, and the anomaly detection method via this generative model showed high sensitivity, demonstrating its potential for breast cancer screening. This method could differentiate between normal and cancer-positive mammogram and help overcome the weakness of current supervised methods.

Developing Students' Mathematical Communication Skill in Junior High School with Various Level of Mathematics Achievement through Generative Learning Model

Mathematical communication skill was the missing piece of the mathematics education failure puzzle for centuries despite its central role in learning activities. Built on top of other process skills, the skill was the tool to construct a more comprehensive conceptual understanding by learning math from different perspectives, sharpening other cognitive skills, and providing important feedback about students' understanding. This study observed how the students' mathematical communication skill was correlated with the application of the generative learning model and how this correlation was influenced by various level of the school's academic achievement in mathematics. It employed a fractional 2x2-factorial design, careful non-probability sampling combined with simple random sampling to pick 171 students, and a validated and reliable scoring system to measure communication skills. Based on the results, the students' mathematical communication skill significantly correlates with the application of the generative learning model. The use of factorial design revealed that this correlation was more determined by applying the generative learning model, not by the level of schools' or students' mathematics achievement. Teachers should be more optimistic about using generative learning models to improve mathematical communication skills, even in classes with lower mathematics achievement levels.

Funneling modulatory peptide design with generative models: Discovery and characterization of disruptors of calcineurin protein-protein interactions.

Design of peptide binders is an attractive strategy for targeting "undruggable" protein-protein interfaces. Current design protocols rely on the extraction of an initial sequence from one known protein interactor of the target protein, followed by in-silico or in-vitro mutagenesis-based optimization of its binding affinity. Wet lab protocols can explore only a minor portion of the vast sequence space and cannot efficiently screen for other desirable properties such as high specificity and low toxicity, while in-silico design requires intensive computational resources and often relies on simplified binding models. Yet, for a multivalent protein target, dozens to hundreds of natural protein partners already exist in the cellular environment. Here, we describe a peptide design protocol that harnesses this diversity via a machine learning generative model. After identifying putative natural binding fragments by literature and homology search, a compositional Restricted Boltzmann Machine is trained and sampled to yield hundreds of diverse candidate peptides. The latter are further filtered via flexible molecular docking and an in-vitro microchip-based binding assay. We validate and test our protocol on calcineurin, a calcium-dependent protein phosphatase involved in various cellular pathways in health and disease. In a single screening round, we identified multiple 16-length peptides with up to six mutations from their closest natural sequence that successfully interfere with the binding of calcineurin to its substrates. In summary, integrating protein interaction and sequence databases, generative modeling, molecular docking and interaction assays enables the discovery of novel protein-protein interaction modulators.

A Deep Learning Approach to Visualize Aortic Aneurysm Morphology Without the Use of Intravenous Contrast Agents.

Intravenous contrast agents are routinely used in CT imaging to enable the visualization of intravascular pathology, such as with abdominal aortic aneurysms. However, the injection is contraindicated in patients with iodine allergy and is associated with renal complications. In this study, we investigate if the raw data acquired from a noncontrast CT image contains sufficient information to differentiate blood and other soft tissue components. A deep learning pipeline underpinned by generative adversarial networks was developed to simulate contrast enhanced CTA images using noncontrast CTs. Two generative models (cycle- and conditional) are trained with paired noncontrast and contrast enhanced CTs from seventy-five patients (total of 11,243 pairs of images) with abdominal aortic aneurysms in a 3-fold cross-validation approach with a training/testing split of 50:25 patients. Subsequently, models were evaluated on an independent validation cohort of 200 patients (total of 29,468 pairs of images). Both deep learning generative models are able to perform this image transformation task with the Cycle-generative adversarial network (GAN) model outperforming the Conditional-GAN model as measured by aneurysm lumen segmentation accuracy (Cycle-GAN: 86.1% ± 12.2% vs Con-GAN: 85.7% ± 10.4%) and thrombus spatial morphology classification accuracy (Cycle-GAN: 93.5% vs Con-GAN: 85.7%). This pipeline implements deep learning methods to generate CTAs from noncontrast images, without the need of contrast injection, that bear strong concordance to the ground truth and enable the assessment ofimportant clinical metrics. Our pipeline is poised to disrupt clinical pathways requiring intravenous contrast.

Analyzing drop coalescence in microfluidic devices with a deep learning generative model.

Predicting drop coalescence based on process parameters is crucial for experimental design in chemical engineering. However, predictive models can suffer from the lack of training data and more importantly, the label imbalance problem. In this study, we propose the use of deep learning generative models to tackle this bottleneck by training the predictive models using generated synthetic data. A novel generative model, named double space conditional variational autoencoder (DSCVAE) is developed for labelled tabular data. By introducing label constraints in both the latent and the original space, DSCVAE is capable of generating consistent and realistic samples compared to the standard conditional variational autoencoder (CVAE). Two predictive models, namely random forest and gradient boosting classifiers, are enhanced on synthetic data and their performances are evaluated based on real experimental data. Numerical results show that a considerable improvement in prediction accuracy can be achieved by using synthetic data and the proposed DSCVAE clearly outperforms the standard CVAE. This research clearly provides more insights into handling imbalanced data for classification problems, especially in chemical engineering.

Deep Generative Learning Models for Cloud Intrusion Detection Systems.

Intrusion detection (ID) on the cloud environment has received paramount interest over the last few years. Among the latest approaches, machine learning-based ID methods allow us to discover unknown attacks. However, due to the lack of malicious samples and the rapid evolution of diverse attacks, constructing a cloud ID system (IDS) that is robust to a wide range of unknown attacks remains challenging. In this article, we propose a novel solution to enable robust cloud IDSs using deep neural networks. Specifically, we develop two deep generative models to synthesize malicious samples on the cloud systems. The first model, conditional denoising adversarial autoencoder (CDAAE), is used to generate specific types of malicious samples. The second model (CDAEE-KNN) is a hybrid of CDAAE and the K -nearest neighbor algorithm to generate malicious borderline samples that further improve the accuracy of a cloud IDS. The synthesized samples are merged with the original samples to form the augmented datasets. Three machine learning algorithms are trained on the augmented datasets and their effectiveness is analyzed. The experiments conducted on four popular IDS datasets show that our proposed techniques significantly improve the accuracy of the cloud IDSs compared with the baseline technique and the state-of-the-art approaches. Moreover, our models also enhance the accuracy of machine learning algorithms in detecting some currently challenging distributed denial of service (DDoS) attacks, including low-rate DDoS attacks and application layer DDoS attacks.