Generative Learning Model Research Articles

DeepPlacer: A custom integrated OpAmp placement tool using deep models

Mechanisms towards the automatic analog integrated circuit layout design have been an intensive research topic in the past few decades. Still, the industrial environment has no automatic approach established. The advances of machine learning applications in electronic design automation come with the promise to change this reality. This paper proposes a deep learning generative model for the placement “optimization” of analog integrated circuit basic blocks. The model behaves as an argmin operator for the placement cost function and can provide placement solutions instantly. Moreover, the model can be fed with unlabeled data, greatly facilitating data collection. A generic and innovative circuits’ representation at the network’s input layer is proposed, encoding the devices’ dimensions, connectivity, and topological constraints. Besides, the randomness found in generative models is embedded directly into the feature vector, as the order of the features per device is shuffled in the input vector. Shuffling the order of the devices’ features in the input not only brings multi-modality but also solves a generalization problem, as there is not any natural order defined to place devices in the feature vector. As a proof of concept, a deep artificial neural network capable of proposing different placement solutions, in less than 150 ms each, for six amplifier topologies and, in multiple technology nodes ranging from 350 nm down to 65 nm, is demonstrated. DeepPlacer was capable of producing correct solutions for topologies and technology nodes not present in the training set, showing good generalization while not hindering circuit performance due to the placement.

A fast neighborhood search scheme for identical parallel machine scheduling problems under general learning curves

There is a limited number of results concerning fast and efficient solution algorithms for scheduling problems with general learning models. Therefore, we develop a fast insert neighborhood search (FINS) for a group of identical parallel machine scheduling problems with general learning curves under the following minimization objectives: the maximum completion time (makespan) and the maximum lateness. Our fast search scheme allows to calculate criterion values in a constant time per solution in a neighborhood. The application is presented on the basis of Nawaz–Enscore–Ham’s method (NEH), iterative local search, tabu search, particle swarm optimization and hybrid simulated annealing — tabu search algorithms, where the efficiency of their standard implementations are compared to their versions enhanced by our fast insert neighborhood search. The computational experiments confirm the theoretical analysis that our method essentially overwhelms the well known approaches. The metaheuristics augmented by our FINS are about 120 times faster than their standard versions and improve the criterion values by up to 25% for analyzed instances with 100–800 jobs and 5–80 machines.

Feasibility of Physics Teaching Materials With Qur'an Using Generative Learning Model on Sound Waves

Preliminary observations show the limitations of teaching materials that have a relationship with spiritual aspects, especially the Qur'an, which is due to the limitations of teaching materials that contain these aspects. Required learning that stimulates students to generalize religious aspects and learn physics. Therefore, the purpose of this study is to describe the feasibility of physics teaching materials containing the Qur'an by using a generative model on sound wave material. This study aims to describe the validity, practicality, and effectiveness of the teaching materials developed. The research was conducted on 33 students at class XI MIPA 1 MAN 2 Model Banjarmasin using the ADDIE model. Data was taken from the validation results of teaching materials using validation sheet, lesson plan implementation using lesson plan implementation sheet, and students' learning outcomes at the pretest and post-test using learning outcomes tests. The results of this study were 1) the validity of the teaching materials was in the "valid" category with a value of 3.33, 2) the practicality of the teaching materials was in the "very practical" category with a value of 3.63, and 3) the effectiveness of the teaching material was in the "effective/moderate" category with a value of 0,44. This study concluded that physics teaching materials containing Al-Qur'an verses on sound wave materials using generative learning models are declared feasible to be used based on the validity, practicality, and effectiveness of the teaching materials. The implication of this research is the creation of teaching materials containing the Qur'an that are used in schools and creating student habits to link physics and the Qur'an.

An Efficient Method for Capturing the High Peak Concentrations of PM2.5 Using Gaussian-Filtered Deep Learning

This paper proposes a deep learning model that integrates a convolutional neural network with a gate circulation unit that captures patterns of high-peak PM2.5 concentrations. The purpose is to accurately predict high-peak PM2.5 concentration data that cannot be trained in general deep learning models. For the training of the proposed model, we used all available weather and air quality data for three years from 2015 to 2017 from 25 stations of the National Institute of Environmental Research (NIER) and the Korea Meteorological Administration (KMA) observatory in Seoul, South Korea. Our model trained three years of data and predicted high-peak PM2.5 concentrations for the year 2018. In addition, we propose a Gaussian filter algorithm as a preprocessing method for capturing high concentrations of PM2.5 in the Seoul area and predicting them more accurately. This model overcomes the limitations of conventional deep learning approaches that are unable to predict high peak PM2.5 concentrations. Comparing model measurements at each of the 25 monitoring sites in 2018, we found that the deep learning model with a Gaussian filter achieved an index of agreement of 0.73–0.89 and a proportion of correctness of 0.89–0.96, and compared to the conventional deep learning method (average POC = 0.85), the Gaussian filter algorithm (average POC = 0.94) improved the accuracy of high-concentration PM2.5 prediction by an average of about 9%. Applying this algorithm in the preprocessing stage could be updated to predict the risk of high PM2.5 concentrations in real time.

Leaders in Limbo: The Role of Collaborative Inquiry Influencing School Leaders’ Levels of Efficacy

Using collaborative inquiry as a methodology for investigation, this study examined the implementation of a model of generative leadership (Adams et al., 2019) to explore the question: In what ways can implementation of a generative leadership model of professional learning for school leaders influence their sense of efficacy? An adapted model of generative leadership was adopted by seven school leaders in Alberta, Canada over a period of three years. Data was collected at the end of each year through journal reflections (Bashan & Holsblat, 2017) to ascertain school leaders’ levels of efficacy. Findings revealed four themes that identified how a generative leadership model influenced participants’ effectiveness in their roles. These findings offer insight into how models of school leader professional learning might be re-imagined in order to positively influence their sense of efficacy.

A deep generative model for probabilistic energy forecasting in power systems: normalizing flows

Greater direct electrification of end-use sectors with a higher share of renewables is one of the pillars to power a carbon-neutral society by 2050. However, in contrast to conventional power plants, renewable energy is subject to uncertainty raising challenges for their interaction with power systems. Scenario-based probabilistic forecasting models have become a vital tool to equip decision-makers. This paper presents to the power systems forecasting practitioners a recent deep learning technique, the normalizing flows, to produce accurate scenario-based probabilistic forecasts that are crucial to face the new challenges in power systems applications. The strength of this technique is to directly learn the stochastic multivariate distribution of the underlying process by maximizing the likelihood. Through comprehensive empirical evaluations using the open data of the Global Energy Forecasting Competition 2014, we demonstrate that this methodology is competitive with other state-of-the-art deep learning generative models: generative adversarial networks and variational autoencoders. The models producing weather-based wind, solar power, and load scenarios are properly compared in terms of forecast value by considering the case study of an energy retailer and quality using several complementary metrics. The numerical experiments are simple and easily reproducible. Thus, we hope it will encourage other forecasting practitioners to test and use normalizing flows in power system applications such as bidding on electricity markets, scheduling power systems with high renewable energy sources penetration, energy management of virtual power plan or microgrids, and unit commitment.

Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications

Uncertainties in machine learning are a significant roadblock for its application in safety-critical cyber-physical systems (CPS). One source of uncertainty arises from distribution shifts in the input data between training and test scenarios. Detecting such distribution shifts in real-time is an emerging approach to address the challenge. The high dimensional input space in CPS applications involving imaging adds extra difficulty to the task. Generative learning models are widely adopted for the task, namely out-of-distribution (OoD) detection. To improve the state-of-the-art, we studied existing proposals from both machine learning and CPS fields. In the latter, safety monitoring in real-time for autonomous driving agents has been a focus. Exploiting the spatiotemporal correlation of motion in videos, we can robustly detect hazardous motion around autonomous driving agents. Inspired by the latest advances in the Variational Autoencoder (VAE) theory and practice, we tapped into the prior knowledge in data to further boost OoD detection’s robustness. Comparison studies over nuScenes and Synthia data sets show our methods significantly improve detection capabilities of OoD factors unique to driving scenarios, 42% better than state-of-the-art approaches. Our model also generalized near-perfectly, 97% better than the state-of-the-art across the real-world and simulation driving data sets experimented. Finally, we customized one proposed method into a twin-encoder model that can be deployed to resource limited embedded devices for real-time OoD detection. Its execution time was reduced over four times in low-precision 8-bit integer inference, while detection capability is comparable to its corresponding floating-point model.

Guided structure-based ligand identification and design via artificial intelligence modelling

ABSTRACT Introduction: The implementation of Artificial Intelligence (AI) methodologies to drug discovery (DD) are on the rise. Several applications have been developed for structure-based DD, where AI methods provide an alternative framework for the identification of ligands for validated therapeutic targets, as well as the de novo design of ligands through generative models. Areas covered: Herein, the authors review the contributions between the 2019 to present period regarding the application of AI methods to structure-based virtual screening (SBVS) which encompasses mainly molecular docking applications – binding pose prediction and binary classification for ligand or hit identification-, as well as de novo drug design driven by machine learning (ML) generative models, and the validation of AI models in structure-based screening. Studies are reviewed in terms of their main objective, used databases, implemented methodology, input and output, before their key results are summarized. Expert opinion: It is evident that the emerging methods driven by AI will become standard in the DD pipeline. With the rise of new applications, more profound analyses regarding the validity and applicability of these methods have begun to appear. In the near future, we expect to see more structure-based generative models- which are scarce in comparison to ligand-based generative models-, the implementation of standard guidelines for validating the generated structures, and more analyses regarding the validation of AI methods in structure-based DD.

Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks.

A magnetic resonance imaging (MRI) exam typically consists of the acquisition of multiple MR pulse sequences, which are required for a reliable diagnosis. With the rise of generative deep learning models, approaches for the synthesis of MR images are developed to either synthesize additional MR contrasts, generate synthetic data, or augment existing data for AI training. While current generative approaches allow only the synthesis of specific sets of MR contrasts, we developed a method to generate synthetic MR images with adjustable image contrast. Therefore, we trained a generative adversarial network (GAN) with a separate auxiliary classifier (AC) network to generate synthetic MR knee images conditioned on various acquisition parameters (repetition time, echo time, and image orientation). The AC determined the repetition time with a mean absolute error (MAE) of 239.6 ms, the echo time with an MAE of 1.6 ms, and the image orientation with an accuracy of 100%. Therefore, it can properly condition the generator network during training. Moreover, in a visual Turing test, two experts mislabeled 40.5% of real and synthetic MR images, demonstrating that the image quality of the generated synthetic and real MR images is comparable. This work can support radiologists and technologists during the parameterization of MR sequences by previewing the yielded MR contrast, can serve as a valuable tool for radiology training, and can be used for customized data generation to support AI training.

PENGARUH MODEL PEMBELAJARAN GENERATIF UNTUK MENINGKATKAN KEMAMPUAN KONEKSI MATEMATIS SISWA KELAS III

This study aims to determine the effect of the generative learning model to improve the mathematical connection of elementary school students. This research was conducted at SDN Tanjungwangi in the 2020/2021 academic year with a research sample of 20 class III A students and 20 class III B students. This research method uses a quasi-experimental method. The data analysis technique in this study uses N-gain calculations to measure the increase in students' mathematical connection abilities, (t-test) to determine the effect of using generative learning models on students' mathematical connection abilities. The results showed that (1) an increase in mathematical connection ability with an average N-gain in the experimental class obtained a result of 0.79 with the criteria "high" and the average N-gain in the control obtained a result of 0.05 with the criteria "very low" . (2) there is an effect of the use of generative models on the ability of mathematical connections with the results of tcount < ttable 0.830 < 2.024. Thus, it can be concluded that there is an influence of the generative learning model on the mathematical connection ability of elementary school students.

A Survey of Unsupervised Generative Models for Exploratory Data Analysis and Representation Learning

For more than a century, the methods for data representation and the exploration of the intrinsic structures of data have developed remarkably and consist of supervised and unsupervised methods. However, recent years have witnessed the flourishing of big data, where typical dataset dimensions are high and the data can come in messy, incomplete, unlabeled, or corrupted forms. Consequently, discovering the hidden structure buried inside such data becomes highly challenging. From this perspective, exploratory data analysis plays a substantial role in learning the hidden structures that encompass the significant features of the data in an ordered manner by extracting patterns and testing hypotheses to identify anomalies. Unsupervised generative learning models are a class of machine learning models characterized by their potential to reduce the dimensionality, discover the exploratory factors, and learn representations without any predefined labels; moreover, such models can generate the data from the reduced factors’ domain. The beginner researchers can find in this survey the recent unsupervised generative learning models for the purpose of data exploration and learning representations; specifically, this article covers three families of methods based on their usage in the era of big data: blind source separation, manifold learning, and neural networks, from shallow to deep architectures.

Computational Discovery of New 2D Materials Using Deep Learning Generative Models.

Two-dimensional (2D) materials have emerged as promising functional materials with many applications such as semiconductors and photovoltaics because of their unique optoelectronic properties. Although several thousand 2D materials have been screened in existing materials databases, discovering new 2D materials remains challenging. Herein, we propose a deep learning generative model for composition generation combined with a random forest-based 2D materials classifier to discover new hypothetical 2D materials. Furthermore, a template-based element substitution structure prediction approach is developed to predict the crystal structures of a subset of the newly predicted hypothetical formulas, which allows us to confirm their structure stability using DFT calculations. So far, we have discovered 267 489 new potential 2D materials compositions, where 1485 probability scores are more then 0.95. Among them, we have predicted 101 crystal structures and confirmed 92 2D/layered materials by DFT formation energy calculation. Our results show that generative machine learning models provide an effective way to explore the vast chemical design space for new 2D materials discovery.

WGNCS: A robust hybrid cross-version defect model via multi-objective optimization and deep enhanced feature representation

For a constantly-evolving software project with multiple releasing versions, Cross-Version Defect Prediction (CVDP) can identify the potential defect in the latter one by mining historical defect information from the prior releasing software versions. Unfortunately, the imbalanced class distribution and the complex intrinsic structure in software projects make it challenging to obtain suitable defect features and construct a predominant CVDP model. To address these challenges, we propose a robust hybrid CVDP model named WGNCS based on WGAN-GP (Wasserstein GAN with Gradient Penalty), multi-objective NSGA-III (Non-dominated Sorting Genetic Algorithm - III) algorithm and hybrid CNN-SVM (Convolutional Neural Network – Support Vector Machine) in this study, which has three main merits: (1) employ a powerful deep learning generative model – WGAN-GP to conduct data augmentation tasks, thereby achieving defect class balance and generating more training instances. (2) utilize the multi-objective NSGA-III algorithm to select the fewest representative training feature subset for the minimum error. (3) construct a single powerful defect predictor CNN-SVM by cascading a high-level deep semantic feature extractor (CNN) and a classifier (SVM) with the fixed non-linear Gaussian kernel function. We verify the CVDP performance of WGNCS on 32 cross-version pairs derived from 45 software project versions. The experimental results demonstrate that the WGNCS model can exhibit encouraging performance.

Enabling rapid COVID-19 small molecule drug design through scalable deep learning of generative models

We improved the quality and reduced the time to produce machine learned models for use in small molecule antiviral design. Our globally asynchronous multi-level parallel training approach strong scales to all of Sierra with up to 97.7% efficiency. We trained a novel, character-based Wasserstein autoencoder that produces a higher quality model trained on 1.613 billion compounds in 23 minutes while the previous state of the art takes a day on 1 million compounds. Reducing training time from a day to minutes shifts the model creation bottleneck from computer job turnaround time to human innovation time. Our implementation achieves 318 PFLOPs for 17.1% of half-precision peak. We will incorporate this model into our molecular design loop enabling the generation of more diverse compounds; searching for novel, candidate antiviral drugs improves and reduces the time to synthesize compounds to be tested in the lab.

Deep Mutual GAN for Life-Detection Radar Super Resolution

To improve the life-detection radar resolution under certain hardware conditions, in this letter, a deep mutual learning generative adversarial network model (Deep Mutual GAN) is proposed. In the proposed model, the generator can improve the angular resolution of the input low-resolution radar image by five times, which is enough to meet our requirements for the resolution of life detection. We innovatively use two generators in GAN with the same network structure and make the two generators learn from each other. In this way, the learning process of a generator is not only achieved by its confrontation with the discriminator but also guided by another generator. As a result, the knowledge of the generator is no longer only obtained through its own learning; each generator learns knowledge from another generator while learning knowledge by itself. The proposed model can effectively make the convergence of GAN more stable and improves the super resolution effect. We also introduce the details of the network structure of generator and discriminator, in which residual learning and a symmetrical network structure are applied. The experimental results show that the proposed method can achieve state-of-the-art imaging effect, which is meaningful for subsequent target detection and recognition.

Imbalanced Learning with Oversampling based on Classification Contribution Degree

AbstractImbalanced datasets exist commonly in the real world, which leads to poor performance of general machine learning models because of skewed class distribution. To address the data‐imbalance problem, a novel oversampling method based on classification contribution degree, called OS‐CCD is presented. First a new concept, classification contribution degree, is established based on micro and macro information extracted from raw datasets. With the classification contribution degree, OS‐CCD enables positive samples near the class boundary and located in an area with high density of positive samples to generate more synthetic samples than others. Furthermore, the neighbor selection for oversampling is no longer random but in the light of a selected probability. Experimental results on 12 benchmark datasets substantiate that four commonly used classifiers with the oversampling method outperform those with six popular oversampling methods in terms of accuracy, F1‐score and AUC.

The Use of Generative Learning Model in Improving Students’ Understanding of Mathematical Concepts of Al-Azhar 19 Islamic High School

Background. Distance learning during the COVID-19 pandemic has changed the landscape of mathematics learning as a whole. One of the mathematics learning models that offer new concepts in distance learning is generative learning model. The Aim of the Research. This study will evaluate the effectiveness of the implementation of generative learning models in improving students' mathematical understanding abilities. The appropriate research method to test the effectiveness is quasi-experimental design because the class being experimented with is not changed, so that it does not have much effect on overall learning stability. The research sample used cluster random sampling technique where class X MIPA 3 was the experimental class while class X MIPA 2 was the control class. The research instrument used was a test of students' ability to understand mathematical concepts in the material of Logarithmic Equations as many as seven essay questions. Conclusion. There is an effect (high) of the generative learning model on the ability to understand students' mathematical concepts applied to the distance learning system at Al-Azhar 19 Ciracas Islamic Senior High School logarithmic equation material with a significance level of α = 0.05. Contribution. This study offers a new concept for the application of distance learning mathematics by implementing a generative learning model which in this study has contributed positively to student understanding

Challenges and Corresponding Solutions of Generative Adversarial Networks (GANs): A Survey Study

Generative Adversarial Networks (GANs) are an innovative class of deep learning generative model that has been popular among academics recently. GANs are able to learn distributions on complex high-dimensional data which made it efficient in images and audio processing. Nevertheless, in the training of GANs, some major challenges exist namely mode collapse, non-convergence, and instability. In recent years, in order to overcome these challenges, researchers have proposed many variants of GANs by redesigning network architecture, changing the form of objective functions, and altering optimization algorithms. In this research, we conducted a comprehensive investigation on the progress of GANs design and optimization solutions. Finally, according to the classification method, we provided a problem-solving structure to solve conquer the GANs training challenges.

Conditional Text Generation for Harmonious Human-Machine Interaction

In recent years, with the development of deep learning, text-generation technology has undergone great changes and provided many kinds of services for human beings, such as restaurant reservation and daily communication. The automatically generated text is becoming more and more fluent so researchers begin to consider more anthropomorphic text-generation technology, that is, the conditional text generation, including emotional text generation, personalized text generation, and so on. Conditional Text Generation (CTG) has thus become a research hotspot. As a promising research field, we find that much attention has been paid to exploring it. Therefore, we aim to give a comprehensive review of the new research trends of CTG. We first summarize several key techniques and illustrate the technical evolution route in the field of neural text generation, based on the concept model of CTG. We further make an investigation of existing CTG fields and propose several general learning models for CTG. Finally, we discuss the open issues and promising research directions of CTG.

A review of three dimensional reconstruction techniques

Three dimensional (3D) modeling is an important stereoscopic representation of an object for multiple viewpoints aggregation and geometrical information. A general 3D modeling pipeline consists of data acquisition, 3D reconstruction and surface reconstruction. The core computational process in 3D modeling is always associated with the 3D reconstruction, which can be categorized into three types, i.e. statistical models, discriminative learning models and generative learning models. Statistical models derive handcrafted feature descriptor from mathematical theory to extract the spatial and geometric features of 3D data. A corresponding matching is performed between multiple viewpoints 3D data to search for the maximum region likelihood across datasets and compute the best match of affine transformation. Discriminative learning models learn spatial coherent of 3D data through data-driven training that leads to the computation of affine transformation with data inferencing. Generative models, on the other hand, have the unsupervised capability of ingesting raw 3D data directly to learn latent representation of input 3D data and later generate ambient output sample from the latent representation. In this paper, a detailed comparison on the three types of 3D reconstruction techniques are reviewed in term of input data structure, correspondence accuracy, precision and recall using four benchmark datasets, i.e. ModelNet10/40, ICL-NUIM, and Semantic3D. The advantages and disadvantages of 3D reconstruction techniques are highlighted for implementation guideline and future improvement.