Neural Generators Research Articles

MTL-TRANSFER: Leveraging Multi-task Learning and Transferred Knowledge for Improving Fault Localization and Program Repair

Fault localization (FL) and automated program repair (APR) are two main tasks of automatic software debugging. Compared with traditional methods, deep learning-based approaches have been demonstrated to achieve better performance in FL and APR tasks. However, the existing deep learning-based FL methods ignore the deep semantic features or only consider simple code representations. And for APR tasks, existing template-based APR methods are weak in selecting the correct fix templates for more effective program repair, which are also not able to synthesize patches via the embedded end-to-end code modification knowledge obtained by training models on large-scale bug-fix code pairs. Moreover, in most of FL and APR methods, the model designs and training phases are performed separately, leading to ineffective sharing of updated parameters and extracted knowledge during the training process. This limitation hinders the further improvement in the performance of FL and APR tasks. To solve the above problems, we propose a novel approach called MTL-TRANSFER, which leverages a multi-task learning strategy to extract deep semantic features and transferred knowledge from different perspectives. First, we construct a large-scale open-source bug datasets and implement 11 multi-task learning models for bug detection and patch generation sub-tasks on 11 commonly used bug types, as well as one multi-classifier to learn the relevant semantics for the subsequent fix template selection task. Second, an MLP-based ranking model is leveraged to fuse spectrum-based, mutation-based and semantic-based features to generate a sorted list of suspicious statements. Third, we combine the patches generated by the neural patch generation sub-task from the multi-task learning strategy with the optimized fix template selecting order gained from the multi-classifier mentioned above. Finally, the more accurate FL results, the optimized fix template selecting order, and the expanded patch candidates are combined together to further enhance the overall performance of APR tasks. Our extensive experiments on widely-used benchmark Defects4J show that MTL-TRANSFER outperforms all baselines in FL and APR tasks, proving the effectiveness of our approach. Compared with our previously proposed FL method TRANSFER-FL (which is also the state-of-the-art statement-level FL method), MTL-TRANSFER increases the faults hit by 8/11/12 on Top-1/3/5 metrics (92/159/183 in total). And on APR tasks, the number of successfully repaired bugs of MTL-TRANSFER under the perfect localization setting reaches 75, which is 8 more than our previous APR method TRANSFER-PR. Furthermore, another experiment to simulate the actual repair scenarios shows that MTL-TRANSFER can successfully repair 15 and 9 more bugs (56 in total) compared with TBar and TRANSFER, which demonstrates the effectiveness of the combination of our optimized FL and APR components.

Residential PV-battery scheduling with stochastic optimization and neural network-driven scenario generation

Residential end-users have increasingly been attracted to installing behind-the-meter (BTM) photovoltaic (PV) and battery energy storage systems (BESS). The inherent stochastic nature of PV introduces challenges to energy management for PV-BESS owners. This work presents a two-stage stochastic optimization (SO) framework for battery operation that takes into account uncertainties in PV outputs to minimize daily grid consumption and battery degradation. A combination of a feed-forward neural network (FFNN) and an error analysis statistical technique is proposed to generate a highly accurate scenario set for estimating PV behavior. Then, the scenario set is reduced using a backward scenario reduction technique to address the issue of dimensionality that SO suffers from. The reduced scenario set is then fed into the two-stage SO model to calculate expected electricity costs and battery degradation for the day-ahead PV-BESS problem. The suggested model's performance in controlling BTM batteries is evaluated using real-world data collected from smart meters installed in a house in Aran Islands, Ireland. The results show that the model can effectively withstand sudden changes in the PV output and generate results that are comparable to an ideal case with accurate PV data available.

Meta-learning based blind image super-resolution approach to different degradations

Although recent studies on blind single image super-resolution (SISR) have achieved significant success, most of them typically require supervised training on synthetic low resolution (LR)-high resolution (HR) paired images. This leads to re-training necessity for different degradations and restricted applications in real-world scenarios with unfavorable inputs. In this paper, we propose an unsupervised blind SISR method with input underlying different degradations, named different degradations blind super-resolution (DDSR). It formulates a Gaussian modeling on blur degradation and employs a meta-learning framework for solving different image degradations. Specifically, a neural network-based kernel generator is optimized by learning from random kernel samples, referred to as random kernel learning. This operation provides effective initialization for blur degradation optimization. At the same time, a meta-learning framework is proposed to resolve multiple degradation modelings on the basis of alternative optimization between blur degradation and image restoration, respectively. Differing from the pre-trained deep-learning methods, the proposed DDSR is implemented in a plug-and-play manner, and is capable of restoring HR image from unfavorable LR input with degradations such as partial coverage, noise addition, and darkening. Extensive simulations illustrate the superior performance of the proposed DDSR approach compared to the state-of-the-arts on public datasets with comparable memory load and time consumption, yet exhibiting better application flexibility and convenience, and significantly better generalization ability towards multiple degradations. Our code is available at https://github.com/XYLGroup/DDSR.

Neural Sequence Generation with Constraints via Beam Search with Cuts: A Case Study on VRP

Neural Sequence Generation with Constraints via Beam Search with Cuts: A Case Study on VRP

A cross-attention augmented model for event-triggered context-aware story generation

Despite recent advancements, existing story generation systems continue to encounter difficulties in effectively incorporating contextual and event features, which greatly influence the quality of generated narratives. To tackle these challenges, we introduce a novel neural generation model, EtriCA, that enhances the relevance and coherence of generated stories by employing a cross-attention mechanism to map context features onto event sequences through residual mapping. This feature capturing mechanism enables our model to exploit logical relationships between events more effectively during the story generation process. To further enhance our proposed model, we employ a post-training framework for knowledge enhancement (KeEtriCA) on a large-scale book corpus. This allows EtriCA to adapt to a wider range of data samples. This results in approximately 5% improvement in automatic metrics and over 10% improvement in human evaluation. We conduct extensive experiments, including comparisons with state-of-the-art (SOTA) baseline models, to evaluate the performance of our framework on story generation. The experimental results, encompassing both automated metrics and human assessments, demonstrate the superiority of our model over existing state-of-the-art baselines. These results underscore the effectiveness of our model in leveraging context and event features to improve the quality of generated narratives.

Swallowing-like activity elicited in neonatal rat medullary slice preparation

Swallowing is induced by a central pattern generator in the nucleus tractus solitarius (NTS). We aimed to create a medullary slice preparation to elucidate the neural architecture of the central pattern generator of swallowing (Sw-CPG) and record its neural activities. Experiments were conducted on 2-day-old Sprague–Dawley rats (n = 46). The brainstem-spinal cord was transected at the pontomedullary and cervicothoracic junctions; the medulla was sliced transversely at thicknesses of 600, 700, or 800 μm. The rostral end of the slice was 100 μm rostral to the vagus nerve. We recorded hypoglossal nerve activity and electrically stimulated the vagus nerve or microinjected bicuculline methiodide (BIC) into the NTS. The 800-μm slices generated both rhythmic respiratory activity and electrically elicited neural activity. The 700-μm slices generated only respiratory activity, while the 600-μm slices did not generate any neural activity. BIC microinjection into the NTS in 800-μm slices resulted in the typical activity that closely resembled the swallowing activity reported in other experiments. This swallowing-like activity consistently lengthened the respiratory interval. Despite complete inhibition of respiratory activity, weak swallowing-like activity was observed under bath application of a non-NMDA receptor antagonist. Contrastingly, bath application of NMDA receptor antagonists resulted in a complete loss of swallowing-like activity and no change in respiratory activity. These results suggest that the 800-μm medullary slice preparation contains both afferent and efferent neural circuits and pattern generators of swallowing activity. Additionally, NMDA receptors may be necessary for generating swallowing activity. This medullary slice preparation can therefore elucidate Sw-CPG neural networks.

Pre-Trained Language Models for Text Generation: A Survey

Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, with the help of neural generation models based on pre-trained language models (PLMs). Text generation based on PLMs is viewed as a promising approach in both academia and industry. In this article, we provide a survey on the utilization of PLMs in text generation. We begin with introducing two key aspects of applying PLMs to text generation: (1) how to design an effective PLM to serve as the generation model; and (2) how to effectively optimize PLMs given the reference text and to ensure that the generated texts satisfy special text properties. Then, we show the major challenges that have arisen in these aspects, as well as possible solutions for them. We also include a summary of various useful resources and typical text generation applications based on PLMs. Finally, we highlight the future research directions which will further improve these PLMs for text generation. This comprehensive survey is intended to help researchers interested in text generation problems to learn the core concepts, the main techniques and the latest developments in this area based on PLMs.

Deep Learning Analysis With Gray Scale and Doppler Ultrasonography Images to Differentiate Graves' Disease.

Thyrotoxicosis requires accurate and expeditious differentiation between Graves' disease (GD) and thyroiditis to ensure effective treatment decisions. This study aimed to develop a machine learning algorithm using ultrasonography and Doppler images to differentiate thyrotoxicosis subtypes, with a focus on GD. This study included patients who initially presented with thyrotoxicosis and underwent thyroid ultrasonography at a single tertiary hospital. A total of 7719 ultrasonography images from 351 patients with GD and 2980 images from 136 patients with thyroiditis were used. Data augmentation techniques were applied to enhance the algorithm's performance. Two deep learning models, Xception and EfficientNetB0_2, were employed. Performance metrics such as accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score were calculated for both models. Image preprocessing, neural network model generation, and neural network training results verification were performed using DEEP:PHI® platform. The Xception model achieved 84.94% accuracy, 89.26% sensitivity, 73.17% specificity, 90.06% PPV, 71.43% NPV, and an F1 score of 89.66 for the diagnosis of GD. The EfficientNetB0_2 model exhibited 85.31% accuracy, 90.28% sensitivity, 71.78% specificity, 89.71% PPV, 73.05% NPV, and an F1 score of 89.99. Machine learning models based on ultrasound and Doppler images showed promising results with high accuracy and sensitivity in differentiating GD from thyroiditis.

What can rhetoric bring us? Incorporating rhetorical structure into neural related work generation

The ever-increasing volume of research literature poses challenges for researchers in keeping up with related works in their fields. Automating the generation of related work section holds promise for saving time and effort. However, current models often fall short of producing coherent and logically correct related work with multiple sentences, a phenomenon we refer to as rhetorical structure chaos. Rhetorical structure describes how adjacent spans of units are connected to each other, and logically correct rhetorical structure is essential for a well-structured related work. Hence, to tackle the rhetorical structure chaos issue, this paper explicitly incorporates rhetorical structure information into related work generation. Firstly, we conduct the first rhetorical structure analysis for related work section, which provides insights into understanding the organization and arrangement of contents within related work. Then, based on two preliminary studies on rhetorical structure, we present a novel related work generation model called RSGen, which incorporates rhetorical structure at both the encoding and decoding stages. The encoding stage is facilitated with a rhetorical structure-based graph encoder, while the decoding process is guided by a rhetorical plan — ordered sequence of rhetorical functions of the related work. We conduct extensive experiments on three related work generation datasets to evaluate the performance of our model. The results show that our approach achieves state-of-the-art performance on ROUGE metrics. An ablation study and more analyses further highlight the remarkable efficacy of introducing rhetorical structure into both the encoding and decoding stages.

Chaotic neural network for information security

The neural network-based key generator has been employed in this work. The chaotic binary sequences were created using one of the best and simplest chaotic Bernoulli maps. Additionally, chaotic sequences' randomness can be improved using neural networks, where the inputs and weights completely determine the output sequence. As the created sequences are based on the perceptron learning approach, the output sequence is unpredictable. The generated sequences passed each test that was conducted using the Diehard battery test and NIST 800–22. To strengthen the security of the images, DNA computing was also used for the ANN binary sequence and encryption.

Event type induction using latent variables with hierarchical relationship analysis

The conventional approach to event extraction requires predefined event types and their corresponding annotations to train event extractors. However, these prerequisites are often difficult to satisfy in real-world applications. To automatically induct event types, most work has been devoted to clustering event triggers, where a cluster of event triggers is represented as an event type. Some works use trigger semantics, while others use co-occurrence relationships to cluster triggers. However, the clustering results of event triggers obtained by the above work are not sufficiently detailed in describing event types, making it difficult to accurately determine the corresponding event types manually. This paper proposes an open-domain event type induction framework that automatically discovers a set of event types from a given corpus. Unlike previous work on event trigger clustering, this paper takes into consideration the hierarchical relationship of event types to partition the event trigger clusters into event mains and subtypes. The framework employs a latent variable-based neural generation module and a semantic-based clustering module, the former of which obtains event trigger clusters representing the main types of events by jointly projecting the co-occurrence and semantic information of event triggers into a latent space for event type latent variable mining, and the latter of which further divides these event trigger clusters into event subtypes based on semantic information. Finally, experiment results show that, compared with the benchmark model, the ETGen-Clus can improve event type quality scores of 6.23% and 3.11% on the two datasets, respectively.

Neural Network-Based Sum-Frequency Generation Spectra of Pure and Acidified Water Interfaces with Air.

The affinity of hydronium ions (H3O+) for the air-water interface is a crucial question in environmental chemistry. While sum-frequency generation (SFG) spectroscopy has been instrumental in indicating the preference of H3O+ for the interface, key questions persist regarding the molecular origin of the SFG spectral changes in acidified water. Here we combine nanosecond long neural network (NN) reactive simulations of pure and acidified water slabs with NN predictions of molecular dipoles and polarizabilities to calculate SFG spectra of long reactive trajectories including proton transfer events. Our simulations show that H3O+ ions cause two distinct changes in phase-resolved SFG spectra: first, a low-frequency tail due to the vibrations of H3O+ and its first hydration shell, analogous to the bulk proton continuum, and second, an enhanced hydrogen-bonded band due to the ion-induced static field polarizing molecules in deeper layers. Our calculations confirm that changes in the SFG spectra of acidic solutions are caused by hydronium ions preferentially residing at the interface.

Are the P600 and P3 ERP components linked to the task-evoked pupillary response as a correlate of norepinephrine activity?

During language comprehension, anomalies and ambiguities in the input typically elicit the P600 event-related potential component. Although traditionally interpreted as a specific signal of combinatorial operations in sentence processing, the component has alternatively been proposed to be a variant of the oddball-sensitive, domain-general P3 component. In particular, both components might reflect phasic norepinephrine release from the locus coeruleus (LC/NE) to motivationally significant stimuli. In this preregistered study, we tested this hypothesis by relating both components to the task-evoked pupillary response, a putative biomarker of LC/NE activity. 36 participants completed a sentence comprehension task (containing 25% morphosyntactic violations) and a non-linguistic oddball task (containing 20% oddballs), while the EEG and pupil size were co-registered. Our results showed that the task-evoked pupillary response and the ERP amplitudes of both components were similarly affected by both experimental tasks. In the oddball task, there was also a temporally specific relationship between the P3 and the pupillary response beyond the shared oddball effect, thereby further linking the P3 to NE. Because this link was less reliable in the linguistic context, we did not find conclusive evidence for or against a relationship between the P600 and the pupillary response. Still, our findings further stimulate the debate on whether language-related ERPs are indeed specific to linguistic processes or shared across cognitive domains. However, further research is required to verify a potential link between the two ERP positivities and the LC/NE system as the common neural generator.

Процессы когнитивного контроля в тесте Струпа и их отражение в связанных с событиями потенциалах (обзор)

The review summarizes the data of Russian and foreign studies on the psychophysiological parameters of the Stroop task execution. In addition, the article considers the main modifications of the task and similar paradigms as well as the key hypotheses explaining the causes of the interference effect and the increase in reaction time to stimuli containing conflicting information. Further, the paper presents data on eventrelated potential (ERP) waves, which are supposed to reflect cognitive control processes, such as conflict detection, overcoming of interference and conflict resolution. The frontocentral N2 wave characterizes the processes of conflict detection and overcoming of interference, and its main neural generator is the anterior cingulate cortex. The N450 wave is primarily generated in the anterior cingulate cortex and prefrontal cortex and is thought to reflect interference suppression. The centroparietal P300 wave and the late positive complex, which appears to be generated in the middle or inferior frontal gyrus and in the extrastriate cortex, are associated with the conflict resolution process. The readiness potential is, supposedly, generated in the motor cortex and reflects the process of selecting and preparing a motor response. Moreover, the review presents the main directions of studies that use the Stroop test paradigm and its modifications. Despite a significant number of existing psychophysical and neurophysiological papers, the question of the brain mechanisms of cognitive control in tasks that cause cognitive conflict remains open, and the neuropsychological role of the ERP waves recorded in such tasks is still not fully explored. Studies into the influence of different types of competing information and the degree of conflict on the psychophysiological indicators in the Stroop task are of considerable interest.

Generative adversarial reduced order modelling

In this work, we present GAROM, a new approach for reduced order modeling (ROM) based on generative adversarial networks (GANs). GANs attempt to learn to generate data with the same statistics of the underlying distribution of a dataset, using two neural networks, namely discriminator and generator. While widely applied in many areas of deep learning, little research is done on their application for ROM, i.e. approximating a high-fidelity model with a simpler one. In this work, we combine the GAN and ROM framework, introducing a data-driven generative adversarial model able to learn solutions to parametric differential equations. In the presented methodology, the discriminator is modeled as an autoencoder, extracting relevant features of the input, and a conditioning mechanism is applied to the generator and discriminator networks specifying the differential equation parameters. We show how to apply our methodology for inference, provide experimental evidence of the model generalization, and perform a convergence study of the method.

Electromyography-Based Biomechanical Cybernetic Control of a Robotic Fish Avatar

This study introduces a cybernetic control and architectural framework for a robotic fish avatar operated by a human. The behavior of the robot fish is influenced by the electromyographic (EMG) signals of the human operator, triggered by stimuli from the surrounding objects and scenery. A deep artificial neural network (ANN) with perceptrons classifies the EMG signals, discerning the type of muscular stimuli generated. The research unveils a fuzzy-based oscillation pattern generator (OPG) designed to emulate functions akin to a neural central pattern generator, producing coordinated fish undulations. The OPG generates swimming behavior as an oscillation function, decoupled into coordinated step signals, right and left, for a dual electromagnetic oscillator in the fish propulsion system. Furthermore, the research presents an underactuated biorobotic mechanism of the subcarangiform type comprising a two-solenoid electromagnetic oscillator, an antagonistic musculoskeletal elastic system of tendons, and a multi-link caudal spine composed of helical springs. The biomechanics dynamic model and control for swimming, as well as the ballasting system for submersion and buoyancy, are deduced. This study highlights the utilization of EMG measurements encompassing sampling time and μ-volt signals for both hands and all fingers. The subsequent feature extraction resulted in three types of statistical patterns, namely, Ω,γ,λ, serving as inputs for a multilayer feedforward neural network of perceptrons. The experimental findings quantified controlled movements, specifically caudal fin undulations during forward, right, and left turns, with a particular emphasis on the dynamics of caudal fin undulations of a robot prototype.

Node-Wise Monotone Barrier Coupling Law for Formation Control.

We study a node-wise monotone barrier coupling law, motivated by the synaptic coupling of neural central pattern generators. It is illustrated that this coupling imitates the desirable properties of neural central pattern generators. In particular, the coupling law (1) allows us to assign multiple central patterns on the circle and (2) allows for rapid switching between different patterns via simple 'kicks'. In the end, we achieve full control by partitioning the state space by utilizing a barrier effect and assigning a unique steady-state behavior to each element of the resulting partition. We analyze the global behavior and study the viability of the design.

Content policy and access limitations on commercial neural networks as an incentive to artivism

This article employs a case‐study method to investigate the artivism neural network community concentrated on Twitter (since renamed X), which has been ideologically influenced by the content policy and limitations of OpenAI. Today, many young artists using machine learning technologies in their artworks (Midjourney, Stable Diffusion, Kandinsky) note that despite significant progress in the field of neural network generators of image through prompts present in museums and exhibitions of contemporary digital art, a significant number of artworks are still made chiefly using outdated text-to-image algorithms created in 2021. These neural networks continue to be popular in art to this day. The reasons for the sustainability of such practices can be found in the soft ideological conflict between artists and OpenAI in 2021. At that time, neural networks had not yet become mainstream, and the dominant theme was deep fakes, which became the basis for a comprehensive discussion about the possibilities and consequences of implementing AI algorithms in modern society. A series of scandals related to the work of neural networks alerted businesses, which feared the reputational costs of neural network errors and biases. At the same time, the existing discourse on freedom of speech, thought, and self-expression in contemporary art has led to ideological conflict, as the creators have introduced constraints on tools of artistic expression. Previously, the actions of artists were not moderated by technical means. Thus, the community did not accept this state of affairs, and as a result of cooperation and “collective intelligence” created, on the GitHub and Google Colab platforms, their own algorithms with open code, with which everyone could carry out their visual experiments. Artists face the ideological question of fighting globalism and anti-progress in art to be outside the system but to riot against it. This process led to a division of artistic practises in neural network art, outlined by media artist Ryan Murdock as a gateway to text-guided visual art by the hacker effort of 2021 or the modern generation of algorithm text-to-images (after 2022).

Three-stage Transferable and Generative Crowdsourced Comment Integration Framework Based on Zero- and Few-shot Learning with Domain Distribution Alignment

Online shopping has become a crucial way to encourage daily consumption, where the User-generated, or crowdsourced product comments, can offer a broad range of feedback on e-commerce products. As a result, integrating critical opinions or major attitudes from the crowdsourced comments can provide valuable feedback for marketing strategy adjustment or product-quality monitoring. Unfortunately, the scarcity of annotated ground truth on the integrated comment, or the limited gold integration reference, has incurred the infeasibility of the regular supervised-learning-based comment integration. To resolve this problem, in this article, inspired by the principle of Transfer Learning, we propose a three-stage transferable and generative crowdsourced comment integration framework ( TTGCIF ) based on zero-and-few-shot learning with the support of domain distribution alignment. The proposed framework aims at generating abstractive integrated comment in target domain via the enhanced neural text generation model, by referring the available integration resource in related source domains, to avoid the exhausted effort on resource annotation devoted to the target domain. Specifically, at the first stage, to enhance the domain transferability, representations on the crowdsourced comments have been aligned up between the source and target domain, by minimizing the domain distribution discrepancy in the kernel space. At the second stage, Zero-shot comment integration mechanism has been adopted to deal with the dilemma that none of the gold integration reference may be available in target domain. In other words, taking the sample-level semantic prototype as input, the enhanced neural text generation model in TTGCIF is trained to learn data semantic association among different domains via semantic prototype transduction, so that the “ unlabeled ” crowdsourced comments in target domain can be associated with existing integration references in related source domains. At the third stage, based on the parameters trained at the second stage, fast domain adaptation mechanism in a Few-shot manner has also been adopted by seeking most potential parameters along the gradient direction constrained by instances across multiple source domains. In this way, parameters in TTGCIF can be sensitive to any alteration on training data, ensuring that even if only few annotated resource in target domain are available for “Fine-tune,” TTGCIF can still react promptly to achieve effective target domain adaptation. According to the experimental results, TTGCIF can achieve the best transferable product comment integration performance in target domain, with fast and stable domain adaption effect depending on no more than 10% annotated resource in target domain. More importantly, even if TTGCIF has not been fine-tuned on the target domain, yet by referring to the available integration resource in related source domains, the integrated comments generated by TTGCIF on the target domain are still superior to those generated by models already fine-tuned on the target domain.

Virtual Topologies for Populating Overhead Low-Voltage Broadband over Powerlines Topology Classes by Exploiting Neural Network Topology Generator Methodology (NNTGM) - Part 1: Theory

Based on a set of indicative overhead low voltage broadband over power lines (OV LV BPL) topologies of respective OV LV BPL topology classes, Neural Network Topology Generator Methodology (NNTGM) is theoretically proposed in this paper, so that its generated OV LV BPL topologies (NNTGM OV LV BPL topologies) can populate the OV LV BPL topology classes. Given the indicative topology of the OV LV BPL topology class, the NNTGM OV LV BPL topologies can be statistically familiar with the corresponding indicative OV LV BPL topology in terms of the theoretical channel attenuation behavior. Actually, NNTGM is based on the reverse procedure of the Neural Network Identification Methodology for the distribution line and branch Line Length Approximation (NNIM-LLA); say, NNTGM generates theoretical channel attenuation behaviors given the number of branches, the distribution line lengths and the branch line lengths, when appropriate NNTGM default operation settings are assumed. The statistical familiarity between the examined indicative OV LV BPL topology and an NNTGM OV LV BPL topology is examined after the application of appropriate channel attenuation metrics. On the basis of the channel attenuation metrics, class maps are theoretically defined so that the relative positions of indicative OV LV BPL topologies and their respective NNTGM OV LV BPL topologies (NNTGM virtual topologies) can be further studied.