Input Domain Research Articles

The Domain-Specific Neural Basis of Auditory Statistical Learning in 5-7-Year-Old Children.

Statistical learning (SL) is the ability to rapidly track statistical regularities and learn patterns in the environment. Recent studies show that SL is constrained by domain-specific features, rather than being a uniform learning mechanism across domains and modalities. This domain-specificity has been reflected at the neural level, as SL occurs in regions primarily involved in processing of specific modalities or domains of input. However, our understanding of how SL is constrained by domain-specific features in the developing brain is severely lacking. The present study aims to identify the functional neural profiles of auditory SL of linguistic and nonlinguistic regularities among children. Thirty children between 5 and 7 years old completed an auditory fMRI SL task containing interwoven sequences of structured and random syllable/tone sequences. Using traditional group univariate analyses and a group-constrained subject-specific analysis, frontal and temporal cortices showed significant activation when processing structured versus random sequences across both linguistic and nonlinguistic domains. However, conjunction analyses failed to identify overlapping neural indices across domains. These findings are the first to compare brain regions supporting SL of linguistic and nonlinguistic regularities in the developing brain and indicate that auditory SL among developing children may be constrained by domain-specific features.

Domain knowledge-powered attention for air traffic management hazardous events classification

The automated classification of hazardous events in Air Traffic Management (ATM) involves concise texts with specialized terminology. The problem is currently addressed by machine learning text classification methods; however, some methods overlook domain knowledge, while others require it excessively. This research aims to identify a process for integrating low-cost, text-based domain knowledge into classification to address the issue of domain shift. To achieve this, the Wide and Deep Bidirectional Encoder Representations from Transformers (WD-BERT) model is proposed, featuring a unique knowledge-powered attention mechanism. WD-BERT's Wide Attention module assesses the compatibility between input text and domain knowledge to determine class probabilities, while its Deep Attention module extracts contextual features guided by domain knowledge. Additionally, a text mining method is employed to extract domain knowledge from ATM regulation texts. The model is trained and evaluated using a dataset derived from the China ATM Hazard Source Database. It achieves a multi-label classification accuracy of 80.24% and an F1-micro score of 0.88, outperforming comparative models with state-of-the-art performance. Breakdown analyses of the model indicate that the carefully designed attention mechanism endows WD-BERT with the ability to integrate domain knowledge, allowing it to excel in contexts involving complex sentences, multiple terms, and domain shifts. The innovation of this study lies in the proposed domain knowledge-powered attention mechanism, which allows simply organized domain knowledge to effectively guide ATM hazardous events classification, alleviating the impact of domain shift. This method can also be applied to other vertical domains without knowledge graphs.

Non-interactive Private Multivariate Function Evaluation using Homomorphic Table Lookup

To address security issues in cloud computing, fully homomorphic encryption (FHE) enables a third party to evaluate functions using ciphertexts that do not leak information to the cloud server. The remaining problems of FHE include high computational costs and limited arithmetic operations, only evaluating additions and multiplications. Arbitrary functions can be evaluated using a precomputed lookup table (LUT), which is one of the solutions for those problems. Previous studies proposed LUT-enabled computation methods 1) with bit-wise FHE and 2) with word-wise FHE. The performance of LUT-enabled computation with bit-wise FHE drops quickly when evaluating BigNum functions because of the complexity being O(s·2^d·m), where m represents the number of inputs, d and s represent the bit lengths of the inputs and outputs, respectively. Thus, LUT-enabled computation with word-wise FHE, which handles a set of bits with one operation, has also been proposed; however, previous studies are limited in evaluating multivariate functions within two inputs and cannot speed up the evaluation when the domain size of the integer exceeds 2N, where N is the number of elements packed into a single ciphertext. In this study, we propose a non-interactive model, in which no decryption is required, to evaluate arbitrary multivariate functions using homomorphic table lookup with word-wise FHE. The proposed LUT-enabled computation method 1) decreases the complexity to O(2^d·m/l), where l is the element size of FHE packing; 2) extends the input and output domain sizes to evaluate multivariate functions over two inputs; and 3) adopts a multidimensional table for enabling multithreading to reduce latency. The experimental results demonstrate that evaluating a 10-bit two-input function and a 5-bit three-input function takes approximately 90.5 and 105.5 s with 16-thread, respectively. Our proposed method achieves 3.2x and 23.1x speedup to evaluate two-bit and three-bit 3-input functions compared with naive LUT-enabled computation with bit-wise FHE.

Real-time ray transfer for lens flare rendering using sparse polynomials

AbstractWhile lens flares are often undesired artifacts of the imaging process, they are also essential for increasing the level of realism in video games and serve as a powerful artistic tool for photography and filmography. For these reasons, computationally reproducing lens flares has always received special attention. Due to the cost of analytical ray tracing, existing solutions are unable to simultaneously achieve the performance needed for real-time environments and retain the ability to simulate arbitrarily complex ghost shapes. Although polynomial optics has been successfully used to increase the efficiency of ray tracing in multiple rendering areas, no complete and validated solution exists that correctly models all aspects of lens flares. This paper presents our polynomial optics-based method for efficiently and accurately ray tracing lens flare ghosts. Our approach successfully models the shape, energy absorption, chromatic effects, and blocking of lens flare rays by partitioning the input domain into local fitting zones. We demonstrate that our model provides a considerable speedup and high accuracy compared to the analytical approach and achieves better fitting speed, output quality, and rendering performance than the naïve application of polynomial optics. The source code for our implementation is available on GitHub.

Review of machine learning with sentimental analysis method for cross-model stock price prediction

Stock market prediction has been a popular area of research for years. Machine learning, as a fast-developing popular algorithm, is applied to stock market prediction by many previous researchers to better solve the time series involving problems over the changing prices. Different from traditional machine learning algorithms that focus on stock prices only, this paper gives a brief description and review of stock price predictions models that contain sentimental analysis over the recent paper works. Different from the prices in number format, sentimental analysis is more based on textual information extraction and mining, converting them into usable input to pass to a prediction model. This extends the prediction model's takeable input domain and strengthens the accuracy. To better classify the differences between the models, discussion and introduction are given based on different model types about whether they are traditional type or deep neural network embedded. Even though traditional types of models are more popular for sentimental analysis and neural networks perform better in prediction tasks, traditional methods are relatively easy to build or train with more explainability, compared to deep learning models suitable for larger data sets.

Latent Relations at Steady-state with Associative Nets.

Models of word meaning that exploit patterns of word usage across large text corpora to capture semantic relations, like the topic model and word2vec, condense word-by-context co-occurrence statistics to induce representations that organize words along semantically relevant dimensions (e.g., synonymy, antonymy, hyponymy, etc.). However, their reliance on latent representations leaves them vulnerable to interference, makes them slow learners, and commits to a dual-systems account of episodic and semantic memory. We show how it is possible to construct the meaning of words online during retrieval to avoid these limitations. We implement a spreading activation account of word meaning in an associative net, a one-layer highly recurrent network of associations, called a Dynamic-Eigen-Net, that we developed to address the limitations of earlier variants of associative nets when scaling up to deal with unstructured input domains like natural language text. We show that spreading activation using a one-hot coded Dynamic-Eigen-Net outperforms the topic model and reaches similar levels of performance as word2vec when predicting human free associations and word similarity ratings. Latent Semantic Analysis vectors reached similar levels of performance when constructed by applying dimensionality reduction to the Shifted Positive Pointwise Mutual Information but showed poorer predictability for free associations when using an entropy-based normalization. An analysis of the rate at which the Dynamic-Eigen-Net reaches asymptotic performance shows that it learns faster than word2vec. We argue in favor of the Dynamic-Eigen-Net as a fast learner, with a single-store, that is not subject to catastrophic interference. We present it as an alternative to instance models when delegating the induction of latent relationships to process assumptions instead of assumptions about representation.

A Neural Network Approach to Physical Information Embedding for Optimal Power Flow

With the increasing share of renewable energy in the power system, traditional power flow calculation methods are facing challenges of complexity and efficiency. To address these issues, this paper proposes a new framework for AC optimal power flow analysis based on a physics-informed convolutional neural network (PICNN) approach, which enables the neural network to learn solutions that follow physical laws by embedding the power flow equations and other physical constraints into the loss function of the network. Compared with the traditional power flow calculation method, the calculation speed of this method is improved by 10–30 times. Compared to traditional neural network models, the method provides higher accuracy, with an average increase in accuracy of up to 2.5–10 times. In addition, this paper introduces a methodology to extract worst-case guarantees for violations of the neural network’s predicted power generation constraints, determining the worst possible violation that could result from any neural network output across the entire input domain, and taking appropriate measures to reduce the violation. The method is experimentally shown to be highly accurate and reliable for the AC optimal power flow (AC-OPF) analysis problem, while reducing the dependence on a large amount of labelled data.

On the Impact of Input Models on the Fault Detection Capabilities of Combinatorial Testing

Testing is an important activity to detect faults before software deployment. We focus on black-box combinatorial testing, where fault detection is one of the main objectives. In this paper, we argue that input model abstraction notably impacts the fault detection capability of a combinatorial test suite. First, we present experiments from previous work that support this argument. We then perform new experiments on a more diverse set of programs. These experiments use mutation testing to estimate fault detection capability, but we also include structural coverage measures in the new experiments. Finally, we elaborate on two possible improvements to obtain an optimal input abstraction strategy for not just continuous but all input domains. Both experiments suggest that input abstraction affects the fault detection capability. We claim that the improvements will produce a better input abstraction with which we can achieve better fault detection capability without increasing the test suite size.

Verifying the Generalization of Deep Learning to Out-of-Distribution Domains

Deep neural networks (DNNs) play a crucial role in the field of machine learning, demonstrating state-of-the-art performance across various application domains. However, despite their success, DNN-based models may occasionally exhibit challenges with generalization, i.e., may fail to handle inputs that were not encountered during training. This limitation is a significant challenge when it comes to deploying deep learning for safety-critical tasks, as well as in real-world settings characterized by substantial variability. We introduce a novel approach for harnessing DNN verification technology to identify DNN-driven decision rules that exhibit robust generalization to previously unencountered input domains. Our method assesses generalization within an input domain by measuring the level of agreement between independently trained deep neural networks for inputs in this domain. We also efficiently realize our approach by using off-the-shelf DNN verification engines, and extensively evaluate it on both supervised and unsupervised DNN benchmarks, including a deep reinforcement learning (DRL) system for Internet congestion control—demonstrating the applicability of our approach for real-world settings. Moreover, our research introduces a fresh objective for formal verification, offering the prospect of mitigating the challenges linked to deploying DNN-driven systems in real-world scenarios.

ProactiV: Studying Deep Learning Model Behavior Under Input Transformations.

Deep learning (DL) models have shown performance benefits across many applications, from classification to image-to-image translation. However, low interpretability often leads to unexpected model behavior once deployed in the real world. Usually, this unexpected behavior is because the training data domain does not reflect the deployment data domain. Identifying a model's breaking points under input conditions and domain shifts, i.e., input transformations, is essential to improve models. Although visual analytics (VA) has shown promise in studying the behavior of model outputs under continually varying inputs, existing methods mainly focus on per-class or instance-level analysis. We aim to generalize beyond classification where classes do not exist and provide a global view of model behavior under co-occurring input transformations. We present a DL model-agnostic VA method (ProactiV) to help model developers proactively study output behavior under input transformations to identify and verify breaking points. ProactiV relies on a proposed input optimization method to determine the changes to a given transformed input to achieve the desired output. The data from this optimization process allows the study of global and local model behavior under input transformations at scale. Additionally, the optimization method provides insights into the input characteristics that result in desired outputs and helps recognize model biases. We highlight how ProactiV effectively supports studying model behavior with example classification and image-to-image translation tasks.

SFIDMT-ART: A metamorphic group generation method based on Adaptive Random Testing applied to source and follow-up input domains

Context:The performance of metamorphic testing relates strongly to the quality of test cases. However, most related research has only focused on source test cases, ignoring follow-up test cases to some extent. In this paper, we identify a potential problem that may be encountered with existing metamorphic group generation algorithms. We then propose a possible solution to address this problem. Based on this solution, we design a new algorithm for generating effective source and follow-up test cases. Objective:To improve the performance (test effectiveness and efficiency) of metamorphic testing. Methods:We introduce the concept of the input-domain difference problem, which is likely to affect the performance of metamorphic group generation algorithms. We propose a new test-case distribution criterion for metamorphic testing to address this problem. Based on our proposed criterion, we further present a new metamorphic group generation algorithm, from a black-box perspective, with new distance metrics to facilitate this algorithm. Results:Our algorithm performs significantly better than existing algorithms, in terms of test effectiveness, efficiency and test-case diversity. Conclusions:Through experiments, we find that the input-domain difference problem is likely to affect the performance of metamorphic group generation algorithms. The experimental results demonstrate that our algorithm can achieve good test efficiency, effectiveness, and test-case diversity.

Bucketized Active Sampling for learning ACOPF

This paper considers optimization proxies for Optimal Power Flow (OPF), i.e., machine-learning models that approximate the input/output relationship of OPF. Recent work has focused on showing that such proxies can be of high fidelity. However, their training requires significant data, each instance necessitating the (offline) solving of an OPF. To meet the requirements of market-clearing applications, this paper proposes Bucketized Active Sampling (BAS), a novel active learning framework that aims at training the best possible OPF proxy within a time limit. BAS partitions the input domain into buckets and uses an acquisition function to determine where to sample next. By applying the same partitioning to the validation set, BAS leverages labeled validation samples in the selection of unlabeled samples. BAS also relies on an adaptive learning rate that increases and decreases over time. Experimental results demonstrate the benefits of BAS.

Active learning for adaptive surrogate model improvement in high-dimensional problems

This paper investigates a novel approach to efficiently construct and improve surrogate models in problems with high-dimensional input and output. In this approach, the principal components and corresponding features of the high-dimensional output are first identified. For each feature, the active subspace technique is used to identify a corresponding low-dimensional subspace of the input domain; then a surrogate model is built for each feature in its corresponding active subspace. A low-dimensional adaptive learning strategy is proposed to identify training samples to improve the surrogate model. In contrast to existing adaptive learning methods that focus on a scalar output or a small number of outputs, this paper addresses adaptive learning with high-dimensional input and output, with a novel learning function that balances exploration and exploitation, i.e., considering unexplored regions and high-error regions, respectively. The adaptive learning is in terms of the active variables in the low-dimensional space, and the newly added training samples can be easily mapped back to the original space for running the expensive physics model. The proposed method is demonstrated for the numerical simulation of an additive manufacturing part, with a high-dimensional field output quantity of interest (residual stress) in the component that has spatial variability due to the stochastic nature of multiple input variables (including process variables and material properties). Various factors in the adaptive learning process are investigated, including the number of training samples, range and distribution of the adaptive training samples, contributions of various errors, and the importance of exploration versus exploitation in the learning function.

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

Deep neural networks (DNNs) are widely used in various application domains such as image processing, speech recognition, and natural language processing. However, testing DNN models may be challenging due to the complexity and size of their input domain. In particular, testing DNN models often requires generating or exploring large unlabeled datasets. In practice, DNN test oracles, which identify the correct outputs for inputs, often require expensive manual effort to label test data, possibly involving multiple experts to ensure labeling correctness. In this article, we propose DeepGD , a black-box multi-objective test selection approach for DNN models. It reduces the cost of labeling by prioritizing the selection of test inputs with high fault-revealing power from large unlabeled datasets. DeepGD not only selects test inputs with high uncertainty scores to trigger as many mispredicted inputs as possible but also maximizes the probability of revealing distinct faults in the DNN model by selecting diverse mispredicted inputs. The experimental results conducted on four widely used datasets and five DNN models show that in terms of fault-revealing ability, (1) white-box, coverage-based approaches fare poorly, (2) DeepGD outperforms existing black-box test selection approaches in terms of fault detection, and (3) DeepGD also leads to better guidance for DNN model retraining when using selected inputs to augment the training set.

THE MODELLING OF A HYSTERESIS GRAPH OF PIEZOELECTRIC ELEMENTS USING DEEP LEARNING BIDIRECTIONAL LSTM

The phenomenon of hysteresis is an integral part of dynamic systems in many fields of science such as physics, chemistry, biology and many more. It describes an inherent dependence of a system state based on the history of varying number of its previous states. Hysteresis can manifest as a dynamic lag between an input signal and an output system behaviour, which depends on the degree of that system dy-namics. Modelling systems containing hysteresis is a challenging mathematical task given their highly non-linear behaviour. This paper discusses and develop a deep learning model using bidirectional LSTM (long short-term memory) for predicting voltages necessary to stimulate a piezoelectric element to produce displacements in order to cancel or minimize vibrations. The predicted voltages rely on given displacements and time domain of the initial noise input. This noise input can then be amplified to match the resonance frequency of another piezoelectric element to generate the maximum voltage capable by this later piezoelectric element. This sinusoidal voltage then travels to a piezoelectric actuator to generate displacement that can cancel the initial noise. The model resulted a coefficient of determination score of 0.99983, a loss score of 0.0092 and MSE (mean squared error) of 8.5568e-05. Created model has proven that machine learning is a viable method for hysteresis modelling and can be further improved with increased input data availability and further investigation into different deep learning algorithms.

Validity and Reliability of the Residency and Immersion Program (PRIme) Assessment Instrument in Enhancing the Quality of New Principals

The Residency and Immersion Program (PRIme) is an induction for new school principals serving their first year. PRIme was implemented in 2014, and it requires a comprehensive evaluation for program improvement. Hence, this study aims to assess the validity and reliability of the PRIme assessment instrument. The assessment data of content validity by professional experts and face validity by new principals were analyzed using CVR, CVI, and FVI methods. Meanwhile, pilot study data was obtained from 53 respondents and further analyzed using the Rasch measurement model. The questionnaire includes 102 items for assessing the domains of input (courses, coaching, and mentoring), process (type of leadership practiced), and product (leadership and organizational management) in the implementation of PRIme. The Rasch analysis results showed that the items in the questionnaire have good content validity, construct validity, and reliability. Therefore, the Rasch analysis has proven that this questionnaire can be used as an instrument to assess PRIme in an effort to enhance the quality of new principals in Malaysia.

Assessing Closed-Loop Data-Driven Wind Farm Control Strategies within a Wind Tunnel

This paper introduces a framework to assess a hybrid data-driven wake steering strategy combining the advantages of an analytical model for the initial data gathering together with a data-driven model for modelling error correction. The main building blocks of the control strategy (required input domain, data gathering and model generation) are evaluated within the wind tunnel, utilising a newly developed setup that can simulate dynamic wind direction variations and which was specifically designed to examine wake steering control strategies. The control strategy employs a machine-learning algorithm to generate a turbine and wake model based on the measurement data, allowing this approach to be extrapolated for an arbitrarily large wind farm. The machine-learning model is used to estimate the optimal yaw angle in a closed-loop format. To implement the developed control strategy, steady measurements are analysed first to determine the required input variables. This is followed by an investigation of the potential power gain and the impact of utilising a grid-search procedure. The procedure minimises adverse conditions as it collects the required data points, still achieving a power gain.

Neural style transfer of weak lensing mass maps

We propose a new generative model of projected cosmic mass density maps inferred from weak gravitational lensing observations of distant galaxies (weak lensing mass maps). We construct the model based on a neural style transfer so that it can transform Gaussian weak lensing mass maps into deeply non-Gaussian counterparts as predicted in ray-tracing lensing simulations. We develop an unpaired image-to-image translation method with Cycle-Consistent Generative Adversarial Networks (Cycle GAN), which learn efficient mapping from an input domain to a target domain. Our model is designed to enjoy important advantages; it is trainable with no need for paired simulation data, flexible to make the input domain visually meaningful, and expandable to rapidly-produce a map with a larger sky coverage than training data without additional learning. Using 10,000 lensing simulations, we find that appropriate labeling of training data based on field variance allows the model to reproduce a correct scatter in summary statistics for weak lensing mass maps. Compared with a popular log-normal model, our model improves in predicting the statistical natures of three-point correlations and local properties of rare high-density regions. We also demonstrate that our model enables us to produce a continuous map with a sky coverage of ∼166deg2 but similar non-Gaussian features to training data covering ∼12deg2 in a GPU minute. Hence, our model can be beneficial to massive productions of synthetic weak lensing mass maps, which is of great importance in future precise real-world analyses.

A flexible tropical cyclone vortex initialization technique for GFDL SHiELD

Tropical cyclone (TC) intensity forecasting poses challenges due to complex dynamical processes and data inadequacies during model initialization. This paper describes efforts to improve TC intensity prediction in the Geophysical Fluid Dynamics Laboratory (GFDL) System for High-resolution prediction on Earth-to-Local Domains (SHiELD) model by implementing a Vortex Initialization (VI) technique. The GFDL SHiELD model, relying on the Global Forecast System (GFS) analysis for initialization, faces deficiencies in initial TC structure and intensity. The VI method involves adjusting the TC vortex inherited from the GFS analysis and merging it back into the environment at the observed location, enhancing the analyzed representation of storm structure. We made modifications to the VI package implemented in the operational Hurricane Analysis and Forecast System, including handling initial condition data, reducing input domain size, and improving storm intensity enhancement. Experiments using the T-SHiELD configuration demonstrate that using VI significantly improves the representation of initial TC intensity and size, enhancing TC predictions, particularly in storm intensity and outer wind forecasts within the first 48 h.

PENGUJIAN BLACKBOX PADA SISTEM PEMESANAN UNTUK SALES ORDER DI PT BUKIT MURIA JAYA BERBASIS EQUIVALENCE PARTITIONS

This research aims to conduct blackbox testing on the sales order booking system at PT. BMJ based on the concept of equivalence partitions. The blackbox testing method is employed to assess the functionality of the system without considering its internal implementation. The study identifies representative equivalence partitions within the input and output domains of the sales order booking system and utilizes these partitions to design test cases. Through a series of tests, the system's performance is analyzed under various input conditions representing each partition. The testing results provide a better understanding of the system's reliability, identify potential defects, and offer recommendations for further improvement. This study contributes to enhancing the quality of the sales order booking system at PT. BMJ through a systematic and structured blackbox testing approach.