Static Datasets Research Articles

Region adjacency graph based GNN approach for static signature classification

Graph Neural Networks (GNNs) have gained popularity across various research fields in recent years. GNNs utilize graphs to construct an embedding that includes details about the nodes and edges in a graph’s neighborhood. In this work, a set of Region Adjacency Graphs (RAG) derives the attribute values from Static Signature (SS) images. These attribute values are used to label the nodes of the complete graph, which is formed by considering each signature as a node taken from the sample of signatures of a specific signer. The complete graph is trained by using GraphSAGE, an inductive representation learning method. This trained model helps to determine any newly introduced node (static signature to be tested) as genuine or fake. Standard static signature datasets, notably GPDSsynthetic and MCYT-75 are used to test the prevailing model. Experimental results on genuine and counterfeit signature networks demonstrate that our computed model enables a high rate of accuracy (GPDSsynthetic 99.91% and MCYT-75 99.56%) and minimum range of loss (GPDSsynthetic 0.0061 and MCYT-75 0.0070) on node classification.

Explainable data stream mining: Why the new models are better

Explainable Machine Learning brings expandability, interpretability, and accountability to Data Mining Algorithms. Existing explanation frameworks focus on explaining the decision process of a single model in a static dataset. However, in data stream mining changes in data distribution over time, called concept drift, may require updating the learning models to reflect the current data environment. It is therefore important to go beyond static models and understand what has changed among the learning models before and after a concept drift. We propose a Data Stream Explanability framework (DSE) that works together with a typical data stream mining framework where support vector machine models are used. DSE aims to help non-expert users understand model dynamics in a concept drifting data stream. DSE visualizes differences between SVM models before and after concept drift, to produce explanations on why the new model fits the data better. A survey was carried out between expert and non-expert users on the effectiveness of the framework. Although results showed non-expert users on average responded with less understanding of the issue compared to expert users, the difference is not statistically significant. This indicates that DSE successfully brings the explanability of model change to non-expert users.

Fair-CMNB: Advancing Fairness-Aware Stream Learning with Naïve Bayes and Multi-Objective Optimization

Fairness-aware mining of data streams is a challenging concern in the contemporary domain of machine learning. Many stream learning algorithms are used to replace humans in critical decision-making processes, e.g., hiring staff, assessing credit risk, etc. This calls for handling massive amounts of incoming information with minimal response delay while ensuring fair and high-quality decisions. Although deep learning has achieved success in various domains, its computational complexity may hinder real-time processing, making traditional algorithms more suitable. In this context, we propose a novel adaptation of Naïve Bayes to mitigate discrimination embedded in the streams while maintaining high predictive performance through multi-objective optimization (MOO). Class imbalance is an inherent problem in discrimination-aware learning paradigms. To deal with class imbalance, we propose a dynamic instance weighting module that gives more importance to new instances and less importance to obsolete instances based on their membership in a minority or majority class. We have conducted experiments on a range of streaming and static datasets and concluded that our proposed methodology outperforms existing state-of-the-art (SoTA) fairness-aware methods in terms of both discrimination score and balanced accuracy.

Brain-inspired chaotic spiking backpropagation.

Spiking neural networks (SNNs) have superior energy efficiency due to their spiking signal transmission, which mimics biological nervous systems, but they are difficult to train effectively. Although surrogate gradient-based methods offer a workable solution, trained SNNs frequently fall into local minima because they are still primarily based on gradient dynamics. Inspired by the chaotic dynamics in animal brain learning, we propose a chaotic spiking backpropagation (CSBP) method that introduces a loss function to generate brain-like chaotic dynamics and further takes advantage of the ergodic and pseudo-random nature to make SNN learning effective and robust. From a computational viewpoint, we found that CSBP significantly outperforms current state-of-the-art methods on both neuromorphic data sets (e.g. DVS-CIFAR10 and DVS-Gesture) and large-scale static data sets (e.g. CIFAR100 and ImageNet) in terms of accuracy and robustness. From a theoretical viewpoint, we show that the learning process of CSBP is initially chaotic, then subject to various bifurcations and eventually converges to gradient dynamics, consistently with the observation of animal brain activity. Our work provides a superior core tool for direct SNN training and offers new insights into understanding the learning process of a biological brain.

GMFGRN: a matrix factorization and graph neural network approach for gene regulatory network inference.

The recent advances of single-cell RNA sequencing (scRNA-seq) have enabled reliable profiling of gene expression at the single-cell level, providing opportunities for accurate inference of gene regulatory networks (GRNs) on scRNA-seq data. Most methods for inferring GRNs suffer from the inability to eliminate transitive interactions or necessitate expensive computational resources. To address these, we present a novel method, termed GMFGRN, for accurate graph neural network (GNN)-based GRN inference from scRNA-seq data. GMFGRN employs GNN for matrix factorization and learns representative embeddings for genes. For transcription factor-gene pairs, it utilizes the learned embeddings to determine whether they interact with each other. The extensive suite of benchmarking experiments encompassing eight static scRNA-seq datasets alongside several state-of-the-art methods demonstrated mean improvements of 1.9 and 2.5% over the runner-up in area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). In addition, across four time-series datasets, maximum enhancements of 2.4 and 1.3% in AUROC and AUPRC were observed in comparison to the runner-up. Moreover, GMFGRN requires significantly less training time and memory consumption, with time and memory consumed <10% compared to the second-best method. These findings underscore the substantial potential of GMFGRN in the inference of GRNs. It is publicly available at https://github.com/Lishuoyy/GMFGRN.

D $i$ E$v$D: Disruptive Event Detection from Dynamic Datastreams using Continual Machine Learning: A Case Study with Twitter

Identifying disruptive events (riots, protests, natural calamities) from social media is important for maintaining social order and addressing geopolitical concerns. Existing works on identifying disruptive events use classical machine learning (ML) models on static datasets. However, social networks are dynamic entities and cannot be practically modeled using static techniques. A viable alternative is the emerging Continual Machine Learning (CML) approach which applies the knowledge acquired from the past to learn future tasks. However, existing CML techniques are trained and tested on static data and are incapable of handling real-time data obtained from dynamic environments. This paper presents a novel D <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$i$</tex-math></inline-formula> E <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$v$</tex-math></inline-formula> D framework for disruptive event detection using Continual Machine Learning (CML) specifically for dynamic data streams. We have used Twitter social media as a case study of the real-time and dynamic data provider. To the best of our knowledge, this is the first attempt to use CML for socially disruptive event detection. Comprehensive performance analysis show that our framework effectively identifies disruptive events with 98% accuracy and can classify them with an average incremental accuracy of 76.8%. Moreover, computational analysis is performed to establish the effectiveness of D <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$i$</tex-math></inline-formula> E <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$v$</tex-math></inline-formula> D by applying various language models and statistical tests.

On Transforming Reinforcement Learning With Transformers: The Development Trajectory.

Transformers, originally devised for natural language processing (NLP), have also produced significant successes in computer vision (CV). Due to their strong expression power, researchers are investigating ways to deploy transformers for reinforcement learning (RL), and transformer-based models have manifested their potential in representative RL benchmarks. In this paper, we collect and dissect recent advances concerning the transformation of RL with transformers (transformer-based RL (TRL)) to explore the development trajectory and future trends of this field. We group the existing developments into two categories: architecture enhancements and trajectory optimizations, and examine the main applications of TRL in robotic manipulation, text-based games (TBGs), navigation, and autonomous driving. Architecture enhancement methods consider how to apply the powerful transformer structure to RL problems under the traditional RL framework, facilitating more precise modeling of agents and environments compared to traditional deep RL techniques. However, these methods are still limited by the inherent defects of traditional RL algorithms, such as bootstrapping and the "deadly triad". Trajectory optimization methods treat RL problems as sequence modeling problems and train a joint state-action model over entire trajectories under the behavior cloning framework; such approaches are able to extract policies from static datasets and fully use the long-sequence modeling capabilities of transformers. Given these advancements, the limitations and challenges in TRL are reviewed and proposals regarding future research directions are discussed. We hope that this survey can provide a detailed introduction to TRL and motivate future research in this rapidly developing field.

Gotham Testbed: A Reproducible IoT Testbed for Security Experiments and Dataset Generation

The scarcity of available Internet of Things (IoT) datasets remains a limiting factor in developing machine learning based security systems. Static datasets get outdated due to evolving IoT threat landscape. Meanwhile, the testbeds used to generate them are rarely published. This paper presents the Gotham testbed, a reproducible and flexible network security testbed, implemented as a middleware over the GNS3 emulator, that is extendable to accommodate new emulated devices, services or attackers. The testbed is used to build an IoT scenario composed of 100 emulated devices communicating via MQTT, CoAP and RTSP protocols in a topology composed of 30 switches and 10 routers. The scenario presents three threat actors, including the entire Mirai botnet lifecycle and additional red-teaming tools performing DoS, scanning and various attacks targeting the MQTT and CoAP protocols. The generated network traffic and application logs can be used to capture datasets containing legitimate and attacking traces. We hope that researchers can leverage the testbed and adapt it to include other types of devices and state-of-the-art attacks to generate new datasets that reflect the current threat landscape and IoT protocols. The source code to reproduce the scenario is publicly accessible.

TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks.

Spiking neural networks (SNNs) are attracting widespread interest due to their biological plausibility, energy efficiency, and powerful spatiotemporal information representation ability. Given the critical role of attention mechanisms in enhancing neural network performance, the integration of SNNs and attention mechanisms exhibits tremendous potential to deliver energy-efficient and high-performance computing paradigms. In this article, we present a novel temporal-channel joint attention mechanism for SNNs, referred to as TCJA-SNN. The proposed TCJA-SNN framework can effectively assess the significance of spike sequence from both spatial and temporal dimensions. More specifically, our essential technical contribution lies on: 1) we employ the squeeze operation to compress the spike stream into an average matrix. Then, we leverage two local attention mechanisms based on efficient 1-D convolutions to facilitate comprehensive feature extraction at the temporal and channel levels independently and 2) we introduce the cross-convolutional fusion (CCF) layer as a novel approach to model the interdependencies between the temporal and channel scopes. This layer effectively breaks the independence of these two dimensions and enables the interaction between features. Experimental results demonstrate that the proposed TCJA-SNN outperforms the state-of-the-art (SOTA) on all standard static and neuromorphic datasets, including Fashion-MNIST, CIFAR10, CIFAR100, CIFAR10-DVS, N-Caltech 101, and DVS128 Gesture. Furthermore, we effectively apply the TCJA-SNN framework to image generation tasks by leveraging a variation autoencoder. To the best of our knowledge, this study is the first instance where the SNN-attention mechanism has been employed for high-level classification and low-level generation tasks. Our implementation codes are available at https://github.com/ridgerchu/TCJA.

COPSRO: An Offline Empirical Game Theoretic Method With Conservative Critic.

This article studies how to learn approximate Nash equilibrium (NE) from static historical datasets by empirical game-theoretic analysis (EGTA), which provides a simulation-based framework to model complex multiagent interactions. Generally, EGTA requires plentiful interactions with the environment or simulator to estimate a cogent and tractable game model approximating the underlying game. However, these exploratory interactions often suffer from low data utilization efficiency and may not be feasible in risk-sensitive applications. To address these problems, this article investigates a new EGTA paradigm for offline settings and introduces a novel algorithm called conservative offline policy space response oracle (COPSRO) to identify NE from fixed datasets without active data collection. COPSRO initiates by extracting a set of strategies from the offline dataset to construct an overcomplete strategy population, achieving an approximation to the policy space of the original game. Then, COPSRO integrates the conservative critic (CC) to tackle the challenge of overestimation inherent in offline learning scenarios. Additionally, it devises the offline NE solver to iteratively compute approximate NE. Consequently, COPSRO can ascertain equilibrium strategies without real-world interaction, markedly enhancing its utility in risk-averse settings. This article provides both theoretical analysis and empirical evaluation to demonstrate the effectiveness and superiority of COPSRO across various real-world tasks in the offline setting. Our method surpasses existing approaches in terms of convergence and exploitability, especially when the coverage ration of dataset is low (20% or 10%).

Vegetation and inundation characteristics of waterbird breeding sites in the Murray–Darling Basin, Australia

Context The Murray–Darling Basin serves as a crucial habitat for aggregating waterbirds; however, decades of large-scale regulation of rivers and water resources have adversely affected waterbird breeding in the Basin. Aims To understand the characteristics of wetlands that attract and support aggregating waterbirds, focusing on identifying environmental conditions conducive to waterbird breeding. Methods In total, 52 wetland sites across the Murray–Darling Basin, with high waterbird abundances, were identified, of which 26 supported waterbird breeding. Classification models were developed using temporally static and dynamic environmental datasets to discern wetland characteristics associated with waterbird breeding. Key results Analyses showed that wetlands supporting waterbird breeding contained a maximum inundated area of ‘other shrublands’ exceeding 3.635 km2 and variation in normalised difference vegetation index, possibly reflective of a ‘boom and bust’ ecological response. Conclusions Understanding the habitat requirements of wetlands to prompt waterbird breeding is critical for effective environmental water management and conservation strategies. Implications Targeted wetland management and environmental water allocation to support waterbird breeding populations in the Murray–Darling Basin is essential for continued waterbird breeding. There is a need for continued research to refine management strategies and ensure the long-term sustainability of waterbird populations in the face of ongoing environmental challenges.

Intelligent Target Classification Algorithm for 77G Radar Based on Correction Data Set

Traffic participant classification is crucial in autonomous driving perception. Millimeter wave radio detection and ranging radar is a cost-effective and powerful method to perform the task in adverse traffic scenarios, especially in bad inclement weather (e.g. fog, snow and rain) and poor lighting conditions. This paper presents an intelligent target classification algorithm for 77G radar based on correction data set. First, in order to handle the problem that the original data set may easily be interfered by obstacles, the angle information is filtered by analyzing the spatial information of radar signals, which means the interference clutter of obstacles can be effectively removed. Second, the primary data set is corrected using the significant difference between the micro-Doppler of the human body and car. Finally, the characteristic information of the radar signal is extracted, including distance, speed, orientation, micro-Doppler and reflection intensity, and the obtained data sets containing three types of targets (vehicles, human bodies and obstacles) are generated. The generated dynamic and static data sets are collected by sufficient experiments to construct deep learning classification models. The results show that the classification accuracy is improved by the measure of data set correction.

Geometry effects on wind fields and consequent wave generation in the Strait of Hormuz: A case study

This study investigates the impact of surface wind forcing on wave conditions in the Strait of Hormuz, a region with complex wave interactions. The WRF model is used to simulate the wind field with higher accuracy, enabling the generation of waves for both normal and storm conditions in 2011. A sensitivity analysis examines the WRF model's simulated wind field with variations in initial and boundary conditions, spatial resolutions, and adopting two static topographic datasets. Comparisons between simulated wave parameters and observed data from ADCPs at two stations flanking the strait reveal the importance of accurate wind forcing for obtaining a realistic estimation of wave conditions. Wave errors are found to be influenced by fetch length, with larger errors observed for shorter fetches (∼100 km). However, these errors gradually decrease as the distance from the coast increases. The study emphasizes the importance of incorporating accurate wind data and considering fetch characteristics when simulating wave conditions, which can enhance maritime safety, coastal engineering, and offshore operations in the Strait of Hormuz and other similar regions worldwide. In conclusion, small-scale wind and waves features over the Strait of Hormuz can be better captured with a combination of higher-resolution wind modeling and wave simulation grids with higher spatial resolution.

Ensuring the accuracy of indirect nonlinear dynamic reduced-order models

Numerous powerful methods exist for developing reduced-order models (ROMs) from finite element (FE) models. Ensuring the accuracy of these ROMs is essential; however, the validation using dynamic responses is expensive. In this work, we propose a method to ensure the accuracy of ROMs without extra dynamic FE simulations. It has been shown that the well-established implicit condensation and expansion (ICE) method can produce an accurate ROM when the FE model’s static behaviour are captured accurately. However, this is achieved via a fitting procedure, which may be sensitive to the selection of load cases and ROM’s order, especially in the multi-mode case. To alleviate this difficulty, we define an error metric that can evaluate the ROM’s fitting error efficiently within the displacement range, specified by a given energy level. Based on the fitting result, the proposed method provides a strategy to enrich the static dataset, i.e. additional load cases are found until the ROM’s accuracy reaches the required level. Extending this to the higher-order and multi-mode cases, some extra constraints are incorporated into the standard fitting procedure to make the proposed method more robust. A curved beam is utilised to validate the proposed method, and the results show that the method can robustly ensure the accuracy of the static fitting of ROMs.

Creating a multi-linked dynamic dataset: a case study of plant genera named for women

A discussion on social media led to the formation of a multidisciplinary group working on this project to highlight women’s contributions to science. The role of marginalised groups in science has been a topic of much discussion, but data on these contributions are largely lacking. Our motivation for the development of this dataset was not only to highlight names of plant genera that honour women, but to enrich this information with data that would allow the names, roles and lives of these women to be shared more widely with others, both researchers and data sources like Wikidata. Amplification of the contributions of women to botany through multiple means will enable the community to better recognise and celebrate the role of this particular marginalised group in the history and development of science. The innovative approach of our study resulted in a dataset that is dynamic, expansive and widely shared. We have published a static dataset with this paper and have also created a dynamic dataset by linking flowering plant genera and the women in whose honour those genera were named in Wikidata. This concurrent addition of the data to Wikidata, a linked open data repository, enabled it to be enriched, queried and proactively shared during the whole process of dataset creation and into the future. This innovative workflow allowed wide, open participation throughout the research process. The methodology and workflows applied can be used to create future datasets celebrating and amplifying the contributions of marginalised groups in science.

Automated machine learning for deep learning based malware detection

Deep learning (DL) has proven to be effective in detecting sophisticated malware that is constantly evolving. Even though deep learning has alleviated the feature engineering problem, finding the most optimal DL model's architecture and set of hyper-parameters, remains a challenge that requires domain expertise. In addition, many of the proposed state-of-the-art models are very complex and may not be the best fit for different datasets. A promising approach, known as Automated Machine Learning (AutoML), can reduce the domain expertise required to develop custom DL models by automating the ML pipeline key components, namely hyperparameter optimization and neural architecture search (NAS). AutoML reduces the amount of human trial-and-error involved in designing DL models, and in more recent implementations can find new model architectures with relatively low computational overhead.Research on the feasibility of using AutoML for malware detection is very limited. This work provides a comprehensive analysis and insights on using AutoML for both static and online malware detection. For static, our analysis is performed on two widely used malware datasets: SOREL-20M to demonstrate efficacy on large datasets; and EMBER-2018, a smaller dataset specifically curated to hinder the performance of machine learning models. In addition, we show the effects of tuning the NAS process parameters on finding a more optimal malware detection model on these static analysis datasets. Further, we also demonstrate that AutoML is performant in online malware detection scenarios using Convolutional Neural Networks (CNNs) for cloud IaaS. We compare an AutoML technique to six existing state-of-the-art CNNs using a newly generated online malware dataset with and without other applications running in the background during malware execution. We show that the AutoML technique is more performant than the state-of-the-art CNNs with little overhead in finding the architecture. In general, our experimental results show that the performance of AutoML based static and online malware detection models are on par or even better than state-of-the-art models or hand-designed models presented in literature.

Exploiting nonlinear dendritic adaptive computation in training deep Spiking Neural Networks

Inspired by the information transmission process in the brain, Spiking Neural Networks (SNNs) have gained considerable attention due to their event-driven nature. However, as the network structure grows complex, managing the spiking behavior within the network becomes challenging. Networks with excessively dense or sparse spikes fail to transmit sufficient information, inhibiting SNNs from exhibiting superior performance. Current SNNs linearly sum presynaptic information in postsynaptic neurons, overlooking the adaptive adjustment effect of dendrites on information processing. In this study, we introduce the Dendritic Spatial Gating Module (DSGM), which scales and translates the input, reducing the loss incurred when transforming the continuous membrane potential into discrete spikes. Simultaneously, by implementing the Dendritic Temporal Adjust Module (DTAM), dendrites assign different importance to inputs of different time steps, facilitating the establishment of the temporal dependency of spiking neurons and effectively integrating multi-step time information. The fusion of these two modules results in a more balanced spike representation within the network, significantly enhancing the neural network’s performance. This approach has achieved state-of-the-art performance on static image datasets, including CIFAR10 and CIFAR100, as well as event datasets like DVS-CIFAR10, DVS-Gesture, and N-Caltech101. It also demonstrates competitive performance compared to the current state-of-the-art on the ImageNet dataset.

Transformer-based convolutional forgetting knowledge tracking

Knowledge tracking is to analyze the mastery of students' knowledge through the learning track. This is very important for online education, since it can determine a learner’s current knowledge level by analyzing the learning history and then make recommendations for future learning. In the past, the commonly used model for knowledge tracking is the convolutional neural network, but it has long-term sequence dependencies. With the invention of Transformer, it has excellent performance in long-sequence modeling by virtue of the attention mechanism, and is gradually introduced into the field of knowledge tracking. However, through our research, some knowledge tracking data sets have a large number of continuous and repetitive training, which will cause Transformer model to ignore the potential connections between some knowledge points. To overcome this problem, we introduce a convolutional attention mechanism to help the model perceive contextual information better. In addition, we simulate the forgetting phenomenon of students during the learning process by calculating the forgetting factor, and fuse it with the weight matrix generated by the model to improve the accuracy of the model. As a result, a Transformer-based Convolutional Forgetting Knowledge Tracking (TCFKT) model is presented in this paper. According to the experimental results conducted on the real world ASSITments2012, ASSISTments2017, KDD a, STATIC datasets, the TCFKT model outperforms other knowledge tracking models.

Dynamic scRNA-seq of live human pancreatic slices reveals functional endocrine cell neogenesis through an intermediate ducto-acinar stage

Human pancreatic plasticity is implied from multiple single-cell RNA sequencing (scRNA-seq) studies. However, these have been invariably based on static datasets from which fate trajectories can only be inferred using pseudotemporal estimations. Furthermore, the analysis of isolated islets has resulted in a drastic underrepresentation of other cell types, hindering our ability to interrogate exocrine-endocrine interactions. The long-term culture of human pancreatic slices (HPSs) has presented the field with an opportunity to dynamically track tissue plasticity at the single-cell level. Combining datasets from same-donor HPSs at different time points, with or without a known regenerative stimulus (BMP signaling), led to integrated single-cell datasets storing true temporal or treatment-dependent information. This integration revealed population shifts consistent with ductal progenitor activation, blurring of ductal/acinar boundaries, formation of ducto-acinar-endocrine differentiation axes, and detection of transitional insulin-producing cells. This study provides the first longitudinal scRNA-seq analysis of whole human pancreatic tissue, confirming its plasticity in a dynamic fashion.

Conservative network for offline reinforcement learning

Offline reinforcement learning (RL) aims to learn policies from static datasets. The value overestimation of out-of-distribution (OOD) actions makes it difficult to directly apply general RL methods in the offline setting. To overcome this problem, many works focus on estimating the value function conservatively or pessimistically. However, existing methods require additional OOD sampling or uncertainty estimation to underestimate OOD values, making them complex and vulnerable to hyperparameters. Is it possible to design a specific value function that can automatically be conservative on OOD samples? In this study, we reveal the anti-conservation property of the widely used ReLU network under certain conditions and explain the reason theoretically. Based on the analysis of the ReLU network, we propose a novel neural network architecture that pushes down the value of those samples far away from the datasets; we call this kind of new architecture the Conservative Network (ConsNet). Based on ConsNet, a new offline RL algorithm with simple implementation and high performance is proposed. Since we can obtain additional conservation from the ConsNet itself, by integrating the ConsNet into several existing offline RL methods, we find that it can significantly improve the performance or reduce the original algorithm complexity. With its simplicity and superiority, we hope that ConsNet could be a new fundamental network architecture for offline RL.