Real-world Datasets Research Articles

Event-driven nearshore and shoreline coastline detection on SpiNNaker neuromorphic hardware

Abstract Coastline detection is vital for coastal management, involving frequent observation and assessment to understand coastal dynamics and inform decisions on environmental protection. 
Continuous streaming of high-resolution images demands robust data processing and storage solutions to manage large datasets efficiently, posing challenges that require innovative solutions for real-time analysis and meaningful insights extraction.
This work leverages low-latency event-based vision sensors coupled with neuromorphic hardware in an attempt to decrease a two-fold challenge, reducing the computational burden to ~0.375 mW whilst obtaining a coastline detection map in as little as 20 ms.
The proposed Spiking Neural Network (SNN) runs on the SpiNNaker neuromorphic platform using a total of 18040 neurons reaching 98.33% accuracy.
The model has been characterised and evaluated by computing the accuracy of Intersection over Union (IoU) scores over the ground truth of a real-world coastline dataset across different time windows. The system's robustness was further assessed by evaluating its ability to avoid coastline detection in non-coastline profiles and funny shapes, achieving a success rate of 97.3%.

Traffic Speed Prediction Based on Dynamic Spatial-Temporal Attention Graph Convolutional Network

Abstract The prediction of traffic speed represents a crucial branch within the broader field of traffic forecasting. Its objective is to forecast the average speed or speed distribution on road segments or within a network for a designated future time period . The prediction results can provide scientific decision support for traffic management departments to effectively control and guide traffic flow. However, given the limitations of urban road network topology and its dynamic temporal shifts, traffic prediction has consistently been deemed a perpetual scientific challenge. To simultaneously encapsulate both spatial and temporal dependencies, we introduce an innovative Dynamic Spatio-Temporal Attention Network (DSTANet). This model integrates spatio-temporal attention with graph convolutional networks and temporal convolution to enhance its analytical capabilities. Specifically, the graph convolutional network is leveraged to learn the intricate topology for capturing spatial relations, while temporal convolution is employed to ascertain the dynamic fluctuations of traffic data, thereby acquiring temporal dependencies. Subsequently, we implement our model to forecast traffic patterns within the framework of road networks. Experimental results demonstrate that our model effectively captures the spatio-temporal dependencies among roadway junctions derived from traffic data, outperforming state-of-the-art benchmarks on real-world traffic datasets in terms of predictive accuracy.

ScNODE : generative model for temporal single cell transcriptomic data prediction.

Measurement of single-cell gene expression at different timepoints enables the study of cell development. However, due to the resource constraints and technical challenges associated with the single-cell experiments, researchers can only profile gene expression at discrete and sparsely sampled timepoints. This missing timepoint information impedes downstream cell developmental analyses. We propose scNODE, an end-to-end deep learning model that can predict in silico single-cell gene expression at unobserved timepoints. scNODE integrates a variational autoencoder with neural ordinary differential equations to predict gene expression using a continuous and nonlinear latent space. Importantly, we incorporate a dynamic regularization term to learn a latent space that is robust against distribution shifts when predicting single-cell gene expression at unobserved timepoints. Our evaluations on three real-world scRNA-seq datasets show that scNODE achieves higher predictive performance than state-of-the-art methods. We further demonstrate that scNODE's predictions help cell trajectory inference under the missing timepoint paradigm and the learned latent space is useful for in silico perturbation analysis of relevant genes along a developmental cell path. The data and code are publicly available at https://github.com/rsinghlab/scNODE.

Convolutional Neural Network-Based Lane-Change Strategy via Motion Image Representation for Automated and Connected Vehicles.

The lane-change decision-making module of automated and connected vehicles (ACVs) is one of the most crucial and challenging issues to be addressed. Motivated by human beings' underlying driving paradigm and the convolutional neural network's (CNN) dramatic capability of extracting features and learning strategies, this article proposes a CNN-based lane-change decision-making method via the dynamic motion image representation. Human drivers take proper driving maneuvers after they subconsciously construct the dynamic traffic scene representation in their brains, so this study first proposes the dynamic motion image representation method to reveal informative traffic situations in the motion-sensitive area (MSA), which provides a full view of surrounding cars. Then, this article develops a CNN model to extract the underlying features and learn driving policies from labeled datasets of MSA motion images. Besides, a safety-constrained layer is added to avoid vehicle collisions. We build a simulation platform based on the simulation of urban mobility (SUMO) to collect traffic datasets and test our proposed method. In addition, real-world traffic datasets are also involved to further investigate the proposed method's performance. The rule-based strategy and reinforcement learning (RL)-based method are used to compare with our approach. All results demonstrate that the proposed method performs lane-change decision-making much better than prevailing methods, which suggests our scheme has huge potential to accelerate the deployment of ACVs and is worth further study.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Multi-scale RWKV with 2-dimensional temporal convolutional network for short-term photovoltaic power forecasting

Improving the accuracy of Photovoltaic (PV) power forecasting is crucial for optimizing the schedule of power stations and maintaining the grid stability. However, PV power generation exhibits complex periodicity and is significantly influenced by weather conditions, introducing instability, intermittency, and randomness, making accurate PV power forecasting a challenging task. Therefore, this study proposes a multi-scale Receptance Weighted Key Value with 2-Dimensional Temporal Convolutional Network (MSRWKV-2DTCN) for PV power forecasting, which can learn periodicity and interdependencies of data and improve forecasting accuracy. Firstly, the proposed model identifies multi-periodicity of PV power data with the Fast Fourier Transform (FFT). Subsequently, we combine these identified periods with the canonical time mixing block of Receptance Weighted Key Value (RWKV) and introduce a multi-scale time mixing block to learn periodicity of data. Finally, to explore complex interdependencies of historical data, we replace the channel-mixing block of RWKV with a multi-scale 2-Dimensional Temporal Convolutional Network (2D TCN). Experiments were conducted on real-world datasets collected from Yulara solar system in Australia to validate the performance of the proposed model. Comparisons with other PV power forecasting models and ablation studies confirm that the MSRWKV-2DTCN achieves higher accuracy in short-term PV power forecasting.

Physics-Informed Explainable Continual Learning on Graphs.

Temporal graph learning has attracted great attention with its ability to deal with dynamic graphs. Although current methods are reasonably accurate, most of them are unexplainable due to their black-box nature. It remains a challenge to explain how temporal graph learning models adapt to information evolution. Furthermore, with the increasing application of artificial intelligence in various scientific domains, such as chemistry and biomedicine, the importance of delivering not only precise outcomes but also offering explanations regarding the learning models becomes paramount. This transparency aids users in comprehending the decision-making procedures and instills greater confidence in the generated models. To address this issue, this article proposes a novel physics-informed explainable continual learning (PiECL), focusing on temporal graphs. Our proposed method utilizes physical and mathematical algorithms to quantify the disturbance of new data to previous knowledge for obtaining changed information over time. As the proposed model is based on theories in physics, it can provide a transparent underlying mechanism for information evolution detection, thus enhancing explainability. The experimental results on three real-world datasets demonstrate that PiECL can explain the learning process, and the generated model outperforms other state-of-the-art methods. PiECL shows tremendous potential for explaining temporal graph learning in various scientific contexts.

A privacy-preserving framework with multi-modal data for cross-domain recommendation

Cross-domain recommendation (CDR) aims to enhance the recommendation accuracy in a target domain with sparse data by leveraging rich information in a source domain, thereby addressing the data-sparsity problem. Some existing CDR methods highlight the advantages of extracting domain-common and domain-specific features to learn comprehensive user and item representations. However, these methods cannot effectively disentangle these components, as they often rely on simple user-item historical interaction information (such as ratings, clicks, and browsing), neglecting the rich multi-modal features. In addition, they do not protect user-sensitive data from potential leakage during knowledge transfer between domains. To address these challenges, we propose a Privacy-Preserving Framework with Multi-Modal Data for Cross-Domain Recommendation, called P2M2-CDR. Specifically, we first design a multi-modal disentangled encoder that utilizes multi-modal information to disentangle more informative domain-common and domain-specific embeddings. Furthermore, we introduce a privacy-preserving decoder to mitigate user privacy leakage during knowledge transfer. Local differential privacy (LDP) is used to obfuscate disentangled embeddings before the inter-domain exchange, thereby enhancing privacy protection. To ensure both consistency and differentiation among these obfuscated disentangled embeddings, we incorporate contrastive learning-based domain-inter and domain-intra losses. Extensive experiments conducted on six CDR tasks from two real-world datasets demonstrate that P2M2-CDR outperforms other state-of-the-art single- and cross-domain baselines. The code is available at https://github.com/Lili1013/P2M2-CDR.

GWO-ANFIS-RD3PG: A reinforcement learning approach with dynamic adjustment and dual replay mechanism for building energy forecasting

Reliable prediction of building energy consumption is essential for effective and sustainable energy management. Traditional forecasting methods, however, face challenges when dealing with sudden changes in energy consumption. To address the above-mentioned issue, we introduce the GWO-ANFIS-RD3PG prediction model in this paper. This model aims to ensure global forecasting accuracy while minimizing prediction errors at sudden change points. A Grey Wolf Optimization (GWO) algorithm with the Adaptive Neuro-Fuzzy Inference System (ANFIS) is proposed to predict the energy consumption type and its fluctuation magnitude for the next time step. Within the Recurrent Dual Experience Replay Buffer DDPG (RD3PG) prediction module, we employ Long Short-Term Memory (LSTM), as the actor network in the Deep Deterministic Policy Gradient (DDPG), to account for long-term dependencies in time series data. Notably, we introduce an adaptive action modification mechanism that allows the agent to optimize actions based on the output of the aforementioned ANFIS, enabling real-time adjustments during sudden change. To further enhance the model’s performance, we adopt a dual experience replay mechanism to store data samples from sudden change points, capturing insight information during these anomalies. Experimental results on two real-world datasets demonstrate that this method reduces MAE by 5.81% and 13.47%, and RMSE by 5.69% and 15.21% compared to the state-of-art models, showing the significance of the proposed method for energy efficiency improvement in real-world building operations.

Enhancing Integrated Sensing and Communication (ISAC) Performance for a Searching–Deciding Alternation Radar-Comm System with Multi-Dimension Point Cloud Data

In developing modern intelligent transportation systems, integrated sensing and communication (ISAC) technology has become an efficient and promising method for vehicle road services. To enhance traffic safety and efficiency through real-time interaction between vehicles and roads, this paper proposes a searching–deciding scheme for an alternation radar-communication (radar-comm) system. Firstly, its communication performance is derived for a given detection probability. Then, we process the echo data from real-world millimeter-wave (mmWave) radar into four-dimensional (4D) point cloud datasets and thus separate different hybrid modes of single-vehicle and vehicle fleets into three types of scenes. Based on these datasets, an efficient labeling method is proposed to assist accurate vehicle target detection. Finally, a novel vehicle detection scheme is proposed to classify various scenes and accurately detect vehicle targets based on deep learning methods. Extensive experiments on collected real-world datasets demonstrate that compared to benchmarks, the proposed scheme obtains substantial radar performance and achieves competitive communication performance.

GAP-LSTM: Graph-Based Autocorrelation Preserving Networks for Geo-Distributed Forecasting.

Forecasting methods are important decision support tools in geo-distributed sensor networks. However, challenges such as the multivariate nature of data, the existence of multiple nodes, and the presence of spatio-temporal autocorrelation increase the complexity of the task. Existing forecasting methods are unable to address these challenges in a combined manner, resulting in a suboptimal model accuracy. In this article, we propose GAP-LSTM, a novel geo-distributed forecasting method that leverages the synergic interaction of graph convolution, attention-based long short-term memory (LSTM), 2-D-convolution, and latent memory states to effectively exploit spatio-temporal autocorrelation in multivariate data generated by multiple nodes, resulting in improved modeling capabilities. Our extensive evaluation, involving real-world datasets on traffic, energy, and pollution domains, showcases the ability of our method to outperform state-of-the-art forecasting methods. An ablation study confirms that all method components provide a positive contribution to the accuracy of the extracted forecasts. The method also provides an interpretable visualization that complements forecasts with additional insights for domain experts.

Fast Multiview Clustering by Optimal Graph Mining.

Multiview clustering (MVC) aims to exploit heterogeneous information from different sources and was extensively investigated in the past decade. However, far less attention has been paid to handling large-scale multiview data. In this brief, we fill this gap and propose a fast multiview clustering by an optimal graph mining model to handle large-scale data. We mine a consistent clustering structure from landmark-based graphs of different views, from which the optimal graph based on the one-hot encoding of cluster labels is recovered. Our model is parameter-free, so intractable hyperparameter tuning is avoided. An efficient algorithm of linear complexity to the number of samples is developed to solve the optimization problems. Extensive experiments on real-world datasets of various scales demonstrate the superiority of our proposal.

Bit Reduction for Locality-Sensitive Hashing.

Locality-sensitive hashing (LSH) has gained ever-increasing popularity in similarity search for large-scale data. It has competitive search performance when the number of generated hash bits is large, reversely bringing adverse dilemmas for its wide applications. The first purpose of this work is to introduce a novel hash bit reduction schema for hashing techniques to derive shorter binary codes, which has not yet received sufficient concerns. To briefly show how the reduction schema works, the second purpose is to present an effective bit reduction method for LSH under the reduction schema. Specifically, after the hash bits are generated by LSH, they will be put into bit pool as candidates. Then mutual information and data labels are exploited to measure the correlation and structural properties between the hash bits, respectively. Eventually, highly correlated and redundant hash bits can be distinguished and then removed accordingly, without deteriorating the performance greatly. The advantages of our reduction method include that it can not only reduce the number of hash bits effectively but also boost retrieval performance of LSH, making it more appealing and practical in real-world applications. Comprehensive experiments were conducted on three public real-world datasets. The experimental results with representative bit selection methods and the state-of-the-art hashing algorithms demonstrate that the proposed method has encouraging and competitive performance.

AI-empowered consumer behavior analysis for trustworthy track recommendation over musical dance electronic products

—In the digital music era, accurate and trustworthy track recommendations for musical dance electronic products are becoming increasingly important to improve user experiences and attract more consumers. Consumer behavior modeling is critical in user interest learning and has been extensively used in recommender systems to improve recommendation accuracy. This paper proposes a novel AI-empowered consumer behavior analysis method for trustworthy track recommendations over musical dance electronic products. Specifically, we first model consumer behavior by integrating collaborative filtering and a hidden Markov model to capture the key interactive patterns between consumers and musical dance electronic products. Then, we develop a trustworthy track recommendation method based on multi-layer attention representation learning, which leverages scattering transform for audio preprocessing and attention-based independent recurrent neural networks for encoding user preferences and product features. Extensive experiments on real-world datasets demonstrate the superiority of our proposed method in terms of recommendation accuracy and trustworthiness.

EXPRESS: A Machine Learning Approach to Solve the E-commerce Box-Sizing Problem

E-commerce packages are notorious for their inefficient usage of space. More than one-quarter volume of a typical e-commerce package comprises air and filler material. The inefficient usage of space significantly reduces the transportation and distribution capacity increasing the operational costs. Therefore, designing an optimal set of packaging box sizes is crucial for improving efficiency. We present the first learning-based framework to determine the optimal packaging box sizes. In particular, we propose a three-stage optimization framework that combines unsupervised learning, reinforcement learning, and tree search to design box sizes. The package optimization problem is formulated into a sequential decision-making task called the box-sizing game. A neural network agent is then designed to play the game and learn heuristic rules to solve the problem. In addition, a tree-search operator is developed to improve the performance of the learned networks. When benchmarked with company-based optimization formulation and two alternate optimization models, we find that our ML-based approach can effectively solve large-scale problems within a stipulated time. We evaluated our model on real-world datasets supplied by a large e-commerce platform. The framework is currently adopted by a large e-commerce company across its 28 fulfillment centers, which is estimated to save the company about 7.1 million USD annually. In addition, it is estimated that paper consumption will be reduced by 2080 metric tons and greenhouse gas emissions by 1960 metric tons annually. The presented optimization framework serves as a decision support tool for designing packaging boxes at large e-commerce warehouses.

Application of GWO-attention-ConvLSTM model in customer churn prediction and satisfaction analysis in customer relationship management

Customer Relationship Management (CRM) is vital in modern business, aiding in the management and analysis of customer interactions. However, existing methods struggle to capture the dynamic and complex nature of customer relationships, as traditional approaches fail to leverage time series data effectively. To address this, we propose a novel GWO-attention-ConvLSTM model, which offers more effective prediction of customer churn and analysis of customer satisfaction. This model utilizes an attention mechanism to focus on key information and integrates a ConvLSTM layer to capture spatiotemporal features, effectively modeling complex temporal patterns in customer data. We validate our proposed model on multiple real-world datasets, including the BigML Telco Churn dataset, IBM Telco dataset, Cell2Cell dataset, and Orange Telecom dataset. Experimental results demonstrate significant performance improvements of our model compared to existing baseline models across these datasets. For instance, on the BigML Telco Churn dataset, our model achieves an accuracy of 95.17%, a recall of 93.66%, an F1 score of 92.89%, and an AUC of 95.00%. Similar results are validated on other datasets. In conclusion, our proposed GWO-attention-ConvLSTM model makes significant advancements in the CRM domain, providing powerful tools for predicting customer churn and analyzing customer satisfaction. By addressing the limitations of existing methods and leveraging the capabilities of deep learning, attention mechanisms, and optimization algorithms, our model paves the way for improving customer relationship management practices and driving business success.

Anomaly detection using unsupervised machine learning algorithms: A simulation study

This study presents a comprehensive evaluation of five prominent unsupervised machine learning anomaly detection algorithms: One-Class Support Vector Machine (One-Class SVM), One-Class SVM with Stochastic Gradient Descent (SGD), Isolation Forest (iForest), Local Outlier Factor (LOF), and Robust Covariance (Elliptic Envelope). Through systematic analysis on a synthetically simulated dataset, the study assessed each algorithm’s predictive performance using accuracy, precision, recall, and F1 score specifically for outlier detection. The evaluation reveals that One-Class SVM, Isolation Forest, and Robust Covariance are more effective in identifying outliers in the synthetic simulated dataset, with Isolation Forest slightly outperforming the other algorithms in terms of balancing precision and recall. One-Class SVM with SGD shows promise in precision but needs adjustment to improve recall. Local Outlier Factor may require parameter tuning or may not be as suitable for this particular dataset’s characteristics. The findings reveal significant variations in performance, highlighting the strengths and limitations of each method in identifying anomalies. This research contributes to the field of machine learning by providing insights into the applicability and performance of these algorithms, offering guidance for practitioners in selecting appropriate anomaly detection techniques for their specific needs. Future work should explore parameter optimization, the impact of dataset characteristics on model performance, and the application of these models to real-world datasets to validate their efficacy in practical anomaly detection scenarios.

Generative Variational-Contrastive Learning for Self-Supervised Point Cloud Representation.

Self-supervised representation learning for 3D point clouds has attracted increasing attention. However, existing methods in the field of 3D computer vision generally use fixed embeddings to represent the latent features, and impose hard constraints on the embeddings to make the latent feature values of the positive samples converge to consistency, which limits the ability of feature extractors to generalize over different data domains. To address this issue, we propose a Generative Variational-Contrastive Learning (GVC) model, where Gaussian distribution is used to construct a continuous, smoothed representation of the latent features. A distribution constraint and cross-supervision are constructed to improve the transfer ability of the feature extractor over synthetic and real-world data. Specifically, we design a variational contrastive module to constrain the feature distribution instead of feature values corresponding to each sample in the latent space. Moreover, a generative cross-supervision module is introduced to preserve the invariance features and promote the consistency of feature distribution among positive samples. Experimental results demonstrate that GVC achieves SOTA on different downstream tasks. In particular, with only pre-training on the synthetic dataset, GVC achieves a lead of 8.4% and 14.2% when transferring to the real-world dataset in the linear classification and few-shot classification.

Motif-Based Contrastive Learning for Community Detection.

Community detection has become a prominent task in complex network analysis. However, most of the existing methods for community detection only focus on the lower order structure at the level of individual nodes and edges and ignore the higher order connectivity patterns that characterize the fundamental building blocks within the network. In recent years, researchers have shown interest in motifs and their role in network analysis. However, most of the existing higher order approaches are based on shallow methods, failing to capture the intricate nonlinear relationships between nodes. In order to better fuse higher order and lower order structural information, a novel deep learning framework called motif-based contrastive learning for community detection (MotifCC) is proposed. First, a higher order network is constructed based on motifs. Subnetworks are then obtained by removing isolated nodes, addressing the fragmentation issue in the higher order network. Next, the concept of contrastive learning is applied to effectively fuse various kinds of information from nodes, edges, and higher order and lower order structures. This aims to maximize the similarity of corresponding node information, while distinguishing different nodes and different communities. Finally, based on the community structure of subnetworks, the community labels of all nodes are obtained by using the idea of label propagation. Extensive experiments on real-world datasets validate the effectiveness of MotifCC.

EGST: Enhanced Geometric Structure Transformer for Point Cloud Registration.

We explore the effect of geometric structure descriptors on extracting reliable correspondences and obtaining accurate registration for point cloud registration. The point cloud registration task involves the estimation of rigid transformation motion in unorganized point cloud, hence it is crucial to capture the contextual features of the geometric structure in point cloud. Recent coordinates-only methods ignore numerous geometric information in the point cloud which weaken ability to express the global context. We propose Enhanced Geometric Structure Transformer to learn enhanced contextual features of the geometric structure in point cloud and model the structure consistency between point clouds for extracting reliable correspondences, which encodes three explicit enhanced geometric structures and provides significant cues for point cloud registration. More importantly, we report empirical results that Enhanced Geometric Structure Transformer can learn meaningful geometric structure features using none of the following: (i) explicit positional embeddings, (ii) additional feature exchange module such as cross-attention, which can simplify network structure compared with plain Transformer. Extensive experiments on the synthetic dataset and real-world datasets illustrate that our method can achieve competitive results.