Unified Dataset Research Articles

FusionPortableV2: A unified multi-sensor dataset for generalized SLAM across diverse platforms and scalable environments

Simultaneous Localization and Mapping (SLAM) has been widely applied in various robotic missions, from rescue operations to autonomous driving. However, the generalization of SLAM algorithms remains a significant challenge, as current datasets often lack scalability in terms of platforms and environments. To address this limitation, we present FusionPortableV2, a multi-sensor SLAM dataset featuring sensor diversity, varied motion patterns, and a wide range of environmental scenarios. Our dataset comprises 27 sequences, spanning over 2.5 hours and collected from four distinct platforms: a handheld suite, a legged robot, an unmanned ground vehicle (UGV), and a vehicle. These sequences cover diverse settings, including buildings, campuses, and urban areas, with a total length of 38.7 km. Additionally, the dataset includes ground truth (GT) trajectories and RGB point cloud maps covering approximately 0.3 km2. To validate the utility of our dataset in advancing SLAM research, we assess several state-of-the-art (SOTA) SLAM algorithms. Furthermore, we demonstrate the dataset’s broad application beyond traditional SLAM tasks by investigating its potential for monocular depth estimation. The complete dataset, including sensor data, GT, and calibration details, is accessible at https://fusionportable.github.io/dataset/fusionportable_v2 .

Abstract 4145826: Atrial Fibrillation Screening During Sinus Rhythm Periods by Interrelated Systems Dynamics Analysis

Background: Previous studies have shown that AF screening in at-risk populations can reduce stroke incidence. However, non-targeted screening approaches often result in high false positive rates, placing an unnecessary burden on the healthcare system. In contrast, artificial intelligence-guided screening has been demonstrated to increase diagnostic yield in large prospective clinical trials. This approach, however, requires recording an ECG and a large-scale dataset for model training. Heart rate variability (HRV) analysis has proven effective in deciphering key heart dynamics. By analyzing HRV as interrelated dynamic systems, it may be possible to facilitate targeted AF screening using wearable devices that measure heart rate. The Koopman operator, used for data-driven modeling of interrelated dynamic systems, has been shown to accurately predict complex phenomena in chaotic systems such as climate forecasting and drug adverse reaction prediction. This is achieved by utilizing common characteristics of the systems for most model parameters, with only a small fraction of the parameters being specific to a certain system. Methods: Long (>10 hour) records from 361 individuals (AFDB, LTAFDB) and healthy individuals' datasets from PhysioNet and THEW were analyzed for inter-beat intervals. The unified dataset was then split into 94 training, 17 validation, and 250 test set patients. Recordings from the training set were used to train both the common and specific parts of the interrelated dynamic systems model for each patient, along with a shared small neural network classifying patients into low and high risk for AF based on the unique (not shared between patients) singular values of the dynamic system model. Patient-specific dynamic system models were then fitted for the validation and test sets to calculate the patient dynamic singular values, which were used to classify patients into low and high risk for AF groups. Results: Atrial fibrillation occurred in 48 of 202 (23%) patients classified as low risk and 35 of 48 (72.9%) patients classified as high risk (odds ratio 8.63, 95% CI 4.23-17.64), yielding 72.9% sensitivity with 76.2% specificity. Conclusion: In this retrospective analysis, classification of the dynamic system model singular values identified patients at high risk for atrial fibrillation from sinus rhythm period.

The effect of data complexity on classifier performance

The research area of Software Defect Prediction (SDP) is both extensive and popular, and is often treated as a classification problem. Improvements in classification, pre-processing and tuning techniques, (together with many factors which can influence model performance) have encouraged this trend. However, no matter the effort in these areas, it seems that there is a ceiling in the performance of the classification models used in SDP. In this paper, the issue of classifier performance is analysed from the perspective of data complexity. Specifically, data complexity metrics are calculated using the Unified Bug Dataset, a collection of well-known SDP datasets, and then checked for correlation with the defect prediction performance of machine learning classifiers (in particular, the classifiers C5.0, Naive Bayes, Artificial Neural Networks, Random Forests, and Support Vector Machines). In this work, different domains of competence and incompetence are identified for the classifiers. Similarities and differences between the classifiers and the performance metrics are found and the Unified Bug Dataset is analysed from the perspective of data complexity. We found that certain classifiers work best in certain situations and that all data complexity metrics can be problematic, although certain classifiers did excel in some situations.

Siamese Graph Convolutional Split-Attention Network with NLP based Social Sentimental Data for enhanced stock price predictions

Predicting stock market behavior using sentiment analysis has become increasingly popular, as customer responses on platforms like Twitter can influence market trends. However, most existing sentiment-based models struggle with two major issues: inaccuracy and high complexity. These problems lead to frequent prediction errors and make the models difficult to implement in real-time trading systems. To address these challenges, this paper proposes a new method called Siagra-ConSA-HSOA (Siamese Graph Convolutional Split-Attention Network with NLP-based Social Sentiment Data). Two data sources feed the model: specifically, NIFTY-50 Stock Market and real-time Twitter sentiment. Through Natural Language Processing (NLP), the raw data is pre-processed and key features are extracted before they are fused into a unified dataset using a cross-domain transformer, namely CDSFT, and then Circle-Inspired Optimization Algorithm (CIOA) selects the most important features from this dataset. This decreases the complexity of the model without losing essential information. Finally, a Graph Convolutional Split-Attention Network (SGCSAN) for promisingly predicting whether the stock prices are going to hit the ground and fly high again or is going to nosedive with Humboldt Squid Optimization Algorithm (HSOA) is introduced to further improve accuracy with lesser error generation. The proposed model Siagra-ConSA-HSOA achieved 99.9% accuracy and 99.8% recall in the testing stage, meaning that such a model performs better than the current approaches both in prediction accuracy and efficiency. Thus, this is a glimmer that the model shall be able to overcome some of the main problems with the current techniques used in predicting the behavior of the stock market.GitHub Repository: https://github.com/jramans2/Siamese-GCN-SplitAttention-Stock-Prediction.git

Massive Point Cloud Processing for Efficient Construction Quality Inspection and Control.

The construction of large-scale civil infrastructures requires massive spatiotemporal data to support the management and control of scheduling, quality control, and safety monitoring. Existing artificial-intelligence-based data processing algorithms rely heavily on experienced engineers to adjust the parameters of data processing, which is inefficient and time-consuming when dealing with huge datasets. Limited studies have compared the performance of different algorithms on a unified dataset. This study proposes a framework and evaluation system for comparing different data processing policies for processing huge spatiotemporal data in construction quality control. The proposed method compares the combination of multiple types of algorithms involved in the processing of massive point cloud data. The performance of data processing strategies is evaluated through this framework, and the optimal point cloud processing strategies are explored based on registration accuracy and data fidelity. Results show that a reasonable choice of combinations of point cloud sampling, filtering, and registration algorithms can significantly improve the efficiency of point cloud data processing and satisfy engineering demands for data accuracy and completeness. The proposed method can be applied to the civil engineering problem of processing a large amount of point cloud data and selecting the optimal processing method.

A unified longitudinal trajectory dataset for automated vehicle

Automated Vehicles (AVs) promise significant advances in transportation. Critical to these improvements is understanding AVs’ longitudinal behavior, relying heavily on real-world trajectory data. Existing open-source trajectory datasets of AV, however, often fall short in refinement, reliability, and completeness, hindering effective performance metrics analysis and model development. This study addresses these challenges by creating a Unified longitudinal trajectory dataset for AVs (Ultra-AV) to analyze their microscopic longitudinal driving behaviors. This dataset compiles data from 14 distinct sources, encompassing various AV types, test sites, and experiment scenarios. We established a three-step data processing: 1. extraction of longitudinal trajectory data, 2. general data cleaning, and 3. data-specific cleaning to obtain the longitudinal trajectory data and car-following trajectory data. The validity of the processed data is affirmed through performance evaluations across safety, mobility, stability, and sustainability, along with an analysis of the relationships between variables in car-following models. Our work not only furnishes researchers with standardized data and metrics for longitudinal AV behavior studies but also sets guidelines for data collection and model development.

Machine Learning Models for Predicting Bioavailability of Traditional and Emerging Aromatic Contaminants in Plant Roots.

To predict the behavior of aromatic contaminants (ACs) in complex soil-plant systems, this study developed machine learning (ML) models to estimate the root concentration factor (RCF) of both traditional (e.g., polycyclic aromatic hydrocarbons, polychlorinated biphenyls) and emerging ACs (e.g., phthalate acid esters, aryl organophosphate esters). Four ML algorithms were employed, trained on a unified RCF dataset comprising 878 data points, covering 6 features of soil-plant cultivation systems and 98 molecular descriptors of 55 chemicals, including 29 emerging ACs. The gradient-boosted regression tree (GBRT) model demonstrated strong predictive performance, with a coefficient of determination (R2) of 0.75, a mean absolute error (MAE) of 0.11, and a root mean square error (RMSE) of 0.22, as validated by five-fold cross-validation. Multiple explanatory analyses highlighted the significance of soil organic matter (SOM), plant protein and lipid content, exposure time, and molecular descriptors related to electronegativity distribution pattern (GATS8e) and double-ring structure (fr_bicyclic). An increase in SOM was found to decrease the overall RCF, while other variables showed strong correlations within specific ranges. This GBRT model provides an important tool for assessing the environmental behaviors of ACs in soil-plant systems, thereby supporting further investigations into their ecological and human exposure risks.

From CSV to Arrow: Creating a Unified Data Set for Efficient Cross-Platform Analysis

Handling open data, like the vast repository of New York City (NYC) 311 service requests, often starts with the ubiquitous CSV (comma-separated value) file format. However, CSV files are notoriously inefficient for curation, bogged down by redundancy and potential misinterpretations. Enter Apache Arrow, a game-changing approach that not only slashes storage requirements but also primes data for seamless analysis across popular platforms like R, Python, and Julia. Using the NYC 311 service request data, we demonstrate the conversion of a CSV file to the Arrow IPC (Inter-Process Communication) format. An Arrow file stores the table schema with the data in a binary format that can be memorymapped for reading, enabling instantaneous access to potentially large datasets. The Arrow IPC data serves as a universal starting point for analysis across various environments. In our example, this conversion is done in Julia, which has powerful packages for reading and writing CSV or Arrow files and calling functions in other popular environments such as R and Python.

The Association of Cerebrospinal Fluid Biomarkers of Alzheimer’s Disease with Cognitive Intra-Individual Variability

Abstract Objective Cognitive intra-individual variability (IIV) is a sensitive predictor of cognitive decline and disease severity in Alzheimer’s disease (AD). Despite its sensitivity, whether IIV is merely a psychometric artifact or disease pathology biomarker remains unknown. The present study evaluated the association between IIV and cerebrospinal fluid (CSF) biomarkers of AD. Method Participants included 422 individuals from the National Alzheimer’s Coordinating Center Unified Dataset 3.0 who completed baseline neuropsychological assessment and had CSF data. All participants were adult monolingual English speakers. Participants with vision or hearing impairments were excluded. Cases were stratified into cognitively normal (n = 267) and AD groups (n = 155; mild cognitive impairment = 50, dementia = 105). Cross-sectional IIV was calculated by obtaining the individual standard deviation (iSD) of ten cognitive tests. Finally, the coefficient of variation (CoV) was computed by dividing iSD by the participant’s overall test battery mean. Non-parametric Spearman-rank correlations were used to assess the relation between CoV and CSF biomarkers, including beta-amyloid 1–42, total tau (t-tau), and phosphorylated tau (p-tau181). Results In the AD group, CoV was positively correlated with p-tau181 (r = 0.285, p = 0.003) and t-tau (r = 0.241, p = 0.009), and negatively correlated with beta-amyloid 1–42 (r = −0.314, p < 0.001). Correlations between CoV and CSF biomarkers were not significant in the cognitively normal group for beta-amyloid 1–42 (r = 0.076, p = 0.215), p-tau181 (r = −0.014, p = 0.840), and t-tau (r = −0.046, p = 0.485). Conclusions In people with mild cognitive impairment and dementia of presumed AD etiology, cognitive IIV is associated with CSF biomarkers. These findings provide preliminary evidence that IIV is not merely a statistical artifact, but is associated with AD neuropathology.

Dialogue agents 101: a beginner’s guide to critical ingredients for designing effective conversational systems

Abstract Sharing ideas through communication with peers is the primary mode of human interaction. Consequently, extensive research has been conducted in the area of conversational AI, leading to an increase in the availability and diversity of conversational tasks, datasets, and methods. However, with numerous tasks being explored simultaneously, the current landscape of conversational AI has become fragmented. Consequently, initiating a well-thought-out model for a dialogue agent can pose significant challenges for a practitioner. Toward highlighting the critical ingredients needed for a practitioner to design a dialogue agent from scratch, the current study provides a comprehensive overview of the primary characteristics of a dialogue agent, the supporting tasks, their corresponding open-domain datasets, and the methods used to benchmark these datasets. We observe that different methods have been used to tackle distinct dialogue tasks. However, building separate models for each task is costly and does not leverage the correlation among the several tasks of a dialogue agent. As a result, recent trends suggest a shift toward building unified foundation models. To this end, we propose Unit, a Unified dialogue dataset constructed from conversations of varying datasets for different dialogue tasks capturing the nuances for each of them. We then train a Unified dialogue foundation model, GPT-2 $^{\textrm{U}}$ and present a concise comparative performance of GPT-2 $^{\textrm{U}}$ against existing large language models. We also examine the evaluation strategies used to measure the performance of dialogue agents and highlight the scope for future research in the area of conversational AI with a thorough discussion of popular models such as ChatGPT.

A Combined CNN Architecture for Speech Emotion Recognition.

Emotion recognition through speech is a technique employed in various scenarios of Human-Computer Interaction (HCI). Existing approaches have achieved significant results; however, limitations persist, with the quantity and diversity of data being more notable when deep learning techniques are used. The lack of a standard in feature selection leads to continuous development and experimentation. Choosing and designing the appropriate network architecture constitutes another challenge. This study addresses the challenge of recognizing emotions in the human voice using deep learning techniques, proposing a comprehensive approach, and developing preprocessing and feature selection stages while constructing a dataset called EmoDSc as a result of combining several available databases. The synergy between spectral features and spectrogram images is investigated. Independently, the weighted accuracy obtained using only spectral features was 89%, while using only spectrogram images, the weighted accuracy reached 90%. These results, although surpassing previous research, highlight the strengths and limitations when operating in isolation. Based on this exploration, a neural network architecture composed of a CNN1D, a CNN2D, and an MLP that fuses spectral features and spectogram images is proposed. The model, supported by the unified dataset EmoDSc, demonstrates a remarkable accuracy of 96%.

A holistic approach to software fault prediction with dynamic classification

Software Fault Prediction is a critical domain in machine learning aimed at pre-emptively identifying and mitigating software faults. This study addresses challenges related to imbalanced datasets and feature selection, significantly enhancing the effectiveness of fault prediction models. We mitigate class imbalance in the Unified Dataset using the Random-Over Sampling technique, resulting in superior accuracy for minority-class predictions. Additionally, we employ the innovative Ant-Colony Optimization algorithm (ACO) for feature selection, extracting pertinent features to amplify model performance. Recognizing the limitations of individual machine learning models, we introduce the Dynamic Classifier, a ground-breaking ensemble that combines predictions from multiple algorithms, elevating fault prediction precision. Model parameters are fine-tuned using the Grid-Search Method, achieving an accuracy of 94.129% and superior overall performance compared to random forest, decision tree and other standard machine learning algorithms. The core contribution of this study lies in the comparative analysis, pitting our Dynamic Classifier against Standard Algorithms using diverse performance metrics. The results unequivocally establish the Dynamic Classifier as a frontrunner, highlighting its prowess in fault prediction. In conclusion, this research introduces a comprehensive and innovative approach to software fault prediction. It pioneers the resolution of class imbalance, employs cutting-edge feature selection, and introduces dynamic ensemble classifiers. The proposed methodology, showcasing a significant advancement in performance over existing methods, illuminates the path toward developing more accurate and efficient fault prediction models.

Evaluation of 17 thermal conductivity models for frozen soil

Accurate prediction of thermal conductivity (λ) in frozen soils is essential for understanding their thermal behavior and improving thermal response analyses. Currently, there are numerous models for calculating λ, but unified laboratory data to compare and evaluate these models are lacking. This study focuses on evaluating the performance of 17 models for applied to frozen soils using a unified laboratorial dataset. The heat-pulse-probe (HPP) method was employed to measure the thermal conductivity of silty clay, sand, and sand loam. Nuclear magnetic resonance experiments were conducted to measure the unfrozen water content (θw) and ice content(θi). This provides unified laboratorial dataset of λ-θi-θw. The predictive performance of 17 models demonstrated that the model of Zhang et al. (2018), Sass et al. (1971) and Tian et al. (2016) are the three optimal models: 1). The model of Zhang et al. (2018) is the provided the best predictive performance, with mean absolute error (MAE) values of 0.17 Wm-1K-1, 0.07 Wm-1K-1, and 0.06 Wm-1K-1 for silty clay, sand and sand loam, respectively. This model is advantageous due to its simplicity and the absence of undetermined parameters. 2). The model of Sass et al. (1971) is suitable for soils with low sensitivity to frozen volume deformation, such as sand and sand loam, achieving an MAE of 0.06 Wm-1K-1. 3). The model of Tian et al. (2016) exhibited strong performance for silty clay with an MAE of 0.11 Wm-1K-1, contingent on the precise determination of soil-specific parameters. This study provides a unified laboratory data for the establishment of thermal conductivity models. The evaluation of existing models can serve as a reference for future related research.

Leveraging machine learning to generate a unified and complete building height dataset for Germany

Building geometry data is crucial for detailed, spatially-explicit analyses of the building stock in energy systems analysis and beyond. Despite the existence of diverse datasets and methods, a standardized and validated approach for creating a nation-wide unified and complete dataset of German building heights is not yet available. This study develops and validates such a methodology, combining different data sources for building footprints and heights and filling gaps in height data using an XGBoost machine learning algorithm. The XGBoost model achieves a mean absolute error of 1.78 m at the national level and between 1.52 m and 3.47 m at the federal state level. The goal is proving the applicability of the methodology at a large scale and creating a useful dataset. The resulting dataset is thoroughly evaluated on a building-by-building level and spatially resolved statistics on the quality of the dataset are reported. This detailed validation found that the building number and footprint area of German building stock is 90.31 % and 94.84 % correct, respectively, and the building height accuracy is 0.59 m at the national level. However, errors are not homogeneous across Germany and further research is needed into the impact of including additional datasets, especially for regions and building types with lower accuracies. This study proves that the chosen methodology is useful for generating a building height dataset and the workflow, with some modifications for regional data availability, can be transferred to other countries. The generated building dataset for Germany constitutes a valuable data basis for the research community in fields such as energy research, urban planning and building decarbonization policy development.

A unified Foot and Mouth Disease dataset for Uganda: evaluating machine learning predictive performance degradation under varying distributions.

In Uganda, the absence of a unified dataset for constructing machine learning models to predict Foot and Mouth Disease outbreaks hinders preparedness. Although machine learning models exhibit excellent predictive performance for Foot and Mouth Disease outbreaks under stationary conditions, they are susceptible to performance degradation in non-stationary environments. Rainfall and temperature are key factors influencing these outbreaks, and their variability due to climate change can significantly impact predictive performance. This study created a unified Foot and Mouth Disease dataset by integrating disparate sources and pre-processing data using mean imputation, duplicate removal, visualization, and merging techniques. To evaluate performance degradation, seven machine learning models were trained and assessed using metrics including accuracy, area under the receiver operating characteristic curve, recall, precision and F1-score. The dataset showed a significant class imbalance with more non-outbreaks than outbreaks, requiring data augmentation methods. Variability in rainfall and temperature impacted predictive performance, causing notable degradation. Random Forest with borderline SMOTE was the top-performing model in a stationary environment, achieving 92% accuracy, 0.97 area under the receiver operating characteristic curve, 0.94 recall, 0.90 precision, and 0.92 F1-score. However, under varying distributions, all models exhibited significant performance degradation, with random forest accuracy dropping to 46%, area under the receiver operating characteristic curve to 0.58, recall to 0.03, precision to 0.24, and F1-score to 0.06. This study underscores the creation of a unified Foot and Mouth Disease dataset for Uganda and reveals significant performance degradation in seven machine learning models under varying distributions. These findings highlight the need for new methods to address the impact of distribution variability on predictive performance.

Machine learning for cyanobacteria inversion via remote sensing and AlgaeTorch in the Třeboň fishponds, Czech Republic

Cyanobacteria blooms in fishponds, driven by climate change and anthropogenic activities, have become a critical concern for aquatic ecosystems worldwide. The diversity in fishpond sizes and fish densities further complicates their monitoring. This study addresses the challenge of accurately predicting cyanobacteria concentrations in turbid waters via remote sensing, hindered by optical complexities and diminished light signals. A comprehensive dataset of 740 sampling points was compiled, encompassing water quality metrics (cyanobacteria levels, total chlorophyll, turbidity, total cell count) and spectral data obtained through AlgaeTorch, alongside Sentinel-2 reflectance data from three Třeboň fishponds (UNESCO Man and Biosphere Reserve) in the Czech Republic over 2022–2023. Partial Least Squares Regression (PLSR) and three machine learning algorithms, Random Forest (RF), Support Vector Machine (SVM), and Extreme Gradient Boosting (XGBoost), were developed based on seasonal and annual data volumes. The SVM algorithm demonstrated commendable performance on the one-year data validation dataset from the Svět fishpond for the prediction of cyanobacteria, reflected by the key performance indicators: R2 = 0.88, RMSE = 15.07 μg Chl-a/L, and RPD = 2.82. Meanwhile, SVM displayed steady results in the unified one-year validation dataset from Naděje, Svět, and Vizír fishponds, with metrics showing R2 = 0.56, RMSE = 39.03 μg Chl-a/L, RPD = 1.50. Thus, Sentinel data proved viable for seasonal cyanobacteria monitoring across different fishponds. Overall, this study presents a novel approach for enhancing the precision of cyanobacteria predictions and long-term ecological monitoring in fishponds, contributing significantly to the water quality management strategies in the Třeboň region.

A Bayesian Merging of Earthquake Magnitudes Determined by Multiple Seismic Networks

Abstract We introduce a Bayesian algorithm designed to integrate earthquake magnitudes of the same type reported by various seismic networks, aiming to create unified and standardized catalogs suitable for widespread use. The fundamental concept underpinning this algorithm is the utilization of the inherent consistency within each individual network’s magnitude determination process. Assuming that the magnitudes for an earthquake measured by all networks conform to a Gaussian distribution, with a linear function of the unknown true magnitude serving as its mean, we derive the posterior probability distribution of the true magnitude under four different assumptions for the prior distribution: the uninformative uniform distribution, the unbounded Gutenberg–Richter (GR) magnitude–frequency law, the GR magnitude–frequency relationship restricted by the detection rate, and the truncated GR law as priors. We assess the robustness of the method by a test on several synthetic catalogs and then use it to merge the catalogs compiled by five seismic networks in Italy. The results demonstrate that our proposed magnitude-merging algorithm effectively combines the catalogs, resulting in robust and unified data sets that are suitable for seismic hazard assessment and seismicity analysis.

LTFM-net framework: Advanced intelligent diagnostics and interpretability of insulated bearing faults in offshore wind turbines under complex operational conditions

The rapid advancement of intelligent theories and models, exemplified by deep learning, has achieved remarkable success across numerous fields. However, given that the complexity and variability of offshore wind turbine systems, particularly in the context of high-power variable-frequency control of insulated bearings in offshore wind turbines, the problem of fault identification has become a recognized technical challenge. Additionally, fully exploiting the temporal characteristics of insulated bearing fault data poses a key problem that demands urgent resolution. To address these gaps, this paper pioneers a novel Lightweight Temporal Feature-focused framework, named LTFM-Net, aimed at solving the difficult problem of identifying insulated bearing faults in offshore wind turbines in practical engineering applications. Specifically, this framework enables the intelligent identification of insulated bearing faults under harsh operating conditions such as alternating voltage and variable loads, marking a first in this field. Furthermore, an innovative strategy named Weighted Diminish Recurrent Unit (WDRU) was developed, along with the derivation of its backpropagation formula, which is innovatively applied to the feature extraction module of the LTFM-Net framework for the first time. Thus, an efficient method for acquiring fault data from insulated bearings in offshore wind turbines was proposed. Based on a unified dataset, the diagnostic performance of the LTFM-Net framework was evaluated and compared with seven advanced methods. The results demonstrate that the LTFM-Net achieves precise identification of insulated bearing faults, confirming its excellent generalization, robustness, and superiority. The introduction of t-SNE for visualizing the fault characteristics of insulated bearings uncovered by the LTFM-Net framework further enhances its reliability, accuracy, and credibility.

Unifying community-wide whole-brain imaging datasets enables robust automated neuron identification and reveals determinants of neuron positioning in C. elegans.

We develop a data harmonization approach for C. elegans volumetric microscopy data, still or video, consisting of a standardized format, data pre-processing techniques, and a set of human-in-the-loop machine learning based analysis software tools. We unify a diverse collection of 118 whole-brain neural activity imaging datasets from 5 labs, storing these and accompanying tools in an online repository called WormID (wormid.org). We use this repository to train three existing automated cell identification algorithms to, for the first time, enable accuracy in neural identification that generalizes across labs, approaching human performance in some cases. We mine this repository to identify factors that influence the developmental positioning of neurons. To facilitate communal use of this repository, we created open-source software, code, web-based tools, and tutorials to explore and curate datasets for contribution to the scientific community. This repository provides a growing resource for experimentalists, theorists, and toolmakers to (a) study neuroanatomical organization and neural activity across diverse experimental paradigms, (b) develop and benchmark algorithms for automated neuron detection, segmentation, cell identification, tracking, and activity extraction, and (c) inform models of neurobiological development and function.

A ten time smaller version of CPC global daily precipitation dataset for parallel distributed processing in Matlab and R

A ten times smaller version of CPC Global Unified Gauge-Based Daily Precipitation Dataset (1979–2021) is released and described in this paper. Its usability is tested and proved, firstly, by illustrating that the transition to the derived smaller dataset is a case of Pearson correlation transitivity starting with the scale of global yearly data and, secondly, by using the original correlation performance criterion that the original dataset satisfies relative to the set of global actual measurements on the record. Subsequently, the daily, weekly, and monthly data cases are considered and discussed. The dataset is (re)structured for parallel processing in Matlab and R from the level of global daily data. Considering the above arguments and its reduced size, the derived dataset is appropriate to be used with Matlab and R as a replacement for the original dataset, especially for the case when much faster exploratory master-slave parallel and distributed Matlab and/or R tasks will run over locally distributed data on the slaves.