Training Data Research Articles

Fair and optimal prediction via post‐processing

AbstractWith the development of machine learning algorithms and the increasing computational resources available, artificial intelligence has achieved great success in many application domains. However, the success of machine learning has also raised concerns about the fairness of the learned models. For instance, the learned models can perpetuate and even exacerbate the potential bias and discrimination in the training data. This issue has become a major obstacle to the deployment of machine learning systems in high‐stakes domains, for example, criminal judgment, medical testing, online advertising, hiring process, and so forth. To mitigate the potential bias exhibited by machine learning models, fairness criteria can be integrated into the training process to ensure fair treatment across all demographics, but it often comes at the expense of model performance. Understanding such tradeoffs, therefore, is crucial to the design of optimal and fair algorithms. My research focuses on characterizing the inherent tradeoff between fairness and accuracy in machine learning, and developing algorithms that can achieve both fairness and optimality. In this article, I will discuss our recent work on designing post‐processing algorithms for fair classification, which can be applied to a wide range of fairness criteria, including statistical parity, equal opportunity, and equalized odds, under both attribute‐aware and attribute‐blind settings, and is particularly suited to large‐scale foundation models where retraining is expensive or even infeasible. I will also discuss the connections between our work and other related research on trustworthy machine learning, including the connections between algorithmic fairness and differential privacy as well as adversarial robustness.

Using ADTS and CNN methods to effectively monitor CD, crosstalk, and OSNR in an optical network

The article presents the results of a method based on asynchronous delay-tap sampling (ADTS) and convolutional neural network (CNN) for determining simultaneously occurring disturbances described using the chromatic dispersion (CD), crosstalk and optical signal-to-noise ratio (OSNR) parameters. The ADTS method was used to generate training and test data for the convolutional network, which in turn was used to learn to recognize interference from said data. The tests were carried out for a transmission speed of 10 Gbit/s and for on-off keying (OOK) and differential phase shift keying (DPSK) modulation. Very good results were obtained in recognizing simultaneously occurring phenomena. Accuracy of over 99% was achieved for CD and crosstalk for DPSK modulation and over 98% for OOK modulation. In the case of amplified spontaneous emission (ASE) noise, slightly weaker results were obtained, above 95–96% for both modulations. Based on the conducted research, it was determined that the use of ADTS and CNN methods enables monitoring of simultaneously occurring CD, crosstalk, and ASE noise in the physical layer of the optical network, while maintaining the requirements for modern monitoring systems.

AI fairness in practice: Paradigm, challenges, and prospects

AbstractUnderstanding and correcting algorithmic bias in artificial intelligence (AI) has become increasingly important, leading to a surge in research on AI fairness within both the AI community and broader society. Traditionally, this research operates within the constrained supervised learning paradigm, assuming the presence of class labels, independent and identically distributed (IID) data, and batch‐based learning necessitating the simultaneous availability of all training data. However, in practice, class labels may be absent due to censoring, data is often represented using non‐IID graph structures that capture connections among individual units, and data can arrive and evolve over time. These prevalent real‐world data representations limit the applicability of existing fairness literature, which typically addresses fairness in static and tabular supervised learning settings. This paper reviews recent advances in AI fairness aimed at bridging these gaps for practical deployment in real‐world scenarios. Additionally, opportunities are envisioned by highlighting the limitations and significant potential for real applications.

Large Language Models Outperform Expert Coders and Supervised Classifiers at Annotating Political Social Media Messages

Instruction-tuned Large Language Models (LLMs) have recently emerged as a powerful new tool for text analysis. As these models are capable of zero-shot annotation based on instructions written in natural language, they obviate the need of large sets of training data—and thus bring potential paradigm-shifting implications for using text as data. While the models show substantial promise, their relative performance compared to human coders and supervised models remains poorly understood and subject to significant academic debate. This paper assesses the strengths and weaknesses of popular fine-tuned AI models compared to both conventional supervised classifiers and manual annotation by experts and crowd workers. The task used is to identify the political affiliation of politicians based on a single X/Twitter message, focusing on data from 11 different countries. The paper finds that GPT-4 achieves higher accuracy than both supervised models and human coders across all languages and country contexts. In the US context, it achieves an accuracy of 0.934 and an inter-coder reliability of 0.982. Examining the cases where the models fail, the paper finds that the LLM—unlike the supervised models—correctly annotates messages that require interpretation of implicit or unspoken references, or reasoning on the basis of contextual knowledge—capacities that have traditionally been understood to be distinctly human. The paper thus contributes to our understanding of the revolutionary implications of LLMs for text analysis within the social sciences.

Physically Guided Estimation of Vehicle Loading-Induced Low-Frequency Bridge Responses with BP-ANN

The intersectional relationship in bridge health monitoring refers to the mapping function that correlates bridge responses across different locations. This relationship is pivotal for estimating structural responses, which are then instrumental in assessing a bridge’s service status and identifying potential damage. The current research landscape is heavily focused on high-frequency responses, especially those associated with single-mode vibration. When it comes to low-frequency responses triggered by multi-mode vehicle loading, a prevalent strategy is to regard these low-frequency responses as “quasi-static” and subsequently apply time-series prediction techniques to simulate the intersectional relationship. However, these methods are contingent upon data regarding external loading, such as traffic conditions and air temperatures. This necessitates the collection of long-term monitoring data to account for fluctuations in traffic and temperature, a task that can be quite daunting in real-world engineering contexts. To address this challenge, our study shifts the analytical perspective from a static analysis to a dynamic analysis. By delving into the physical features of bridge responses of the vehicle–bridge interaction (VBI) system, we identify that the intersectional relationship should be inherently time-independent. The perceived time lag in quasi-static responses is, in essence, a result of low-frequency vibrations that are aligned with driving force modes. We specifically derive the intersectional relationship for low-frequency bridge responses within the VBI system and determine it to be a time-invariant transfer matrix associated with multiple mode shapes. Drawing on these physical insights, we adopt a time-independent machine learning method, the backpropagation–artificial neural network (BP-ANN), to simulate the intersectional relationship. To train the network, monitoring data from various cross-sections were input, with the responses at a particular section designated as the output. The trained network is now capable of estimating responses even in scenarios where time-related traffic conditions and temperatures deviate from those present in the training data set. To substantiate the time-independent nature of the derived intersectional relationship, finite element models were developed. The proposed method was further validated through the in-field monitoring of a continuous highway bridge. We anticipate that this method will be highly effective in estimating low-frequency responses under a variety of unknown traffic and air temperature conditions, offering significant convenience for practical engineering applications.

Optimalisasi Performa Penjualan di PT Mustika Jaya Lestari Menggunakan Algoritma K-NN dan ARIMA (2021-2022)

Sales are a crucial aspect for companies, including PT Mustika Jaya Lestari, which uses sales predictions to determine which products will sell in the future. However, sometimes predictions can go wrong, such as when a product that is predicted to increase actually decreases. This research aims to optimize sales by utilizing the K-Nearest Neighbor (K-NN) method to predict sales results. The K-NN method works by classifying new data based on its proximity to old data. This research produces graph visualization to predict sales with 90% accuracy and 10% error, using 762 test data and 2285 training data.

Comparing durability and compressive strength predictions of hyperoptimized random forests and artificial neural networks on a small dataset of concrete containing nano SiO2 and RHA

Through the increasing use of supplementary cementitious materials, the properties of concrete have taken on increased significance in a design code. Using reliable prediction models based on a small data set for the mechanical properties and durability of concrete can reduce the number of trial batches and experiments needed to produce useful design data in the laboratory, reducing time as well as resources. In this study, we investigate how the properties of water penetration, chlorine resistance, and compressive strength can be predicted by polynomial regression (PR), random forest (RF) regression, and artificial neural networks (ANNs) based on the input values of density, workability, and the constituent amount of rice husk ash, cement, and nano SiO 2 . We vary the training data used and test the coefficient of determination ( R 2 score) on the remaining data as a test set to measure predictive capability. We show that RFs and ANNs outperform PR in all settings and have unambiguously extrapolating properties when hyperparameter optimization is designed for this purpose. Remarkably, we obtain R 2 scores on the test data of 0.858 − 0.990 for RFs and 0.825 − 0.985 for ANNs.

Implementasi Data Mining Klasifikasi Gejala Penyakit TB Menggunakan Algoritma Naive Bayes pada Studi Kasus Puskesmas Pegangsaan Dua B

Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis. The bacteria can enter the lungs, causing the patient to experience shortness of breath and chronic coughing. The infection process in TB progresses through three main stages: primary infection, latent infection, and active infection. During the primary infection stage, the bacteria enter the lungs through inhaled air. If the immune system successfully controls the bacteria, the infection becomes latent, where the bacteria remain inactive and cause no symptoms. However, if the immune system fails, the infection becomes active, and the bacteria attack healthy lung cells, leading to severe symptoms. Tuberculosis remains a major cause of global morbidity and mortality. According to the 2022 WHO report, Indonesia ranks second in the world with an estimated 969,000 TB cases, or 354 per 100,000 people, and a mortality rate of 144,000, or 52 per 100,000 people. This study aims to improve the accuracy of TB diagnosis through the application of the Naive Bayes algorithm, applied to patient data from the Pegangsaan Dua B Community Health Center. The dataset includes 1,527 training data and 1,029 testing data, covering attributes such as patient name, gender, age, identification number, registration date, address, diagnosis status, and diagnosis results. The results indicate that the Naive Bayes algorithm achieved an accuracy of 99.71%, showing its potential for early and accurate TB diagnosis prediction.

Optimization of Classification Algorithms Performance with k-Fold Cross Validation

Supervised learning is a predictive method used to make predictions or classifications. Supervised learning algorithms work by building a model using training data that includes both independent and dependent variables. Several methods for building classification include Logistic Regression, Naive Bayes, K-Nearest Neighbor (KNN), decision tree, etc. The lack of capacity of a classification algorithm to generalize certain data can be associated with the problem of overfitting or underfitting. K-fold cross-validation is a method that can help avoid overfitting or underfitting and produce a algorithm with good performance on new data. This study will test the Naive Bayes, K-Nearest Neighbor (KNN), Classification and Regression Tree (CART), and Logistic Regression methods with k-fold cross-validation on two different datasets. The values of k set for cross-validation are 2, 3, 5, 7, and 10. The analysis results concluded that each classification algorithm performed best at 10-fold cross-validation. In DATA 1, the Naive Bayes algorithm has the highest average accuracy of 0.67 (67%) and the error rate is 0.33 (33%), followed by the CART algorithm, KNN, and finally logistic regression. While DATA 2, the KNN algorithm has the highest average accuracy of 0.66 (66%) and an error rate of 0.34 (34%), followed by the CART algorithm, Naive Bayes, and finally logistic regressionbut can be a reference if you want to predict the growth direction of the accommodation and food service activities sector.

Improving Compton camera imaging of multi-energy radioactive sources by using machine learning algorithms for event selection

Event selection and background reduction for Compton camera imaging of multi-energy radioactive sources has been performed by employing neural networks. A Compton camera prototype with detectors made of LaBr3 crystals coupled to silicon photomultiplier arrays was used to acquire experimental data from a circular array of 22Na sources. The prototype and two arrays of 22Na sources were simulated with GATE v8.2 Monte Carlo code, to obtain data for neural network training. Neural network models were trained on simulated data for event classification. The optimum models were found by using Weights & Biases platform tools. The trained models were used to classify simulated and real data for selecting signal events and rejecting background prior to image reconstruction. The models performed well on simulated data. The image obtained with experimental data showed an improvement with respect to event selection with energy cuts. The method is promising for Compton camera imaging of multi-energy radioactive sources.

Predicting image quality of forensic footwear impressions

The ability of a footwear examiner to confidently discern features of importance in a forensic examination is directly related to impression quality. As a result, quality directly impacts the strength an examiner can ascribe to any opinion of source attribution. Despite the importance of image quality during both the analysis and comparison phases of an examination, there is limited research on the estimation, variation, and prediction of footwear impression quality. In response, this study aims to develop a methodology for evaluating footwear impression quality by regressing image features against subjective judgments of quality. Using a dataset of more than 450 impressions evaluated by more than 40 participants, estimates of intra- and inter-rater consistency were computed. After identifying reliable raters, matrix completion was performed, thereby permitting data imputation. This was based on an approximate 90:10 split between training and testing data, ultimately resulting in 6,000 quality predictions, which exhibited a mean agreement with ground truth > 95%. Each image in the dataset was then reduced to a 10-dimensional feature vector describing image attributes such as complexity, contrast, sharpness, and noise. These features were used to train multiple ordinal and multinomial regression models aimed at predicting image quality. Model comparison resulted in an overall optimal accuracy of approximately 75% when comparing predicted/modeled quality against the ground truth of subjective rater opinions. The resulting semi-automated, reference-free and numerical prediction tool exhibits reasonable success for impression quality prediction when presented with the types of challenging and domain-specific images often encountered in forensic footwear comparisons. The outcomes and challenges associated with this investigation provide a foundation upon which future studies of quality can be modified and built, with the downstream goal of an increased understanding of how quality impacts weight of evidence.

Robustness enhancement in neural networks with alpha-stable training noise

With the increasing use of deep learning on data collected by non-perfect sensors and in non-perfect environments, the robustness of deep learning systems has become an important issue. A common approach for obtaining robustness to noise has been to train deep learning systems with data augmented with Gaussian noise. In this work, the common choice of Gaussian noise is challenged and the possibility of stronger robustness for non-Gaussian impulsive noise is explored, specifically alpha-stable noise. Justified by the Generalized Central Limit Theorem and evidenced by observations in various application areas, alpha-stable noise is widely present in nature. By comparing the testing accuracy of models trained with Gaussian noise and alpha-stable noise on data corrupted by different noise, it is found that training with alpha-stable noise is more effective than Gaussian noise, especially when the dataset is corrupted by impulsive noise, thus improving the robustness of the model. Moreover, in the testing on the common corruption benchmark dataset, training with alpha-stable noise also achieves promising results, improving the robustness of the model to other corruption types and demonstrating comparable performance with other state-of-the-art data augmentation methods. Consequently, a novel data augmentation method is proposed that replaces Gaussian noise, which is typically added to the training data, with alpha-stable noise.

Deep learning–based statistical robustness evaluation of intensity-modulated proton therapy for head and neck cancer

A dense, dilated 3D U-net was trained to predict the 5th and 95th percentile distributions of planning dose plan using the nominal dose and planning CT images. The data set comprised proton therapy plans for 582 H&N cancer patients. Ground truth percentile values were estimated for each patient through 600 dose recalculations, representing randomly sampled uncertainty scenarios. The comprehensive comparisons of different models were conducted for H&N cancer patients, considering those with and without a beam mask (BM) and diverse beam configurations, including varying beam angles, couch angles, and beam numbers. The performance of our model trained based on a mixture of patients with H&N and prostate cancer was also assessed in contrast with models trained based on data specific for patients with cancer at either site. The DL-based model's predictions of percentile dose distributions exhibited excellent agreement with the ground truth dose distributions. The average gamma index with 2 mm/2%, consistently exceeded 97% for both 5th and 95th percentile dose volumes. Mean dose-volume histogram error analysis revealed that predictions from the combined training set yielded mean errors and standard deviations that were generally similar to those in the specific patient training data sets. Our proposed DL-based method for evaluation of the robustness of proton therapy plans provides precise, rapid predictions of percentile dose for a given confidence level regardless of the beam arrangement and cancer site. This versatility positions our model as a valuable tool for evaluating the robustness of proton therapy across various cancer sites.

Enhanced Sampling of Biomolecular Slow Conformational Transitions Using Adaptive Sampling and Machine Learning.

Biomolecular simulations often suffer from the "time scale problem", hindering the study of rare events occurring over extended time scales. Enhanced sampling techniques aim to alleviate this issue by accelerating conformational transitions, yet they typically necessitate well-defined collective variables (CVs), posing a significant challenge. Machine learning offers promising solutions but typically requires rich training data encompassing the entire free energy surface (FES). In this work, we introduce an automated iterative pipeline designed to mitigate these limitations. Our protocol first utilizes a CV-free count-based adaptive sampling method to generate a data set rich in rare events. From this data set, slow modes are identified using Koopman-reweighted time-lagged independent component analysis (KTICA), which are subsequently leveraged by on-the-fly probability enhanced sampling (OPES) to efficiently explore the FES. The effectiveness of our pipeline is demonstrated and further compared with the common Markov State Model (MSM) approach on two model systems with increasing complexity: alanine dipeptide (Ala2) and deca-alanine (Ala10), underscoring its applicability across diverse biomolecular simulations.

Prediction of preterm birth in multiparous women using logistic regression and machine learning approaches

To predict preterm birth (PTB) in multiparous women, comparing machine learning approaches with traditional logistic regression. A population-based cohort study was conducted using data from the Ontario Better Outcomes Registry and Network (BORN). The cohort included all multiparous women who delivered a singleton birth at 20–42 weeks’ gestation in an Ontario hospital between April 1, 2012 and March 31, 2014. The primary outcome was PTB < 37 weeks, with spontaneous PTB analyzed as a secondary outcome. Stepwise logistic regression and the Boruta machine learning were used to select the important variables during the first and second trimester. For building prediction models, the whole data set were divided for the two independent parts: two-third for training the classifiers (Logistic regression, random forests, decision trees, and artificial neural networks) and one-third for model validation. Then, the training data set were balanced by random over sampling technique. The best hyper parameters were obtained by the tenfold cross validation. The performance of all models was evaluated by sensitivity, specificity, positive predictive value, negative predictive value, and the area under the receiver operating characteristics (AUC). The cohort included 145,846 births, of which 8125 (5.57%) were preterm. In first-trimester models, the strongest predictors of PTB were previous PTB, preexisting diabetes, and abnormal pregnancy‐associated plasma protein-A. In the testing data set, the highest predictive ability was seen for artificial neural networks, with an area under the receiver operating characteristic curve (AUC) of 68.8% (95% CI 67.6–70.1%). In second-trimester models, addition of infant sex, attendance at first-trimester appointment, medication exposure, and abnormal alpha-fetoprotein concentrations increased the AUC to 72.1% (95% CI 71.1–73.1%) with logistic regression. With the inclusion of the variable complications during pregnancy, the AUC increased to 80.5% (95% CI 79.6–81.5%) using logistic regression. For both overall and spontaneous PTB, during both the first and second trimesters, models yielded negative predictive values of 97%. Overall, machine learning and logistic regression produced similar performance for prediction of PTB. For overall and spontaneous PTB, both first- and second-trimester models provided negative predictive values of ~ 97%, higher than that of fetal fibronectin.

Fintech model performance analysis in credit loans: evidence from China city commercial banks

ABSTRACT The analysis of bank loan performance has consistently been a central issue in banking research. With the impact of the COVID-19 pandemic, we have witnessed a significant increase in online loans. Our study employs fintech models to develop a big data credit analysis system, evaluating model efficiency and the effect of ‘data organization’ on predictive performance for city commercial bank loans. Our results show that training data size, timeliness, and inter-bank ‘data sharing’ significantly affect model predictions. We also find that traditional econometric models (Logit and Probit) have the weakest predictive ability. However, ‘most advanced’ deep learning models perform worse than machine learning models with lower algorithmic complexity, such as Random Forest. Besides, ‘sharing training data’ generally increases model efficiency, but there are exceptions for larger city commercial banks. This paper provides ample empirical evidence for the application of the fintech model in bank loan credit. It highlights several characteristics of training data, such as size and timeliness, which greatly affect model efficiencies. We also provide policy recommendations for fintech transition and collaboration among municipal banks.

Large language models and humans converge in judging public figures’ personalities

Abstract ChatGPT-4 and 600 human raters evaluated 226 public figures’ personalities using the Ten-Item Personality Inventory. The correlation between ChatGPT-4 and aggregate human ratings ranged from r=.76 to .87, outperforming the models specifically trained to make such predictions. Notably, the model was not provided with any training data or feedback on its performance. We discuss the potential explanations and practical implications of ChatGPT-4’s ability to mimic human responses accurately.

Self-Supervised Learning across the Spectrum

Satellite image time series (SITS) segmentation is crucial for many applications, like environmental monitoring, land cover mapping, and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine-grained annotation. We propose S4, a new self-supervised pretraining approach that significantly reduces the requirement for labeled training data by utilizing two key insights of satellite imagery: (a) Satellites capture images in different parts of the spectrum, such as radio frequencies and visible frequencies. (b) Satellite imagery is geo-registered, allowing for fine-grained spatial alignment. We use these insights to formulate pretraining tasks in S4. To the best of our knowledge, S4 is the first multimodal and temporal approach for SITS segmentation. S4’s novelty stems from leveraging multiple properties required for SITS self-supervision: (1) multiple modalities, (2) temporal information, and (3) pixel-level feature extraction. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially aligned, multimodal, and geographic-specific SITS that serves as representative pretraining data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data. Through a series of extensive comparisons and ablation studies, we demonstrate S4’s ability as an effective feature extractor for downstream semantic segmentation.

Developing an experiment-based strong machine learning model for performance prediction and full analysis of Maisotsenko dewpoint evaporative air cooler

Among all direct and indirect water-based evaporative air coolers, Maisotsenko indirect evaporative cooler (MIEC) is the only one which can supply sub wet-bulb air temperature while adding no moisture to the product air. For any given MIEC, four thermal/fluid parameters including dry channel flow rate, wet-to-dry-side air flow ratio, ambient air temperature and ambient relative humidity impact the performance of the MIEC. Sub-wet bulb supply temperature while having maximum cooling capacity happen only for a specific range of the mentioned thermal/fluid parameters. For any ambient condition and dry side air flow, a magic value of air-flow-ratio can maximize the cooling capacity of the cooler. Thousands of data is required to provide a simultaneous multi-aspect analysis of each parameter (while other parameters vary as well) to identify all possible curve behaviors and secrets of this technology. That is why, for the first time in this research, a strong machine learning (ML) model, is developed for MIEC cooler using 625 validated analytical/experimental-based training data (full factorial design). This model is employed to generate around 3600 unseen data to fully clarify the impact of each parameter (on supply temperature, total cooling capacity, useful cooling capacity and wet-bulb effectiveness) while other parameters are variable too. This strategy makes it possible to see all hidden extremum or peak point where the performance of the cooler is maximized or minimized. Interesting results are found in this research. For any ambient condition, there is an extremum point of the air flow ratio at which the total cooling capacity is maximized. The results indicate that as the dry side air flow increases, this extremum value decreases. For instance, under an ambient condition of 18 °C and 9 % relative humidity, the extremum point of the air flow ratio is 0.45 when the dry side air flow is 0.8 L/s and 0.25 when the dry side air flow is 4.8 L/s. In real-world commercial cooler, this can be simply happened by appropriate re-adjusting the dry and wet side fan speeds.

RODIN: Raw Diffraction Data for Teaching, Training, and Demonstration

<i>RODIN</i>: Raw Diffraction Data for Teaching, Training, and Demonstration