Small Set Research Articles

An electrochemical series for materials

The electrochemical series is a useful tool in electrochemistry, but its effectiveness in materials chemistry is limited by the fact that the standard electrochemical series is based on a relatively small set of reactions, many of which are measured in aqueous solutions. To address this problem, we have used machine learning to create an electrochemical series for inorganic materials from tens of thousands of entries in the Inorganic Crystal Structure Database. We demonstrate that this series is generally more consistent with oxidation states in solid-state materials than the series based on aqueous ions. The electrochemical series was constructed by developing and parameterizing a physical, human-interpretable model of oxidation states in materials. We show that this model enables the prediction of oxidation states from composition in a way that is more accurate than a state-of-the-art transformer-based neural network model. We present applications of our approach to structure prediction, materials discovery, and materials electrochemistry, and we discuss possible additional applications and areas for improvement. To facilitate the use of our approach, we introduce a freely available website and API.

Multiobjective Optimization Strategy for Enhancing the Efficiency and Quality of Organic Thin-Film Manufacturing with Electrohydrodynamic Atomization Coating.

Electrohydrodynamic atomization coating technology is well-suited for micro-/nanoscale thin-film additive manufacturing. However, there are still some challenges in quality control and parameter adjustment during the coating process. Especially when coating on nonconductive and nonhydrophilic substrates, film quality and thickness uniformity are difficult to control. This paper proposes an optimization strategy for enhancing the efficiency and quality of thin-film manufacturing on nonconductive, nonhydrophilic glass substrates. In this paper, a visual inspection system was developed for in situ inspection and identification of droplet deposition states in the substrate surface. Then, the statistical relationship between the operating parameters and the quality of the deposition state was analyzed by response surface methodology. On this basis, machine learning models and intelligent recommendation frameworks for small data sets were developed to rapidly optimize operating parameters and improve the quality of thin-film coating. Optimization strategy developed by applying the principles of statistical modeling, analysis of variance, and global optimization are more efficient and less costly than traditional parameter screening methods. The experimental results show that optimum deposition quality can be obtained with the recommended operating parameters. And, validation results show a 12.8% improvement in film thickness uniformity. At the same time, no mura defects appeared on the thin-film surface. The proposed optimization strategy can improve the efficiency and quality of additive manufacturing of micro and nano thin films and is beneficial for advancing industrial applications of the electrohydrodynamic atomization coating.

Basis Set Extrapolation from the Vanishing Counterpoise Correction Condition.

Basis set extrapolations are typically rationalized either from analytical arguments involving the partial-wave or principal expansions of the correlation energy in helium-like systems or from fitting extrapolation parameters to reference energetics for a small(ish) training set. Seeking to avoid both, we explore a third alternative: extracting extrapolation parameters from the requirement that the BSSE (basis set superposition error) should vanish at the complete basis set limit. We find this to be a viable approach provided that the underlying basis sets are not too small and reasonably well balanced. For basis sets not augmented by diffuse functions, BSSE minimization and energy fitting yield quite similar parameters.

Modeling Android Security Vulnerabilities: Insights from Statistical Distributions

Android operating system is a mobile operating system that supports multimedia features. Android offers a wide range of applications and integrated features for playing, recording, editing and sharing audio, video, images and other multimedia content. Most Android devices include cameras, speakers, microphones, and other multimedia components. In software security, vulnerabilities are critical concerns that often emerge during software development. Predicting these vulnerabilities post-release is essential for risk assessment and mitigation. While various models have been explored, the Android operating system remains relatively uncharted. This study delves into modeling Android security vulnerabilities using different statistical distributions, comparing their suitability to the widely-used Alhazmi-Malaiya Logistic (AML) model. Data from the National Vulnerability Database (NVD) spanning 2016 to 2018, along with Common Vulnerability Scoring System (CVSS) scores, was analyzed. The study evaluates several distribution models, including Logistic, Weibull, Nakagami, Gamma, and Log-logistic, for monthly vulnerability counts and average monthly impact values. Goodness-of-fit tests and information criteria were applied for model robustness assessment. The findings offer valuable insights for researchers and Android software developers, aiding prediction, risk assessment, resource allocation, and research direction. Logistic and Nakagami distributions emerged as the best-fit models for average monthly impact values and monthly vulnerability counts, respectively. Finally, statistical methods perform better against known artificial intelligence methods for small data sets or more clearly defined data due to their flexible features such as comprehensibility, amount of data, need for calculation, and data independence.

Screening of Anti-Prion Compounds Using the Protein Misfolding Cyclic Amplification Technology.

Prion diseases are 100% fatal infectious neurodegenerative diseases affecting the brains of humans and other mammals. The disease is caused by the formation and replication of prions, composed exclusively of the misfolded prion protein (PrPSc). We invented and developed the protein misfolding cyclic amplification (PMCA) technology for in vitro prion replication, which allow us to replicate the infectious agent and it is commonly used for ultra-sensitive prion detection in biological fluids, tissues and environmental samples. In this article, we studied whether PMCA can be used to screen for chemical compounds that block prion replication. A small set of compounds previously shown to have anti-prion activity in various systems, mostly using cells infected with murine prions, was evaluated for their ability to prevent the replication of prions. Studies were conducted simultaneously with prions derived from 4 species, including human, cattle, cervid and mouse. Our results show that only one of these compounds (methylene blue) was able to completely inhibit prion replication in all species. Estimation of the IC50 for methylene blue inhibition of human prions causing variant Creutzfeldt-Jakob disease (vCJD) was 7.7 μM. Finally, we showed that PMCA can be used for structure-activity relationship studies of anti-prion compounds. Interestingly, some of the less efficient prion inhibitors altered the replication of prions in some species and not others, suggesting that PMCA is useful for studying the differential selectivity of potential drugs.

Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data.

In the field of automatic optical inspection (AOI), this study presents innovative strategies to enhance object detection accuracy while minimizing dependence on large annotated datasets. We initially developed a defect detection model using a dataset of 3579 images across 32 categories, created in collaboration with a major Taiwanese panel manufacturer. This model was evaluated using 12,000 ambiguously labeled images, with improvements achieved through data augmentation and annotation refinement. To address the challenges of limited labeled data, we proposed the Adaptive Fused Semi-Supervised Self-Learning (AFSL) method. This approach, designed for anchor-based object detection models, leverages a small set of labeled data alongside a larger pool of unlabeled data to enable continuous model optimization. Key components of AFSL include the Bounding Box Assigner, Adaptive Training Scheduler, and Data Allocator, which together facilitate dynamic threshold adjustments and balanced training, significantly enhancing the model's performance on AOI datasets. The AFSL method improved the mean average precision (mAP) from 43.5% to 57.1% on the COCO dataset and by 2.6% on the AOI dataset, demonstrating its effectiveness in achieving high levels of precision and efficiency in AOI with minimal labeled data.

Employing Metadynamics to Predict the Membrane Partitioning of Carboxy-2H-Azirine Natural Products.

Natural products containing the carboxy-2H-azirine moiety are an exciting target for investigation due to their broad-spectrum antimicrobial activity and new chemical space they afford for novel therapeutic development. The carboxy-2H-azirine moiety, including those appended to well-characterized chemical scaffolds, is understudied, which creates a challenge for understanding potential modes of inhibition. In particular, some known natural product carboxy-2H-azirines have long hydrophobic tails, which could implicate them in membrane-associated processes. In this study, we examined a small set of carboxy-2H-azirine natural products with varied structural features that could alter membrane partitioning. We compared the predicted membrane partitioning and alignment of these compounds to those of established membrane embedders with similar chemical scaffolds. To accomplish this, we developed parameters within the framework of the CHARMM36 force field for the 2H-azirine functional group and performed metadynamics simulations of the partitioning into a model bacterial membrane from aqueous solution. We determined that the carboxy-2H-azirine functional group is strongly hydrophilic, imbuing the long-chain natural products with amphipathicity similar to the known membrane-embedding molecules to which they were compared. For the long-chain analogs, the carboxy-2H-azirine head group stays within 1 nm of the phosphate layer, while the hydrophobic tail sits within the membrane. The carboxy-2H-azirine lacking the long alkyl chain instead partitions completely into aqueous solution.

External Validation Demonstrates Machine Learning Models Outperform Human Experts in Prediction of Objective and Patient-Reported Overactive Bladder Treatment Outcomes

ObjectivesTo predict treatment response for overactive bladder (OAB) for a specific patient remains elusive. We sought to develop accurate models using machine learning for prediction of objective and patient-reported treatment response to intravesical botulinum toxin (OBTX-A) injection. We sought to validate the models in a challenging setting using an external dataset of a markedly different patient cohort and dosing regimen. We hypothesized the model would outperform human experts and top available algorithms. MethodsAlgorithms using “operator splitting” designed for accuracy and efficiency even in small training data sets with variable completeness, were trained to predict objective response and patient-reported symptomatic improvement using the ROSETTA trial cohort and validated using the ABC trial cohort of patients who underwent OBTX-A. Areas under the curve (AUC) of algorithms were compared to the top publicly-available machine learning classifier XGBoost, logistic regression with cross validation, and human expert predictions in the external validation set. ResultsIn the validation set, the operator splitting neural network had AUC 0.66 and outperformed XGBoost with DART (top available machine learning classifier, AUC 0.58), logistic regression (AUC 0.55), and human experts (AUC 0.47 – 0.53) for prediction of clinical responder status. It was similarly accurate in prediction of patient subjective improvement in symptoms following OBTX-A (AUC: 0.64), again outperforming other algorithms and human experts (AUC 0.41 – 0.62). ConclusionsThe neural network outperformed human experts and other machine learning approaches in prediction of objective and patient-reported OBTX-A outcomes for overactive bladder in a challenging independent validation cohort. Clinical implementation could improve counseling and treatment selection.

Masked particle modeling on sets: towards self-supervised high energy physics foundation models

Abstract We propose masked particle modeling (MPM) as a self-supervised method for learning generic, transferable, and reusable representations on unordered sets of inputs for use in high energy physics (HEP) scientific data. This work provides a novel scheme to perform masked modeling based pre-training to learn permutation invariant functions on sets. More generally, this work provides a step towards building large foundation models for HEP that can be generically pre-trained with self-supervised learning and later fine-tuned for a variety of down-stream tasks. In MPM, particles in a set are masked and the training objective is to recover their identity, as defined by a discretized token representation of a pre-trained vector quantized variational autoencoder. We study the efficacy of the method in samples of high energy jets at collider physics experiments, including studies on the impact of discretization, permutation invariance, and ordering. We also study the fine-tuning capability of the model, showing that it can be adapted to tasks such as supervised and weakly supervised jet classification, and that the model can transfer efficiently with small fine-tuning data sets to new classes and new data domains.

Large Language Models to Help Appeal Denied Radiotherapy Services

PURPOSE Large language model (LLM) artificial intelligences may help physicians appeal insurer denials of prescribed medical services, a task that delays patient care and contributes to burnout. We evaluated LLM performance at this task for denials of radiotherapy services. METHODS We evaluated generative pretrained transformer 3.5 (GPT-3.5; OpenAI, San Francisco, CA), GPT-4, GPT-4 with internet search functionality (GPT-4web), and GPT-3.5ft. The latter was developed by fine-tuning GPT-3.5 via an OpenAI application programming interface with 53 examples of appeal letters written by radiation oncologists. Twenty test prompts with simulated patient histories were programmatically presented to the LLMs, and output appeal letters were scored by three blinded radiation oncologists for language representation, clinical detail inclusion, clinical reasoning validity, literature citations, and overall readiness for insurer submission. RESULTS Interobserver agreement between radiation oncologists' scores was moderate or better for all domains (Cohen's kappa coefficients: 0.41-0.91). GPT-3.5, GPT-4, and GPT-4web wrote letters that were on average linguistically clear, summarized provided clinical histories without confabulation, reasoned appropriately, and were scored useful to expedite the insurance appeal process. GPT-4 and GPT-4web letters demonstrated superior clinical reasoning and were readier for submission than GPT-3.5 letters ( P < .001). Fine-tuning increased GPT-3.5ft confabulation and compromised performance compared with other LLMs across all domains ( P < .001). All LLMs, including GPT-4web, were poor at supporting clinical assertions with existing, relevant, and appropriately cited primary literature. CONCLUSION When prompted appropriately, three commercially available LLMs drafted letters that physicians deemed would expedite appealing insurer denials of radiotherapy services. LLMs may decrease this task's clerical workload on providers. However, LLM performance worsened when fine-tuned with a task-specific, small training data set.

POU4F3 Is a Sensitive and Specific Marker of Merkel Cell Carcinoma

Although of therapeutic importance, a single sensitive and specific immunostain to distinguish Merkel cell carcinoma (MCC) from mimics is not currently available. In addition, single tumor cells are difficult to detect in sentinel lymph node biopsy. Leveraging publicly available data sets of 9264 solid tumors and >600,000 single-cell transcriptomes, we identified POU4F3 to be a specific marker of MCC. Analyses of Pan-Cancer RNA bulk sequencing data of 24 tumor types from Tumor Cancer Genomic Atlas data sets as well as non-Tumor Cancer Genomic Atlas small cell lung carcinoma and MCC data sets confirmed POU4F3 specificity for MCC. Single-cell RNA-sequencing analyses also confirmed the lack of POU4F3 expression in lung small cell carcinoma as well as a variety of normal tissues. Nuclear POU4F3 immunohistochemical expression was noted in 98.7% of 153 MCCs and in only 1.7% of mimics (3 of 180 cases, including 95 small cell carcinomas, of which 55 were from lungs and the remainder from other sites). Three POU4F3-positive non-MCC cases were from lungs (2 cases) and vagina (1 case). All 153 tested MCC cases were negative for ASCL1, a key transcriptional regulator highly expressed in small cell lung carcinoma. NeuroD1 was seen in a subset of MCC cases (20.9%, 32/153). POU4F3 immunostain was performed on 29 sentinel lymph nodes, and strong POU4F3 nuclear expression facilitated the ease of metastasis detection, even single tumor cells. Our study built on prior works shows that POU4F3 is a sensitive and specific clinical marker of MCC.

An exponential reduction in training data sizes for machine learning derived entanglement witnesses

Abstract We propose a support vector machine (SVM) based approach for generating an entanglement witness that requires exponentially less training data than previously proposed methods. SVMs generate hyperplanes represented by a weighted sum of expectation values of local observables whose coefficients are optimized to sum to a positive number for all separable states and a negative number for as many entangled states as possible near a specific target state. Previous SVM-based approaches for entanglement witness generation used large amounts of randomly generated separable states to perform training, a task with considerable computational overhead. Here, we propose a method for orienting the witness hyperplane using only the significantly smaller set of states consisting of the eigenstates of the generalized Pauli matrices and a set of entangled states near the target entangled states. With the orientation of the witness hyperplane set by the SVM, we tune the plane's placement using a differential program that ensures perfect classification accuracy on a limited test set as well as maximal noise tolerance. For \(N\) qubits, the SVM portion of this approach requires only \(O(6^N)\) training states, whereas an existing method needs \(O(2^{4^N})\). We use this method to construct witnesses of 4 and 5 qubit GHZ states with coefficients agreeing with stabilizer formalism witnesses to within 3.7 percent and 1 percent, respectively. We also use the same training states to generate novel 4 and 5 qubit W state witnesses. Finally, we computationally verify these witnesses on small test sets and propose methods for further verification.

Piecewise Developable Modeling via Implicit Neural Deformation and Feature-Guided Cutting.

We propose a novel and automatic method to model shapes using a small set of discrete developable patches. Central to our approach is using implicit neural shape representation that makes our algorithm independent of tessellation and allows us to obtain the Gaussian curvature of each point analytically. With this powerful representation, we first deform the input shape to be an almost developable shape with clear and sparse salient feature curves. Then, we convert the deformed implicit field to a triangle mesh, which is further cut to disk topology along parts of the sparse feature curves. Finally, we achieve the resulting piecewise developable mesh by alternatingly optimizing discrete developability, enforcing manufacturability constraints, and merging patches. The feasibility and practicability of our method are demonstrated over various shapes. Compared to the state-of-the-art methods, our method achieves a better tradeoff between the number of developable patches and the approximation error.

LoGAN: local generative adversarial network for novel structure prediction

Abstract The efficient generation and filtering of candidate structures for new materials is becoming increasingly important as starting points for computational studies. In this work, we introduce an approach to Wasserstein generative adversarial networks for predicting unique crystal and molecular structures. Leveraging translation- and rotation-invariant atom-centered local descriptors addresses some of the major challenges faced by similar methods. Our models require only small sets of known structures as training data. Furthermore, the approach is able to generate both non-periodic and periodic structures based on local coordination. We showcase the data efficiency and versatility of the approach by recovering all stable C5H12O isomers using only 39 C4H10O and C6H14O training examples, as well as a few randomly selected known low-energy SiO2 crystal structures utilizing only 167 training examples of other SiO2 crystal structures. We also introduce a filtration technique to reduce the computational cost of subsequent characterization steps by selecting samples from unique basins on the potential energy surface, which allows to minimize the number of geometry relaxations needed after structure generation. The present method thus represents a new, versatile approach to generative modeling of crystal and molecular structures in the low-data regime, and is available as open-source.

Bootstrap inference on a factor model based average treatment effects estimator

We propose a novel bootstrap procedure for conducting inference for factor model-based average treatment effects estimators. Our method overcomes bias inherent to existing bootstrap procedures and substantially improves upon existing large sample normal inference theory in small sample settings. The finite sample improvements arising from the use of our proposed procedure are illustrated via a set of Monte Carlo simulations, and formal justification for the procedure is outlined.

Automatic Road Crack Detection Using Convolutional Neural Network Based on Semi-Supervised Learning

Crack detection in pavements is a critical task for infrastructure maintenance, but it often requires extensive manual labeling of training samples, which is both time-consuming and labor-intensive. To address this challenge, this paper proposes a semi-supervised learning approach based on a DenseNet classification model to detect pavement cracks more efficiently. The primary objective is to leverage a small set of labeled samples to improve the model's performance by incorporating a large number of unlabeled samples through semi-supervised learning. This method enhances the DenseNet model's ability to generalize by iteratively learning from new unlabeled datasets. As a result, the proposed approach not only reduces the need for extensive manual labeling but also mitigates issues related to label inconsistency and errors in the original labels. The experimental results demonstrate that the semi-supervised DenseNet model achieves a prediction precision of 96.77% and a recall of 94.17%, with an F1 score of 95.45% and an Intersectidn over Union (IoU) of 91.30%. These metrics highlight the model's high accuracy and effectiveness in crack detection. The proposed method not only improves label quality and model performance but also offers practical value for engineering applications in the field of pavement maintenance, making it a valuable tool for infrastructure management.

Cross-lingual Natural Language Processing on Limited Annotated Case/Radiology Reports in English and Japanese: Insights from the Real-MedNLP Workshop.

Textual datasets (corpora) are crucial for the application of natural language processing (NLP) models. However, corpus creation in the medical field is challenging, primarily because of privacy issues with raw clinical data such as health records. Thus, the existing clinical corpora are generally small and scarce. Medical NLP (MedNLP) methodologies perform well with limited data availability. We present the outcomes of the Real-MedNLP workshop, which was conducted using limited and parallel medical corpora. Real-MedNLP exhibits three distinct characteristics: (1) Limited Annotated Documents: The training data comprises only a small set (approximately 100) of case reports (CRs) and radiology reports (RRs) that have been annotated. (2) Bilingually Parallel: The constructed corpora are parallel in Japanese and English. (3) Practical Tasks: The workshop addresses fundamental tasks, such as named entity recognition and applied practical tasks. We propose three tasks: named entity recognition (NER) of approximately 100 available documents (Task 1), NER based only on annotation guidelines for humans (Task 2), and clinical applications (Task 3) consisting of adverse drug effects (ADE) detection for CRs and identical case identification (CI) for RRs. Nine teams participated in this study. The best systems achieved 0.65 and 0.89 F1-scores for CRs and RRs in Task 1, whereas the top scores in Task 2 decreased by 50-70%. In Task 3, ADE reports were detected by up to 0.64 F1-score, and CI scored up to 0.96 binary accuracy. Most systems adopt medical-domain-specific pre-trained language models using data augmentation methods. Despite the challenge of limited corpus size in Tasks 1 and 2, recent approaches are promising because the partial match scores reached approximately 0.8-0.9 F1-scores. Task 3 applications revealed that the different availabilities of external language resources affected the performance per language.

Physics-informed machine learning framework for creep-fatigue life prediction of a Ni-based superalloy using ensemble learning

Under extreme environments, critical components of nuclear power equipment suffer creep-fatigue damage accumulation due to the complex load, threatening the safety operation of equipment. Therefore, creep-fatigue life assessment is crucial to ensure the structural integrity and performance requirement of nuclear power equipment during service. To address the poor prediction accuracy under small data sets, this paper developed a physics-informed machine learning (PIML) framework to model the creep-fatigue interaction behavior of a Ni-based superalloy. In this work, an ensemble learning approach is considered to embed physical information into machine learning (ML) models for the creep-fatigue life prediction. The comparison of each model shows that PIML is capable of utilizing the strengths of purely data-driven models and physics-based models to provide excellent prediction accuracy and promote the generalization ability.

Deep-learning-based image reconstruction with limited data: generating synthetic raw data using deep learning.

Deep learning has shown great promise for fast reconstruction of accelerated MRI acquisitions by learning from large amounts of raw data. However, raw data is not always available in sufficient quantities. This study investigates synthetic data generation to complement small datasets and improve reconstruction quality. An adversarial auto-encoder was trained to generate phase and coil sensitivity maps from magnitude images, which were combined into synthetic raw data. On a fourfold accelerated MR reconstruction task, deep-learning-based reconstruction networks were trained with varying amounts of training data (20 to 160 scans). Test set performance was compared between baseline experiments and experiments that incorporated synthetic training data. Training with synthetic raw data showed decreasing reconstruction errors with increasing amounts of training data, but importantly this was magnitude-only data, rather than real raw data. For small training sets, training with synthetic data decreased the mean absolute error (MAE) by up to 7.5%, whereas for larger training sets the MAE increased by up to 2.6%. Synthetic raw data generation improved reconstruction quality in scenarios with limited training data. A major advantage of synthetic data generation is that it allows for the reuse of magnitude-only datasets, which are more readily available than raw datasets.

Is libraries' use of social media ethical? A phenomenographic investigation of Twitter (X) use at the Bodleian Libraries

PurposeSocial media (SM) has been heavily criticised in recent years for its damaging effects on societies globally. Tasked with empowering those same societies, libraries’ continued use of SM is considered ethically contentious. This study presents the findings of a University of Sheffield study that investigated the ethical tension between libraries and their use of SM by aiming to establish whether: (1) libraries’ use of SM is ethically motivated; (2) ethically informed; (3) and compatible with codes of ethics in the Library and Information Science (LIS) sector.Design/methodology/approachA phenomenographic approach was employed to gather and analyse the data for this study, which is drawn from the transcripts of seven online interviews with Bodleian Libraries staff who used Twitter, now X, in a professional capacity.FindingsThree categories of description were identified among participants: (1) Collectivist conception; (2) Settled conception; (3) Questioning conception. The categories are bound by a shared context of conceptualisation made up of a small set of internal and external influences discussed in the interviews which affected all participants to varying degrees.Originality/valueThe findings were used to support the following determinations: (1) Libraries’ use of SM is ethically motivated. (2) Libraries’ use of SM is ethically informed, in part. Due to lack of evidence, no determination was made about whether libraries’ use of SM is compatible with codes of ethics in the LIS sector. Recommendations for LIS professions and professional bodies are offered based on these determinations.