Learning Methods Research Articles

Research on positive and negative sample sampling strategies in contrastive learning

Abstract. Contrastive learning is an important technique in the field of machine learning for learning data representations. In the field of self-supervised visual representation learning, the strategy of positive and negative sample selection is key to improving the efficiency and effectiveness of model learning. Traditional self-supervised learning methods often employ random sampling strategies to select positive and negative samples, but this approach may result in uneven sample quality when dealing with complex datasets, thereby affecting the learning outcomes. To alleviate this problem, this study is dedicated to exploring more effective strategies for positive and negative sample selection and processing to optimize the self-supervised learning process. To this end, we propose an improved method of self-supervised learning called contrastive learning with enhanced diversity. On the one hand, this method utilizes the weight parameters of the DINO pre-trained model to initialize the feature extraction network of SimCLR, providing more accurate calculations of feature similarity. On the other hand, by setting a threshold on the feature similarity matrix and penalizing (subtracting 0.5 from) similarity scores that do not exceed this threshold, we reduce the excessive impact of high similarity scores on model training, thereby helping the model better distinguish between positive and negative samples. In downstream image classification tasks, we conducted detailed evaluations of the improved model, specifically including fine-tuning and linear evaluation aspects. Experiments show that the proposed approach improves the performance of the loss functions and improves the accuracy of the proposed SimCLR model.

Fatigue trajectories by wearable remote monitoring of breast cancer patients during radiotherapy

The aim of this pilot study was to assess the compliance of breast cancer (BC) patients with fitness tracker (FT) monitoring program during radiotherapy (RT) and to characterize radiation-induced fatigue (RIF) status through objective evaluation using FT-collected parameters. Thirty-six BC patients were invited to wear FT during their RT course for continuous monitoring of heart rate (HR) and step counts (STP). RIF assessment was performed weekly, according to CTCAE v5.0 and dichotomized into G0 vs. any-grade. A novel concept based on patient Repeated Activity Window (RAW) was introduced to evaluate HR and STP variations during RT. Several Machine Learning (ML) methods were trained to characterize RIF on the basis of HR and STP collected data. RIF of any grade was reported by 17 out of 36 patients (47%) included in the study. None of patient clinical variables were significantly correlated with RIF. All patients accepted the FT monitoring program, and for 32 patients FT collection efficiency was greater than 60%. For each patient, a distinct distribution of RAWs was identifiable over RT and across the entire patient cohort, with a total of 7950 RAWs processed. Six features related to RAWs, HR and STP were identified as associated with RIF. The best-performing classifier was the Bagged Trees model, showing a cross-validated ROC-AUC of 89% (95% CI 88–90%). This study confirms the feasibility of continuous biomedical monitoring of BC patients by FT. We successfully identify objective indicators of RIF through HR and STP variation measures within each patient’s RAW, thus providing a novel and practical approach to assess and manage RIF. This can significantly aid medical staff in evaluating RIF trajectories, potentially leading to better individualized care strategies and improved patient outcomes.

Machine learning-based optimal temperature management model for safety and quality control of perishable food supply chain

The management of a food supply chain is difficult and complex because of the product's short shelf-life, time-sensitivity, and perishable nature which must be carefully considered to minimize food waste. Temperature-controlled perishable food supply chain provides the highly crucial facilities necessary to maintain the quality and safety of the product. The storage temperature is the most vital factor in maintaining both the quality and shelf-life of a perishable food. Adequate storage temperature control ensures that perishable foods are transported to the end-users in good quality and safe to consume. This paper presents perishable food storage temperature control through mathematical optimal control model where the storage temperature is regarded as the control variable and the deterioration of the perishable food’s quality follows the first-order reaction. The optimal storage temperature for a single perishable food is determined by applying the Pontryagin's maximum principle to solve the optimal control model problem. For multi-temperature commodities supply chain, an unsupervised machine learning (ML) method, called k-means clustering technique is used to determine the temperature clusters for a range of perishables. Based on descriptive analysis, it is observed that the k-means clustering technique is effective in identifying the best suitable storage temperature clusters for quality control of multi-commodity supply chain.

Processing and Comparison of GBoost, XGBoost, and Random Forest in Titanic Survival Prediction

Abstract. Contemporarily, the machine learning has evolved from its early concepts to a sophisticated field consisting of advanced algorithms and diverse applications. Tree-based classification models have become powerful tools for complex predictive challenges. In this study, the effectiveness of tree-based classification models, such as Random Forest, XGBoost, and Gradient Boosting, is examined on the Titanic survival prediction challenge, which originates from the 1912 Titanic disaster. Passengers survival and death were influenced by various factors in this disaster. By using features such as gender, age, and class, and the survival outcome as the target variable, a binary classification model is developed to predict each passenger's survival status. The study includes data preprocessing, feature selection based on a foundational model, and model training. After the construction, hyper-parameters tuning, and cross-validation of three classifiers, this research compares and analyzes the performance scores to evaluate the characteristics of these tree-based learning methods, aiming to provide a reference for the similar applications.

Unveiling Tim-3 immune checkpoint expression in hepatocellular carcinoma through abdominal contrast-enhanced CT habitat radiomics

IntroductionImmune checkpoint inhibitors (ICIs) are important systemic therapeutic agents for hepatocellular carcinoma (HCC), among which T-cell immunoglobulin and mucin-domain containing protein 3 (Tim-3) is considered an emerging target for ICI therapy. This study aims to evaluate the prognostic value of Tim-3 expression and develop a predictive model for Tim-3 infiltration in HCC.MethodsWe collected data from 424 HCC patients in The Cancer Genome Atlas (TCGA) and data from 102 pathologically confirmed HCC patients from our center for prognostic analysis. Multivariate Cox regression analyses were performed on both datasets to determine the prognostic significance of Tim-3 expression. In radiomics analysis, we used the K-means algorithm to cluster regions of interest in arterial phase enhancement and venous phase enhancement images from patients at our center. Radiomic features were extracted from three subregions as well as the entire tumor using pyradiomics. Five machine learning methods were employed to construct Habitat models based on habitat features and Rad models based on traditional radiomic features. The predictive performance of the models was compared using ROC curves, DCA curves, and calibration curves.ResultsMultivariate Cox analyses from both our center and the TCGA database indicated that high Tim-3 expression is an independent risk factor for poor prognosis in HCC patients. Higher levels of Tim-3 expression were significantly associated with worse prognosis. Among the ten models evaluated, the Habitat model constructed using the LightGBM algorithm showed the best performance in predicting Tim-3 expression status (training set vs. test set AUC 0.866 vs. 0.824).DiscussionThis study confirmed the importance of Tim-3 as a prognostic marker in HCC. The habitat radiomics model we developed effectively predicted intratumoral Tim-3 infiltration, providing valuable insights for the evaluation of ICI therapy in HCC patients.

Gravity Predictions in Data-Missing Areas Using Machine Learning Methods

Gravity data, comprising a key foundational dataset, are crucial for various research, including land subsidence monitoring, geological exploration, and navigational positioning. However, the collection of gravity data in specific regions is difficult because of environmental, technical, and economic constraints, resulting in a non-uniform distribution of the observational data. Traditionally, interpolation methods such as Kriging have been widely used to deal with data gaps; however, their predictive accuracy in regions with sparse data still needs improvement. In recent years, the rapid development of artificial intelligence has opened up a new opportunity for data prediction. In this study, utilizing the EGM2008 satellite gravity model, we conducted a comprehensive analysis of three machine learning algorithms—random forest, support vector machine, and recurrent neural network—and compared their performances against the traditional Kriging interpolation method. The results indicate that machine learning methods exhibit a marked advantage in gravity data prediction, significantly enhancing the predictive accuracy.

Seasonal prediction of North Atlantic sea surface temperature anomalies using the LSTM machine learning method

Seasonal prediction of North Atlantic sea surface temperature anomalies using the LSTM machine learning method

SLOctolyzer: Fully Automatic Analysis Toolkit for Segmentation and Feature Extracting in Scanning Laser Ophthalmoscopy Images.

The purpose of this study was to introduce SLOctolyzer: an open-source analysis toolkit for en face retinal vessels in infrared reflectance scanning laser ophthalmoscopy (SLO) images. SLOctolyzer includes two main modules: segmentation and measurement. The segmentation module uses deep learning methods to delineate retinal anatomy, and detects the fovea and optic disc, whereas the measurement module quantifies the complexity, density, tortuosity, and caliber of the segmented retinal vessels. We evaluated the segmentation module using unseen data and measured its reproducibility. SLOctolyzer's segmentation module performed well against unseen internal test data (Dice for all-vessels = 0.91; arteries = 0.84; veins = 0.85; optic disc = 0.94; and fovea = 0.88). External validation against severe retinal pathology showed decreased performance (Dice for arteries = 0.72; veins = 0.75; and optic disc = 0.90). SLOctolyzer had good reproducibility (mean difference for fractal dimension = -0.001; density = -0.0003; caliber = -0.32 microns; and tortuosity density = 0.001). SLOctolyzer can process a 768 × 768 pixel macula-centered SLO image in under 20 seconds and a disc-centered SLO image in under 30 seconds using a laptop CPU. To our knowledge, SLOctolyzer is the first open-source tool to convert raw SLO images into reproducible and clinically meaningful retinal vascular parameters. It requires no specialist knowledge or proprietary software, and allows manual correction of segmentations and re-computing of vascular metrics. SLOctolyzer is freely available at https://github.com/jaburke166/SLOctolyzer. SLO images are captured simultaneous to optical coherence tomography (OCT), and we believe SLOctolyzer will be useful for extracting retinal vascular measurements from large OCT image sets and linking them to ocular or systemic diseases.

Peak-Based Machine Learning for Plastic Type Classification in Time-of-Flight Secondary Ion Mass Spectrometry.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) measurement data and machine learning were used in this work to classify six different types of plastics. In order to take into account the characteristics of the measurement data, the local maxima of the measurement data were first examined in a preprocessing step. Several machine learning methods were then implemented to create a model that could successfully classify the plastics. To visualize the data distribution, we applied a dimensionality reduction method, namely, principal component analysis. Finally, to distinguish between the six types of plastics, we conducted an ensemble analysis using four tree-based algorithms: decision tree, random forest, gradient boosting, and LIGHTGBM. This approach can identify the feature importance of plastic samples and allow the inference of the chemical properties of each plastic type. In this way, ToF-SIMS data could be utilized to successfully classify plastics and enhance explainability.

A novel epilepsy detection approach using intrinsic multiscale entropy analysis and DSEAM-enhanced 1D-ResNets

Abstract Epilepsy, a prevalent neurological disorder, typically requires a complex diagnostic process involving medical history inquiry, physical examination, head computed tomography, and electroencephalogram (EEG) visual examination. Among the existing epilepsy automated detection algorithms, machine learning methods require manual feature selection. Most of deep learning algorithms for automatic detection have high complexity and computational complexity. To address this issue, this study proposes a high-precision, robust, and low computational epilepsy automatic detection algorithm based on EEG signal processing. The study utilizes ensemble empirical mode decomposition to preprocess the original EEG signal, breaking it down into intrinsic mode functions (IMFs) across various frequency bands. These IMFs contain information about epilepsy occurrence within the signal at different frequency bands. To enhance computational efficiency and reduce data dimension, the refined composite multiscale dispersion entropy of each IMF is further computed at different scales, referred to as intrinsic multiscale entropy (IME) analysis. IME analysis consolidates epilepsy occurrence information from EEG signals across different frequency bands and scales, linking entropy values to generate feature vectors. Drawing inspiration from successful deep residual networks and Squeeze-and-Excitation networks, the study introduces a double Squeeze-and-Excitation attention module enhanced one-dimensional residual network to classify one-dimensional feature vectors. The proposed method was tested on epilepsy dataset from University of Bonn, and the results demonstrated superior classification performance. In the experiment, the distinction between normal and epileptic EEG signals achieved 100% accuracy rate, while distinguishing between normal, epileptic interval, and epileptic EEG signals achieved accuracy rate of 99.41%.

Student Motivation and Effectiveness of Online Learning: A Qualitative Perspective

Even though the COVID-19 era has passed 4 years ago, online learning is still implemented as a hybrid or a necessity, especially in high-level health education. Online learning has many positive sides, such as being able to learn anywhere, cost-effective, and being more time flexible, making online learning considered an efficient learning method compared to face-to-face. As a result, in the current situation (post-pandemic) online learning is still valid even though it is sometimes interspersed with offline learning. This research aims to determine the obstacles faced by students in conducting online learning and their impact on learning motivation through interviews. Apart from that, it also aims to study the effectiveness of online learning through observation. Based on the research objectives, observations and interviews are needed to provide a deeper understanding, and provide an idea of whether there is and how much effectiveness and motivation to learn in online learning. It is expected that this research can also provide updated data that can be useful not only for researchers but also for readers. In addition, it is envisaged that this study would deepen the understanding of the efficiency of online learning and will motivate students. Keywords: student motivation, learning effectiveness online learning

Correction of Sensible Heat Flux from Flux-Gradient Method to Eddy Covariance Method Based on Multi-Layer Perceptron

Abstract. Understanding surface-atmosphere interactions is critical for environmental and meteorological studies. Sensible heat flux, a key component in this interaction, is typically measured using methods such as Eddy Covariance (EC) and Flux-Gradient (FG). The EC method, known for its high temporal resolution and direct measurement capabilities of wind speed, temperature, and humidity changes, requires the use of expensive and complex equipment, making it costly and challenging to implement. On the contrary, the FG method is more accessible and economical, relying on simpler instruments, but often lacks the precision of the EC method. To harness the benefits of both methods, this article uses the Multi-Layer Perceptron (MLP) machine learning method to enhance the accuracy of the FG method's sensible heat flux calculations. Through the MLP model, this paper aims to determine the optimal parameter settings for the specific measurement environment, thereby improving the FG methods accuracy. The data was measured from Guandu Village, Anhui Province, China. This research seeks to demonstrate that the trained MLP model can be applied to similar measurement environments, thus enhancing the FG method's applicability and precision.

Prognostic characteristics and drug sensitivity analysis of hepatocellular carcinoma based on histone modification-related genes: a multi-omics integrated study revealing potential therapeutic targets and individualized treatment strategies

BackgroundHepatocellular carcinoma (HCC) ranks among the most prevalent and lethal malignancies worldwide. Histone modifications (HMs) play a pivotal role in the initiation and progression of HCC. However, our understanding of HMs in HCC remains limited due to the disease’s heterogeneity and the complexity of HMs.MethodsWe integrated multi-omics data from multiple cohorts, including single-cell RNA sequencing, bulk RNA sequencing, and clinical information. Weighted gene co-expression network analysis (WGCNA) and consensus clustering were employed to identify histone-related genes. We developed a histone modification-related signature (HMRS) using 117 machine learning methods. Comprehensive analyses of molecular characteristics, immune landscape, and drug sensitivity associated with the HMRS were performed.ResultsThrough integrative analysis, we defined 110 histone-related genes and identified 45 HCC-HM-related genes (HCC-HMRgenes). The HMRS demonstrated robust prognostic value across multiple cohorts. Patients with high HMRS scores exhibited distinct genomic alterations, including higher tumor heterogeneity and TP53 mutations. The high-risk group showed enrichment in cell cycle, DNA repair, and metabolic pathways. Immune landscape analysis revealed significant differences in immune cell infiltration and pathway activities between high- and low-risk groups. Drug sensitivity prediction suggested potential therapeutic strategies for different risk groups.ConclusionOur study provides a comprehensive understanding of HMs in HCC and establishes a robust prognostic signature. The HMRS not only stratifies patients into distinct risk groups but also offers insights into underlying molecular mechanisms, immune characteristics, and potential therapeutic strategies, paving the way for personalized medicine in HCC.

Leaf disease detection and classification with a CNN-LSTM hybrid model

Abstract. Smart agriculture refers to the use of modern information technology, Internet of Things technology, and artificial intelligence to achieve accurate management, efficient operation and sustainable development of the entire process of agricultural production. Smart agriculture mainly includes the application of data collection and analysis, intelligent agricultural machinery, precise fertilization, disease and pest detection and agricultural products traceability. Leaf disease detection and classification are considered as challenging yet important tasks in smart agriculture. Deep leaning methods have been proven effective for these image-based recognition tasks. In this study, two advanced deep learning methods, namely, Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) are combined together to achieve a further improvement. Numerical results demonstrate that the proposed method outperforms both CNN and LSTM variants.

Federated machine learning in finance: A systematic review on technical architecture and financial applications

Abstract. With the advancement of technology, an increasing amount of data is stored online, including substantial amounts of personal information. Especially in the financial industry, where large sums of money are often involved, protecting personal privacy is crucial. Federated learning, as a privacy-preserving distributed machine learning method, offers a solution to these challenges by enabling data privacy while addressing the difficulties of data sharing in the financial sectorissues that frequently impede innovation, risk management, and fraud detection. This paper delves into the principles of federated learning algorithms, exploring their mechanisms in detail. Besides, applications and case studies in applications such as financial fraud detection, supply chain financing prediction, and other financial services are examined. Furthermore, sample dataset will be introduced. Moreover, the potential benefits and challenges of implementing federated learning within financial contexts are also assessed. Finally, promising research directions for the application of federated learning in the financial industry are outlined.

A Review of Data-Driven Intelligent Monitoring for Geological Drilling Processes

The exploration and development of resources and energy are fundamental to human survival and development, and geological drilling is a key method for deep resource and energy exploration. Intelligent monitoring technology can achieve anomaly detection, fault diagnosis, and fault prediction in the drilling process, which is crucial for ensuring production safety and improving drilling efficiency. The drilling process is characterized by complex geological conditions, variable working conditions, and low information value density, which pose a series of difficulties and challenges for intelligent monitoring. This paper reviews the research progress of the data-driven intelligent monitoring of geological drilling processes, focusing on the above difficulties and challenges. It mainly includes multivariate statistics, machine learning, and multi-model fusion. Multivariate statistical methods can effectively handle and analyze complex geological drilling data, while machine learning methods can efficiently extract key patterns and trends from a large amount of geological drilling data. Multi-model fusion methods, by combining the advantages of the first two methods, enhance the ability to handle complex multivariable and nonlinear problems. This review shows that existing research still faces problems such as limited data processing capabilities and insufficient model generalization capabilities. Improving the efficiency of data processing and the generalization capability of models may be the main research directions in the future.

PET/CT radiomics and deep learning in the diagnosis of benign and malignant pulmonary nodules: progress and challenges

Lung cancer is currently the leading cause of cancer-related deaths, and early diagnosis and screening can significantly reduce its mortality rate. Since some early-stage lung cancers lack obvious clinical symptoms and only present as pulmonary nodules (PNs) in imaging examinations, accurately determining the benign or malignant nature of PNs is crucial for improving patient survival rates. 18F-FDG PET/CT is important in diagnosing PNs, but its specificity needs improvement. Radiomics can provide information beyond traditional visual assessment, overcoming its limitations by extracting high-throughput quantitative features from medical images. Radiomics features based on 18F-FDG PET/CT and deep learning methods have shown great potential in the noninvasive diagnosis of PNs. This paper reviews the latest advancements in these methods and discusses their contributions to improving diagnostic accuracy and the challenges they face.

A study of advances in machine learning-based electronic design automation

Abstract. The electronic design automation (EDA) is a convenient tool for designing integrated circuits (IC), which is employed extensively both in academic and engineering. The design of integrated circuits is conducted in accordance with a defined design flow, commonly referred to as the chip design flow, which can be divided into two distinct parts, the front-end design and the back-end design. Following a long period of evolution, the chip design flow of EDA has been gradually improved. Besides, it achieved some accomplishments during this period. However, with the growing demand of ICs, especially Very Large-Scale Integration Circuit (VLSI), the existing EDA is not adequate for the requirement. In addition, the EDA technology has been so developed that it is relatively less flexible. In this context, concerns about the future of EDA have recently emerged. In response to this challenge, research has mentioned that machine learning methods (ML methods) can improve the functionality of EDA. The ML method covers most of steps of EDAs design flow, especially back-end design. The machine learning-based electronic design automation is still in its infancy, which is presented with a multitude of challenges. Therefore, the paper explores the development of EDA by reviewing and organizing the related literature, and summarizes the application of ML methods in EDA, thereby providing the future development trend of ML-based EDA.

GPS-pPLM: A Language Model for Prediction of Prokaryotic Phosphorylation Sites

In the prokaryotic kingdom, protein phosphorylation serves as one of the most important posttranslational modifications (PTMs) and is involved in orchestrating a broad spectrum of biological processes. Here, we report an updated online server named the group-based prediction system for prokaryotic phosphorylation language model (GPS-pPLM), used for predicting phosphorylation sites (p-sites) in prokaryotes. For model training, two deep learning methods, a transformer and a deep neural network, were employed, and a total of 10 sequence features and contextual features were integrated. Using 44,839 nonredundant p-sites in 16,041 proteins from 95 prokaryotes, two general models for the prediction of O-phosphorylation and N-phosphorylation were first pretrained and then fine-tuned to construct 6 predictors specific for each phosphorylatable residue type as well as 134 species-specific predictors. Compared with other existing tools, the GPS-pPLM exhibits higher accuracy in predicting prokaryotic O-phosphorylation p-sites. Protein sequences in FASTA format or UniProt accession numbers can be submitted by users, and the predicted results are displayed in tabular form. In addition, we annotate the predicted p-sites with knowledge from 22 public resources, including experimental evidence, 3D structures, and disorder tendencies. The online service of the GPS-pPLM is freely accessible for academic research.

FORECASTING FIRE RISK VALUES USING MACHINE LEARNING AND TIME SERIES ANALYSIS

The probability of occurrence of emergencies and their consequences can be minimized by predicting events that contribute to the emergence of such a situation. An optimal and useful tool for such prediction and forecasting can be modeling, which can be carried out on the basis of information technology using computer programs.The methods of modeling and forecasting emergencies related to fires are the same as for other emergencies, but there are some specifics. The forecast of emergencies related to fires is aimed at three main stages: 1. forecasting the probability of occurrence with the determination of the place of the highest probability of fire occurrence, time and factors contributing to it; 2. forecasting the development of the situation and the spread of the fire (direction, speed) when it occurs, the process itself; 3. forecasting the consequences of a fire at a specific facility.The forecast is made based on the analysis of all possible unfavorable factors, their fields of action, features of the facility, its fire hazard and other data.The use of machine learning methods and time series analysis for forecasting fire risk values is substantiated. The possibility of using statistical methods of mathematical modeling of fire occurrence as the basis of the machine learning model is noted. This approach enables the machine learning model to process and analyze a large amount of statistical information for specific time periods and issue forecasts.