Model Training Phase Research Articles

OCJ-HCBM-DenseCRF: a deep learning optimization for urban built-up area extraction from large-size remote sensing imagery

ABSTRACT Accurate extraction of large-area urban built-up areas is of significant importance for urban development and map updating. Traditional methods relying on remote sensing imagery and deep learning exhibit low data utilization during the model training phase and significant discrepancies between extraction accuracy in application scenarios and during training, posing challenges in practical applications. In response to these issues, this paper proposes an urban built-up area extraction method based on deep learning semantic segmentation technology and large-size remote sensing imagery. This method, grounded in the sliding window theory, introduces the Overlapping Cutting and Joining (OCJ) approach. Within the OCJ method, this paper develops the Overlapping Cutting (OC) module to increase the quantity and richness of training samples. In application scenarios, the OC and Overlapping Joining (OJ) modules are utilized to circumvent large-size cross-sampling, thereby preserving original image information and enhancing segmentation accuracy. Additionally, to eliminate stitching seams and optimize detailed segmentation effects, this paper proposes the HCBM-DenseCRF method, which employs DenseCRF to eliminate stitching seams while utilizing the Hard Constraint Buffer Mechanism (HCBM) to protect complex built-up area foreground extraction results from destruction, achieving high-precision urban built-up area extraction. This paper takes the Xception-PSPNet deep learning semantic segmentation model as the base model instance and compares it with four widely used models and three ablation models, including the initial Xception-PSPNet model. OCJ-HCBM-DenseCRF achieved superior results compared to other methods across smaller, medium, and larger remote sensing image instances, particularly leading other models by 1.1%-47.3% on larger-size imagery, with a precision decrement of only 1%-2% across the three sizes. This indicates the high application value of the proposed method in the extraction of large-size imagery.

Enhancing co-pyrolysis process of biomass and coal using machine learning insights and Shapley additive explanations based on cooperative game theory

The co-pyrolysis process is an essential method for energy extraction from waste biomass and coal although the co-pyrolysis technology of biomass and coal presents a complex engineering challenge. To address these challenges, modern data-driven ensemble and tree-based machine learning approaches offer a promising solution. This study provides a comprehensive analysis of various machine learning techniques, including linear regression (LR), decision tree (DT), random forest (RF), extreme gradient boosting (XGBoost), and adaptive boosting (AdaBoost) to predict the outcome models of pyrolysis oil yield, syngas yield, char yield, and syngas lower heating value from co-pyrolysis of biomass and coal. The models are evaluated using different statistical metrics. The DT-based pyrolysis oil yield model outperformed the other four models (LR, RF, XGBoost, and AdaBoost) in predicting pyrolysis oil with robust accuracy, achieving an R2 of 0.999 and a mean squared error (MSE) close to zero during the model training phase. Similarly, the DT-based syngas yield model showed a high R2 of 0.999 and near-zero MSE while the based char yield model excelled the others with a high R2 of 0.999 and negligible MSE during the model training phase. In the subsequent phase, explainable artificial intelligence-based Shapley additive explanation (SHAP) values were estimated for feature importance analysis. The SHAP analysis identified key features for pyrolysis oil and syngas yield, with biomass blending ratio and reaction time being the most crucial, while reaction time and temperature were the most important for the syngas LHV model.

ACEA-Net: Weakly Supervised Prostate 3D MRI Image Segmentation via Advanced Prompt Points.

In prostate 3D MRI image segmentation methods, it is usually necessary to annotate each slice, and these annotations are generally time-consuming and specialized. In this study, we generate pseudo-labels using an annotation method with one foreground seed point and six edge relaxation points. We design a weakly supervised semantic learning segmentation framework, ACEA-Net. This segmentation framework solves the under-expansion problem due to the lack of semantic affinity of the seed point pixels in the pseudo-labeling generation process. We design a Seed Cluster Geodesic Distance Transform (SeedGeo) seed expansion strategy to provide a more complete supervised signal. In the segmentation model training phase, Adaptive Convolutional Normalization (ACN) and Enhanced Simple Parameter-Free Attention Module (SimAM) are utilized to smooth the convolutional layer's output in the U-Net baseline model to suppress noisy labels. The proposed segmentation framework achieves excellent segmentation results on the MSD prostate and PROMISE12 prostate datasets, with Dice similarity coefficients (Dice) of 87.23% and 81.00% for the two segmentation tasks, and Average Symmetry Surface Distances (ASSD) of 1.73mm and 2.02mm, respectively, which are superior to the current state-of-the-art method.

Development of an oral cancer detection system through deep learning

ObjectiveWe aimed to develop an AI-based model that uses a portable electronic oral endoscope to capture intraoral images of patients for the detection of oral cancer.Subjects and methodsFrom September 2019 to October 2023, 205 high-quality annotated images of oral cancer were collected using a portable oral electronic endoscope at the Chinese PLA General Hospital for this study. The U-Net and ResNet-34 deep learning models were employed for oral cancer detection. The performance of these models was evaluated using several metrics: Dice coefficient, Intersection over Union (IoU), Loss, Precision, Recall, and F1 Score.ResultsDuring the algorithm model training phase, the Dice values were approximately 0.8, the Loss values were close to 0, and the IoU values were around 0.7. In the validation phase, the highest Dice values ranged between 0.4 and 0.5, while the Loss values increased, and the training loss began to decrease gradually. In the test phase, the model achieved a maximum Precision of 0.96 with a confidence threshold of 0.990. Additionally, with a confidence threshold of 0.010, the highest F1 score reached was 0.58.ConclusionThis study provides an initial demonstration of the potential of deep learning models in identifying oral cancer.

Deep-GB: A novel deep learning model for globular protein prediction using CNN-BiLSTM architecture and enhanced PSSM with trisection strategy.

Globular proteins (GPs) play vital roles in a wide range of biological processes, encompassing enzymatic catalysis and immune responses. Enzymes, among these globular proteins, facilitate biochemical reactions, while others, such as haemoglobin, contribute to essential physiological functions such as oxygen transport. Given the importance of these considerations, accurately identifying Globular proteins is essential. To address the need for precise GP identification, this research introduces an innovative approach that employs a hybrid-based deep learning model called Deep-GP. We generated two datasets based on primary sequences and developed a novel feature descriptor called, Consensus Sequence-based Trisection-Position Specific Scoring Matrix (CST-PSSM). The model training phase involved the application of deep learning techniques, including the bidirectional long short-term memory network (BiLSTM), gated recurrent unit (GRU), and convolutional neural network (CNN). The BiLSTM and CNN were hybridised for ensemble learning. The CST-PSSM-based ensemble model achieved the most accurate predictive outcomes, outperforming other competitive predictors across both training and testing datasets. This demonstrates the potential of harnessing deep learning for precise GB prediction as a robust tool to expedite research, streamline drug discovery, and unveil novel therapeutic targets.

Optimizing flotation froth image segmentation via parallel branch network and hybrid loss supervision

Optimizing flotation froth image segmentation via parallel branch network and hybrid loss supervision

Better together: Automated app review analysis with deep multi-task learning

Better together: Automated app review analysis with deep multi-task learning

Learning stability on graphs

Abstract In artificial intelligence applications, the model training phase is critical and computationally demanding. In the graph neural networks (GNNs) research field, it is interesting to investigate how varying the graph topological and spectral structure impacts the learning process and overall GNN performance. In this work, we aim to theoretically investigate how the topology and the spectrum of a graph changes when nodes and edges are added or removed. Numerical results highlight stability issues in the learning process on graphs. In this work, we aim to theoretically investigate how the topology and the spectrum of a graph changes when nodes and edges are added or removed. We propose the topological relevance function as a novel method to quantify the stability of graph-based neural networks when graph structures are perturbed. We also explore the relationship between this topological relevance function, Graph Edit Distance, and spectral similarity. Numerical results highlight stability issues in the learning process on graphs.

Research on dangerous flight weather prediction based on machine learning

Abstract With the continuous expansion of the scale of air transport, the demand for aviation meteorological support also continues to grow. The impact of hazardous weather on flight safety is critical. How to effectively use meteorological data to improve the early warning capability of flight dangerous weather and ensure the safe flight of aircraft is the primary task of aviation meteorological services. In this work, support vector machine (SVM) models are used to predict hazardous flight weather, especially for meteorological conditions with high uncertainty such as storms and turbulence. SVM is a supervised learning method that distinguishes between different classes of data by finding optimal decision boundaries in a high-dimensional space. To meet the needs of this study, we chose the radial basis function (RBF) as the kernel function, which helps to deal with nonlinear problems and enables the model to better capture complex meteorological data structures. During the model training phase, we used historical meteorological observations from multiple weather stations, including temperature, humidity, wind speed, wind direction, and other meteorological indicators closely related to flight safety. From this data, the SVM model learns how to distinguish between normal and dangerous flight weather conditions.

End-to-end AI pipeline optimization: Benchmarking and performance enhancement techniques for recommendation systems

End-to-end AI pipeline optimization is critical for improving the efficiency and performance of recommendation systems, which play a pivotal role in personalizing user experiences across various domains. This review explores benchmarking and performance enhancement techniques tailored to recommendation systems within AI pipelines. The objective is to streamline the processes involved in data ingestion, feature engineering, model training, and deployment to achieve optimal system performance and user satisfaction. Recommendation systems typically involve complex workflows that require continuous optimization. Benchmarking serves as a foundational step, enabling the identification of bottlenecks and inefficiencies within the pipeline. By establishing clear performance metrics, such as precision, recall, and latency, benchmarking allows for the comparative analysis of different algorithms, data processing methods, and system configurations. These metrics guide the selection of the most suitable models and techniques, thereby enhancing overall system effectiveness. Performance enhancement techniques are then applied to various stages of the AI pipeline. Advanced methods in feature engineering, such as automated feature selection and dimensionality reduction, can significantly improve model accuracy while reducing computational overhead. In the model training phase, techniques like hyperparameter tuning, gradient-based optimization, and distributed training are employed to accelerate convergence and improve model generalization. Additionally, optimization strategies at the deployment stage, including model compression, quantization, and the use of specialized hardware, are crucial for minimizing latency and resource consumption. This review also highlights the importance of continuous monitoring and feedback loops to maintain the effectiveness of recommendation systems in dynamic environments. By integrating real-time analytics and adaptive algorithms, systems can adjust to changing user behaviors and preferences, ensuring sustained performance improvements. In conclusion, optimizing end-to-end AI pipelines for recommendation systems involves a multifaceted approach that includes benchmarking, feature engineering, model training, and deployment enhancements. These efforts collectively contribute to more efficient, scalable, and accurate recommendation systems, ultimately leading to better user experiences and operational efficiencies. This paper will focus on optimizing AI pipelines for recommendation systems, covering the entire process from data extraction and feature engineering to model deployment and performance benchmarking. It will discuss techniques for identifying and mitigating performance bottlenecks in different computing environments, providing valuable insights for enhancing the efficiency of recommendation systems, which are crucial for various applications.

A deep learning framework for predicting slab transverse crack using multivariate LSTM-FCN in continuous casting

A deep learning framework for predicting slab transverse crack using multivariate LSTM-FCN in continuous casting

Optimizing ash content detection and prediction in flotation tailings using a new approach to enhance feature extraction and deep learning algorithms

ABSTRACT Flotation is a crucial separation technique in coal beneficiation. However, the inability to quickly and accurately detect ash content during the flotation process hinders the development of flotation automation. This paper introduced a novel hybrid model based on convolutional neural networks, establishing the ash content in tailings. In this method, convolutional neural network is utilized to automatically extract features from tailings images. In the model training and optimization phase after inputting the augmented data, the optimal ResNet18 model is identified by comparing five model optimization methods and replacing the SoftMax classifier with the traditional SVR in ResNet18. Additionally, a PCA-SVR model for predicting tailings ash content is constructed for comparative analysis, employing six different feature combinations to validate the effectiveness of the proposed features, with optimal support vector parameters identified through random search and cross-validation algorithms. The prediction results show that the PCA-SVR model has the highest accuracy when the feature dimension is reduced to 16, and the prediction of ResNet18-SVR is more accurate compared to PCA-SVR. This indicates that the proposed method can predict the ash content more accurately than the traditional feature engineering methods and has a broad application prospect in the field of coal beneficiation.

Classification of Toraja Wood Carving Motif Images Using Convolutional Neural Network (CNN)

Wood carving is a cultural heritage with deep meaning and significance for the Toraja ethnic group's culture. By understanding the meaning of each Toraja carving, both tourists and the local community can gain knowledge about Toraja culture, thereby preserving and maintaining the culture amidst modern developments. Image processing approaches, particularly the development of Convolutional Neural Networks (CNN), offer a solution for extracting information from the diverse and intricate patterns of Toraja wood carvings. This study is highly significant as it implements a deep learning model using the CNN algorithm optimized with the ResNet50 architecture. The methodology in this study involves adjusting the batch size during the model training phase and applying weak-to-strong pixel transformation during the double threshold hysteresis phase in the Canny Feature Extraction process on the edges of Toraja carving images, resulting in ResNet50 architecture accurately recognizing the patterns of Toraja wood carvings. The results demonstrate significant improvements in the performance of the ResNet50 architecture with the preprocessed dataset. average precision, recall, precision, and F1-Score improvements in each Toraja carving class. For the Pa' Lulun Pao class, it was found that the precision and recall values were 0.94, and the F1-Score was 0.94. The Pa’ Somba class also showed good results, with a precision value of 0.9697, a recall of 0.96, and an F1-Score of 0.9648. The Pa’ Tangke Lumu class showed even better results, with a precision value of 0.9898, a recall of 0.97, and an F1-Score of 0.9798. The Pa’ Tumuru class also demonstrated good performance, with a precision value of 0.9327, a recall of 0.97, and an F1-Score of 0.9500. This study not only underscores the effectiveness of processing in enhancing CNN capabilities but also opens opportunities for further research in applying these methods to various image types and exploring different CNN architectures.

Combating Phishing in Kenya: A Supervised Learning Model for Enhanced Email Security in Kenyan Financial Institutions

Purpose: Phishing is a serious cybercrime problem that puts people and organizations at risk all around the world, especially in Republic of Kenya's financial institutions. Modern solutions are required because traditional security measures are being challenged by the growing sophistication of phishing attempts. The goal of this thesis is to use artificial intelligence (AI) to create a supervised machine learning model that will alleviate phishing email attacks in the Kenyan financial institutions. Attackers frequently use phishing emails as a means of obtaining unauthorized access to private information, including login credentials, financial information, and personal information. This can lead to identity theft, reputational harm, and monetary losses. Methodology: The suggested framework focuses on using supervised machine learning techniques to recognize and stop phishing emails with accuracy. Four primary parts make up the framework: response, email filtering, feature extraction, and categorization. Four primary steps are involved in developing the framework: gathering data, preparing the data, training the model, and deployment. A thorough dataset of phishing and non-phishing emails is compiled throughout the data collecting phase from a variety of sources, including as public databases, forums on the dark web, and emails from financial institutions staff. Cleaning, classifying, and preprocessing the gathered data are all part of the data preprocessing step, which makes sure the data is appropriate for training the model. Supervised machine learning methods are used in the model training phase to create a reliable detection model. Findings: The system is tested against a dataset of phishing and non-phishing emails particular to the Kenyan financial sectors. The framework's effectiveness is assessed using performance indicators such as accuracy, precision, recall, and the F1-score. The results reveal that the system can correctly detect new phishing emails that were not previously included in the training dataset, demonstrating its adaptability to emerging threats. Unique Contribution to Theory, Practice and Policy: By offering an effective mechanism for detecting and alleviating phishing email attacks, the proposed framework considerably minimizes the risk of data breaches and financial losses in the banking sector.

Sampling approaches to reduce very frequent seasonal time series

AbstractWith technological advancements, much data is being captured by sensors, smartphones, wearable devices, and so forth. These vast datasets are stored in data centres and utilized to forge data‐driven models for the condition monitoring of infrastructures and systems through future data mining tasks. However, these datasets often surpass the processing capabilities of traditional information systems and methodologies due to their significant size. Additionally, not all samples within these datasets contribute valuable information during the model training phase, leading to inefficiencies. The processing and training of Machine Learning algorithms become time‐consuming, and storing all the data demands excessive space, contributing to the Big Data challenge. In this paper, we propose two novel techniques to reduce large time‐series datasets into more compact versions without undermining the predictive performance of the resulting models. These methods also aim to decrease the time required for training the models and the storage space needed for the condensed datasets. We evaluated our techniques on five public datasets, employing three Machine Learning algorithms: Holt‐Winters, SARIMA, and LSTM. The outcomes indicate that for most of the datasets examined, our techniques maintain, and in several instances enhance, the forecasting accuracy of the models. Moreover, we significantly reduced the time required to train the Machine Learning algorithms employed.

ESE-YOLOv8: A Novel Object Detection Algorithm for Safety Belt Detection during Working at Heights.

To address the challenges associated with supervising workers who wear safety belts while working at heights, this study proposes a solution involving the utilization of an object detection model to replace manual supervision. A novel object detection model, named ESE-YOLOv8, is introduced. The integration of the Efficient Multi-Scale Attention (EMA) mechanism within this model enhances information entropy through cross-channel interaction and encodes spatial information into the channels, thereby enabling the model to obtain rich and significant information during feature extraction. By employing GSConv to reconstruct the neck into a slim-neck configuration, the computational load of the neck is reduced without the loss of information entropy, allowing the attention mechanism to function more effectively, thereby improving accuracy. During the model training phase, a regression loss function named the Efficient Intersection over Union (EIoU) is employed to further refine the model's object localization capabilities. Experimental results demonstrate that the ESE-YOLOv8 model achieves an average precision of 92.7% at an IoU threshold of 50% and an average precision of 75.7% within the IoU threshold range of 50% to 95%. These results surpass the performance of the baseline model, the widely utilized YOLOv5 and demonstrate competitiveness among state-of-the-art models. Ablation experiments further confirm the effectiveness of the model's enhancements.

SoK: Wildest Dreams: Reproducible Research in Privacy-preserving Neural Network Training

Machine Learning (ML), addresses a multitude of complex issues in multiple disciplines, including social sciences, finance, and medical research. ML models require substantial computing power and are only as powerful as the data utilized. Due to the high computational cost of ML methods, data scientists frequently use Machine Learning-as-a-Service (MLaaS) to outsource computation to external servers. However, when working with private information, like financial data or health records, outsourcing the computation might result in privacy issues. Recent advances in Privacy-Preserving Techniques (PPTs) have enabled ML training and inference over protected data through the use of Privacy-Preserving Machine Learning (PPML). However, these techniques are still at a preliminary stage and their application in real-world situations is demanding. In order to comprehend the discrepancy between theoretical research suggestions and actual applications, this work examines the past and present of PPML, focusing on Homomorphic Encryption (HE) and Secure Multi-party Computation (SMPC) applied to ML. This work primarily focuses on the ML model's training phase, where maintaining user data privacy is of utmost importance. We provide a solid theoretical background that eases the understanding of current approaches and their limitations. We also provide some preliminaries of SMPC, HE, and ML. In addition, we present a systemization of knowledge of the most recent PPML frameworks for model training and provide a comprehensive comparison in terms of the unique properties and performances on standard benchmarks. Also, we reproduce the results for some of the surveyed papers and examine at what level existing works in the field provide support for open science. We believe our work serves as a valuable contribution by raising awareness about the current gap between theoretical advancements and real-world applications in PPML, specifically regarding open-source availability, reproducibility, and usability.

An AI Approach to Support Student Mental Health: Case of Developing an AI-Powered Web-Platform with Nature-Based Mindfulness

ABSTRACT This case project proposes an Artificial Intelligence (AI) approach to support the improvement of student mental health by developing an AI-powered web-platform for practicing nature-based mindfulness. This project addresses the recent trend of utilizing technological devices to assist human mindfulness and mental health. We created an AI-powered website based on the Monitor and Acceptance Theory (MAT), facilitating virtual nature-based mindfulness practice with nature videography and guided meditations. This case study showcases the four major steps of the project development: (1) data collection to build a database of nature videography, (2) data collection to build a database of guided meditations, (3) AI system and model training phase, and (4) personalized mindfulness website pilot study. The proposed AI approach to mindfulness has the potential to benefit and contribute to knowledge creation in the areas of hospitality and tourism education, human mental health, MAT, AI, human-computer interactions, environmental education, and nature tourism.

Implementación de una aplicación móvil con Machine Learning aplicando la nueva metodología CSKT para el proceso de contratación de personal en el sector privado

This paper evaluates a Machine Learning model developed using the CSKT methodology to improve pension allocation in engineering projects. It addresses the challenges in adopting such models by following a structured approach from understanding the business model to presenting it. The initial decision tree model training phase yields an F1-Score of 0.699825, indicating balanced precision and recall. Individual precision is 0.650974, and recall is 0.756603, demonstrating accurate prediction of positive outcomes. The ROC curve has a value of 0.739770, assessing the discriminatory ability of the model. The results include defining the model parameters, creating a decision tree, and performing initial simulations. The conclusion highlights CSKT's effectiveness in pension allocation and the stresses of ongoing client collaboration in engineering projects. The numerical results highlight the model's contribution to improving pension allocation processes in engineering projects.

Road Network Traffic Flow Prediction Method Based on Graph Attention Networks

With the acceleration of urbanization and the continuous growth of transportation demand, the traffic management of smart city road networks has become increasingly complex and critical. Traffic flow prediction, as an important component of smart transportation systems, is of great significance for optimizing traffic planning and improving traffic efficiency. The study collected and preprocessed traffic data in the smart city road network, including multi-dimensional information such as traffic flow, road conditions, and meteorological data. Then, based on the idea of graph neural networks, we constructed the topological structure of the urban road network and abstracted elements such as roads and intersections into nodes, using edges to represent their connection relationships, thus forming a graph dataset. Next, we introduced an attention mechanism to extract more representative node features through the weighted aggregation of node features, thereby achieving effective modeling of urban road network traffic flow. During the model training phase, we used real traffic datasets for experimental verification and integrated various information such as time, space, and road features into the model. The experimental results show that compared to traditional methods, this research prediction method has achieved better performance in traffic flow prediction tasks, with higher prediction accuracy and robustness. It has stronger applicability and effectiveness in different traffic scenarios. By integrating multi-dimensional information and introducing attention mechanisms, this method has significant advantages in improving the accuracy and robustness of traffic flow prediction, and has important practical significance and application prospects for the construction of smart transportation systems and the development of smart cities.