Capability Of Model Research Articles

Deep Variational Network Toward Blind Image Restoration.

Blind image restoration (IR) is a common yet challenging problem in computer vision. Classical model-based methods and recent deep learning (DL)-based methods represent two different methodologies for this problem, each with their own merits and drawbacks. In this paper, we propose a novel blind image restoration method, aiming to integrate both the advantages of them. Specifically, we construct a general Bayesian generative model for the blind IR, which explicitly depicts the degradation process. In this proposed model, a pixel-wise non-i.i.d. Gaussian distribution is employed to fit the image noise. It is with more flexibility than the simple i.i.d. Gaussian or Laplacian distributions as adopted in most of conventional methods, so as to handle more complicated noise types contained in the image degradation. To solve the model, we design a variational inference algorithm where all the expected posteriori distributions are parameterized as deep neural networks to increase their model capability. Notably, such an inference algorithm induces a unified framework to jointly deal with the tasks of degradation estimation and image restoration. Further, the degradation information estimated in the former task is utilized to guide the latter IR process. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.

Advancements in Data Augmentation and Transfer Learning: A Comprehensive Survey to Address Data Scarcity Challenges

Abstract: Deep Learning (DL) models have demonstrated remarkable proficiency in image classification and recognition tasks, surpassing human capabilities. The observed enhancement in performance can be attributed to the utilization of extensive datasets. Nevertheless, DL models have huge data requirements. Widening the learning capability of such models from limited samples even today remains a challenge, given the intrinsic constraints of small datasets. The trifecta of challenges, encompassing limited labeled datasets, privacy, poor generalization performance, and the costliness of annotations, further compounds the difficulty in achieving robust model performance. Overcoming the challenge of expanding the learning capabilities of Deep Learning models with limited sample sizes remains a pressing concern even today. To address this critical issue, our study conducts a meticulous examination of established methodologies, such as Data Augmentation and Transfer Learning, which offer promising solutions to data scarcity dilemmas. Data Augmentation, a powerful technique, amplifies the size of small datasets through a diverse array of strategies. These encompass geometric transformations, kernel filter manipulations, neural style transfer amalgamation, random erasing, Generative Adversarial Networks, augmentations in feature space, and adversarial and meta- learning training paradigms. : Furthermore, Transfer Learning emerges as a crucial tool, leveraging pre-trained models to facilitate knowledge transfer between models or enabling the retraining of models on analogous datasets. Through our comprehensive investigation, we provide profound insights into how the synergistic application of these two techniques can significantly enhance the performance of classification tasks, effectively magnifying scarce datasets. This augmentation in data availability not only addresses the immediate challenges posed by limited datasets but also unlocks the full potential of working with Big Data in a new era of possibilities in DL applications.

The Role of Sentiment Analysis in Machine Translation- A Review on the Example of ChatGPT

In today’s globalization, machine translation technology has become an indispensable bridge for cross-lingual communication. The integration of sentiment analysis makes machine translation not only able to convey information, but also able to accurately express and understand the emotional color of the original text, which is of great significance in the fields of cross-cultural communication, customer service, and social media interaction. This study overviews the development of machine translation technology, from early rule-based models to modern statistical models to Transformer-based neural network models, with a special focus on the integrated application of ChatGPT in sentiment analysis and machine translation. First, this study introduces the background and research significance of machine translation, and discusses the role of sentiment analysis in improving translation quality and user satisfaction. Subsequently, the historical development of machine translation technology is reviewed in detail, and the basic concepts and methodologies of sentiment analysis and its applications in different fields are elaborated. On this basis, this study analyzes in depth the technical architecture, language model and interaction capability of ChatGPT, as well as its specific application and performance in sentiment analysis. Further, this study explores the importance of sentiment analysis in machine translation, including sentiment recognition, translation strategies for sentiment expression, and the impact of sentiment analysis on improving translation quality and cultural adaptability. Meanwhile, the advantages and limitations of ChatGPT in sentiment analysis are assessed, and future research directions are proposed, including technology convergence, construction of datasets and evaluation criteria, and ethical and bias issues. Finally, this study summarizes the practical application value of sentiment analysis in machine translation and looks forward to the potential areas of future research, aiming to provide reference and inspiration for the further development of machine translation and sentiment analysis.

Flow Field Analysis and Development of a Prediction Model Based on Deep Learning

The velocity of ocean currents significantly affects the trajectory prediction of ocean drifters and the safe navigation of intelligent vessels. Currently, most ocean current predictions focus on time-based forecasts at specific fixed points. In this study, deep learning based on the flow field prediction model (CNNs–MHA–BiLSTMs) is proposed, which predicts the changes in ocean currents by learning from historical flow fields. Unlike conventional models that focus on single-point current velocity data, the CNNs–MHA–BiLSTMs model focuses on the ocean surface current information within a specific area. The CNNs–MHA–BiLSTMs model integrates multiple convolutional neural networks (CNNs) in parallel, multi-head attention (MHA), and bidirectional long short-term memory networks (BiLSTMs). The model demonstrated exceptional modelling capabilities in handling spatiotemporal features. The proposed model was validated by comparing its predictions with those predicted by the MIKE21 flow model of the ocean area within proximity to Dalian Port (which used a commercial numerical model), as well as those predicted by other deep learning algorithms. The results showed that the model offers significant advantages and efficiency in simulating and predicting ocean surface currents. Moreover, the accuracy of regional flow field prediction improved with an increase in the number of sampling points used for training. The proposed CNNs–MHA–BiLSTMs model can provide theoretical support for maritime search and rescue, the control or path planning of Unmanned Surface Vehicles (USVs), as well as protecting offshore structures in the future.

Deep learning enables the fully automated quantification of high-resolution cardiac CT images: human in loop for model development

Abstract Introduction Cardiac computed tomography (CCT) images provide vast spatial and temporal information. Segmentation of the structure of interest is necessary to provide quantitative information. However, manual segmentation of cardiac substructures is highly time-consuming. Objective This study aims to develop a robust deep-learning segmentation model for cardiac substructure segmentation in 3D CCT images. Method We initially developed a 3D U-Net architecture, incorporating extensive augmentation, for the segmentation of the Left Ventricle Myocardium (LVM), the blood cavities of the Left and Right Ventricles (LV, RV), and the Left and Right Atria (LA, RA). This model utilized CT images from 20 patients who had available manual segmentation. Subsequently, we applied this initial model to datasets from two centers, including different generations of scanners. We then visually inspected the segmentation, selected the highest-quality segmentations, and modify it if necessary, and excluded the low-quality data. We continued fine-tuning the model with the new dataset until we reached a count of 456 patients. As there were multiple images from one patient during this model's development, we generated datasets to develop a new model using the entire new dataset to avoid any data leakage in the reported metrics. The model was developed using data from Center 1 (5-fold cross-validation) and then evaluated on an external test set from Center 2 (126 patients) and also 4D images from Center 1. Various metrics were calculated to assess the model's performance. Results In the internal test set, utilizing 5-fold CV, the model demonstrated high segmentation accuracy, achieving an overall Dice coefficient of 0.98, with individual scores of 0.96 for LVM, 0.98 for LV, 0.98 for RV, 0.99 for RA, and 0.98 for LA. We assessed the automatic quantification capability of the segmentation model from 256 patients, resulting in an R2 value greater than 0.92 for both LV end-diastolic and systolic volumes. In contrast, the model maintained robust performance on the external dataset, achieving an overall Dice coefficient of 0.96, with individual scores of 0.95 for LVM, 0.98 for LV, 0.97 for RV, 0.98 for RA, and 0.94 for LA. Conclusion We provided a large cohort of ground truth segmentations for different cardiac CT substructures using a human-in-the-loop strategy. Subsequently, we developed a deep learning model for highly accurate automated segmentation of these substructures. This model allows for fast and fully automatic segmentation and quantification of 3D and 4D cardiac CT images, from which various quantitative metrics can be extracted from different cardiac substructures.

Machine learning for air quality index (AQI) forecasting: shallow learning or deep learning?

In this study, several machine learning (ML) models consisting of shallow learning (SL) models (e.g., random forest (RF), K-nearest neighbor (KNN), weighted K-nearest neighbor (WKNN), support vector machine (SVM), artificial neural network (ANN), and deep learning (DL) models (e.g., long short-term memory (LSTM), gated recurrent unit (GRU), recurrent neural network (RNN), and convolutional neural network (CNN)) have been employed for predicting air pollution and its classification. The models were selected based on factors such as prediction accuracy, model generalization, model complexity, and training time. Our study focuses on analyzing and predicting the air quality index (AQI) using daily PM10 concentration as natural pollutants and nine meteorological parameters from March 2013 to February 2022 in Zabol. We also utilized the information gain (IG) method for feature selection. Several measures including accuracy, F1 score, precision, recall, and the area under the curve (AUC), are computed to assess model performance. This study demonstrates the efficacy of DL models, particularly CNN, in predicting the AQI with remarkable accuracy. Our findings reveal that all models effectively classify air quality levels, with an AUC of 0.95 for the good class in both DL and ANN models, significantly outperforming SL models. The AUC values for the hazardous and moderate classes of DL models were also impressive, at 0.90 and 0.83, respectively, underscoring their effectiveness in critical classifications. In terms of performance, CNN achieved an accuracy of 0.60, leading the models, while RF followed closely at 0.58. RNN, GRU, ANN, and SVM each reached an accuracy of 0.57, demonstrating a competitive edge. LSTM and WKNN recorded an accuracy of 0.55, and KNN was slightly lower at 0.53. These results highlight the superior capabilities of DL models in addressing complex air quality classifications, providing invaluable insights for policymakers. By leveraging these advanced techniques, stakeholders can implement more effective strategies to combat air pollution and safeguard public health. It is worth noting that irregular monitoring of air quality data may affect the robustness of our predictions, highlighting the need for more consistent data collection to ensure an accurate representation of pollution levels.

Soil and Water Assessment Tool (SWAT)-Informed Deep Learning for Streamflow Forecasting with Remote Sensing and In Situ Precipitation and Discharge Observations

In order to anticipate residual errors and improve accuracy while reducing uncertainties, this work integrates the long short-term memory (LSTM) with the Soil and Water Assessment Tool (SWAT) to create a deep learning (DL) model that is guided by physics. By forecasting the residual errors of the SWAT model, the SWAT-informed LSTM model (LSTM-SWAT) differs from typical LSTM approaches that predict the streamflow directly. Through numerical tests, the performance of the LSTM-SWAT was evaluated with both LSTM-only and SWAT-only models in the Upper Heihe River Basin. The outcomes showed that the LSTM-SWAT performed better than the other models, showing higher accuracy and a lower mean absolute error (MAE = 3.13 m3/s). Sensitivity experiments further showed how the quality of the training dataset affects the performance of the LSTM-SWAT. The results of this study demonstrate how the LSTM-SWAT may improve streamflow prediction greatly by remote sensing and in situ observations. Additionally, this study emphasizes the need for detailed consideration of specific sources of uncertainty to further improve the predictive capabilities of the hybrid model.

A deep learning model for syncope recognition in the emergency department

Abstract Introduction Syncope is a common presenting symptom in the emergency departments (EDs) and poses challenges in its recognition and differentiation from alternative causes of transient loss of consciousness. Natural Language Processing (NLP) is an artificial intelligence technique that enables the analysis of natural language, such as clinical notes and medical history within electronic medical records (EMRs). The application of NLP into the medical field is surging but its use for the recognition of syncope is still underexplored. Purpose We aimed to develop a NLP-based deep learning model for syncope recognition in the ED. Methods Two models, based on the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed and evaluated using consecutive EMRs from Humanitas Research Hospital ED. The first "triage" model was only based on textual data contained in the "triage" section of the EMR. The second "anamnesis" model also considered textual data contained in the "medical history" section. Performance metrics including area under the curve (AUC), Matthew’s correlation coefficient (MCC) and Brier’s score (BS) were used to compare the discriminative capabilities of different BERT models. Results The models were developed and tested on 15,098 and 15,222 EMRs, respectively. The Multilanguage BERT discriminated patients with syncope with an AUC of 0.85 in the triage model and of 0.92 in the anamnesis model. The Italian BERT identified patients with syncope with a MCC of 0.50 and a BS of 0.02 in the triage model and of 0.64 and 0.01, respectively, in the anamnesis model. Conclusions We propose two NLP-based models that identify syncope patients with a high discriminative capability. The presented algorithm may provide a starting point for the management of syncope patients using machine learning techniques.ROC and calibration curves of modelsFramework for model implementation

Upregulation of the sodium potassium pump current supresses ectopic activity after stem cell delivery in the human-based heart after myocardial infarction

Abstract Background Cell therapy is being explored to restore the damaged and intrinsically irreparable human heart. Whilst preclinical studies showed improvements in cardiac function after injury, they also highlighted a critical pro-arrhythmic window when delivered cells produce ectopy-induced ventricular tachycardia. Recent experiments investigating the cells in vivo maturation suggested upregulation of the inward rectifier potassium current (IK1) and knock-out of the funny current (If) and sodium calcium exchanger (INaCa) to supress the arrhythmias[1]. Purpose Whilst promising, such modifications may impact the cells’ calcium dynamics and ultimately, their efficacy. The goal of this study is to identify novel ionic targets that improve therapy safety whilst maintaining efficacy by employing the unique predictive capabilities of modelling and simulation. Methods For this, we used our established modelling and simulation framework of cell therapy in the chronically infarcted human heart with Purkinje, validated against experimental and clinical data from the action potential to the ECG. We incorporated experimentally observed gene expression of in vivo cell maturation[1] into our framework to reproduce human pluripotent stem cell-induced cardiomyocyte (hPSC-CM) electrophysiology at day 0 and day 14 after injection. To consider variability amongst experimentally reported hPSC-CMs, we also created a fast-beating hPSC-CM model by upregulating the SERCA pump current, If and INaCa. Results Our results showed that while day 0 hPSC-CMs did not cause spontaneous activity, day 14 hPSC-CMs produced ectopic beats in the ventricles every 1.5s. The rapid beating hPSC-CM phenotype at day 14 elicited tachycardia through spontaneous beating and intermittent re-entrant wavefronts (see Figure 1). Next, we demonstrated 65% upregulation of the sodium-potassium pump current (INaK) together with a 50% increase in IK1 and knock-out of If as a novel pathway to quiescence. Importantly, our simulations highlighted that compared to INaCa knock-out, an increase in INaK allowed for physiological calcium transients. Conclusion To conclude, in this work we have utilised modelling and simulation of the injured human heart to 1) capture pre-clinically observed arrhythmias after cell therapy and 2) identify INaK upregulation as a novel target to mitigate safety concerns, whilst preserving efficacy.

Optimizing the Utilization of Generative Artificial Intelligence (AI) in the AEC Industry: ChatGPT Prompt Engineering and Design

Generative Artificial Intelligence (AI) holds significant potential for revolutionizing the Architecture, Engineering, and Construction (AEC) industry by automating complex tasks such as construction scheduling, hazard recognition, resource leveling, information retrieval from BIM, etc. However, realizing this potential requires a strategic approach to ensure effective utilization and maximum benefit. This paper presents guidelines for prompt design and engineering to elicit desired responses from ChatGPT, a Generative AI tool, in AEC applications. Key steps include understanding user intent, leveraging model capabilities, and optimizing prompt structures. By following these guidelines, stakeholders in the AEC industry can harness the power of Generative AI to improve construction scheduling processes, increase project efficiency, and ultimately drive innovation and growth in the industry. Several illustrative examples on construction scheduling and hazard recognition are provided to demonstrate the methodology proposed in this research. It is concluded that Generative AI, when effectively utilized, significantly enhances project scheduling and hazard recognition capability in the AEC industry with minimal error.

Machine learning based prediction models for cardiovascular disease risk using electronic health records data: systematic review and meta-analysis

Abstract Background Cardiovascular disease (CVD) remains a major cause of mortality in the UK, prompting the need for improved risk predictive models for primary prevention. Machine learning (ML) models utilizing electronic health records (EHR) offer potential enhancements over traditional risk scores like QRISK3 and ASCVD. Objective To systematically evaluate and compare the efficacy of ML models against conventional CVD risk prediction algorithms using EHR data for medium to long-term (5-10 years) CVD risk prediction. Methods A systematic review and random-effect meta-analysis were conducted according to PRISMA guidelines, assessing studies from 2010 to 2024. We retrieved 32 ML models and 26 conventional statistical models from 20 selected studies, focusing on performance metrics such as Area Under the Curve (AUC) and heterogeneity across models. Results ML models, particularly random forest and deep learning, demonstrated superior performance, with the highest recorded pooled AUCs of 0.865 (95% CI: 0.812-0.917) and 0.847 (95% CI: 0.766-0.927), respectively. These significantly outperformed the conventional risk score of 0.765 (95% CI: 0.734-0.796). However, significant heterogeneity (I² > 99%) and potential publication bias were noted across the studies. Conclusion While ML models show enhanced calibration for CVD risk, substantial variability and methodological concerns limit their current clinical applicability. Future research should address these issues by enhancing methodological transparency and standardization to improve the reliability and utility of these models in clinical settings. Implications This study highlights the advanced capabilities of ML models in CVD risk prediction and emphasizes the need for rigorous validation to facilitate their integration into clinical practice. This review was registered with PROSPERO (CRD42021250058).

CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher

Penetration testing, a critical component of cybersecurity, typically requires extensive time and effort to find vulnerabilities. Beginners in this field often benefit from collaborative approaches with the community or experts. To address this, we develop Cybersecurity Intelligent Penetration-testing Helper for Ethical Researchers (CIPHER), a large language model specifically trained to assist in penetration testing tasks as a chatbot. Unlike software development, penetration testing involves domain-specific knowledge that is not widely documented or easily accessible, necessitating a specialized training approach for AI language models. CIPHER was trained using over 300 high-quality write-ups of vulnerable machines, hacking techniques, and documentation of open-source penetration testing tools augmented in an expert response structure. Additionally, we introduced the Findings, Action, Reasoning, and Results (FARR) Flow augmentation, a novel method to augment penetration testing write-ups to establish a fully automated pentesting simulation benchmark tailored for large language models. This approach fills a significant gap in traditional cybersecurity Q&A benchmarks and provides a realistic and rigorous standard for evaluating LLM’s technical knowledge, reasoning capabilities, and practical utility in dynamic penetration testing scenarios. In our assessments, CIPHER achieved the best overall performance in providing accurate suggestion responses compared to other open-source penetration testing models of similar size and even larger state-of-the-art models like Llama 3 70B and Qwen1.5 72B Chat, particularly on insane difficulty machine setups. This demonstrates that the current capabilities of general large language models (LLMs) are insufficient for effectively guiding users through the penetration testing process. We also discuss the potential for improvement through scaling and the development of better benchmarks using FARR Flow augmentation results.

Evaluating Machine Learning Models for Stroke Prognosis and Prediction in Atrial Fibrillation Patients: A Comprehensive Meta-Analysis

Background/Objective: Atrial fibrillation (AF) complicates the management of acute ischemic stroke (AIS), necessitating precise predictive models to enhance clinical outcomes. This meta-analysis evaluates the efficacy of machine learning (ML) models in three key areas: stroke prognosis in AF patients, stroke prediction in AF patients, and AF prediction in stroke patients. The study aims to assess the accuracy and variability of ML models in forecasting AIS outcomes and detecting AF in stroke patients, while exploring the clinical benefits and limitations of integrating these models into practice. Methods: We conducted a systematic search of PubMed, Embase, and Cochrane databases up to June 2024, selecting studies that evaluated ML accuracy in stroke prognosis and prediction in AF patients and AF prediction in stroke patients. Data extraction and quality assessment were performed independently by two reviewers, with random-effects modeling applied to estimate pooled accuracy metrics. Results: The meta-analysis included twenty-four studies comprising 7,391,645 patients, categorized into groups for stroke prognosis in AF patients (eight studies), stroke prediction in AF patients (thirteen studies), and AF prediction in stroke patients (three studies). The pooled AUROC was 0.79 for stroke prognosis and 0.68 for stroke prediction in AF, with higher accuracy noted in short-term predictions. The mean AUROC across studies was 0.75, with models such as Extreme Gradient Boosting (XGB) and Random Forest (RF) showing superior performance. For stroke prognosis in AF, the mean AUROC was 0.78, whereas stroke prediction yielded a mean AUROC of 0.73. AF prediction post-stroke had an average AUROC of 0.75. These findings indicate moderate predictive capability of ML models, underscoring the need for further refinement and standardization. The absence of comprehensive sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) metrics limited the ability to conduct full meta-analytic modeling. Conclusions: While ML models demonstrate potential for enhancing stroke prognosis and AF prediction, they have yet to meet the clinical standards required for widespread adoption. Future efforts should focus on refining these models and validating them across diverse populations to improve their clinical utility.

GLE-net: global-local information enhancement for semantic segmentation of remote sensing images

Remote sensing (RS) images contain a wealth of information with expansive potential for applications in image segmentation. However, Convolutional Neural Networks (CNN) face challenges in fully harnessing the global contextual information. Leveraging the formidable capabilities of global information modeling with Swin-Transformer, a novel RS images segmentation model with CNN (GLE-Net) was introduced. This integration gives rise to a revamped encoder structure. The subbranch initiates the process by extracting features at varying scales within the RS images using the Multiscale Feature Fusion Module (MFM), acquiring rich semantic information, discerning localized finer features, and adeptly handling occlusions. Subsequently, Feature Compression Module (FCM) is introduced in main branch to downsize the feature map, effectively reducing information loss while preserving finer details, enhancing segmentation accuracy for smaller targets. Finally, we integrate local features and global features through Spatial Information Enhancement Module (SIEM) for comprehensive feature modeling, augmenting the segmentation capabilities of model. We performed experiments on public datasets provided by ISPRS, yielding notably remarkable experimental outcomes. This underscores the substantial potential of our model in the realm of RS image segmentation within the context of scientific research.

Evaluating and Predicting Meteorological Drought Using Different Climate Reanalysis Datasets over New South Wales, Australia

AbstractDroughts are one of the most disastrous natural hazards, primarily due to their persistence and spatial distribution. Drought prediction is one of the key challenges for effective drought management and to do so, studies often involve the use of station-based data which are effective only in regions with high-gauge density. Therefore, there is growing interest in the use of interpolated climatic grids to predict droughts. In recent decades, drought conditions have been aggravated by climate change and for that reason the use of climatic variables is important to accurately predict droughts. The analysis of any aspect of drought can be affected by the choice of data and drought index. Therefore, this study aims to identify the most suitable dataset and drought index for the New South Wales (NSW) region of Australia. The present study evaluates various precipitation datasets (Climate Research Unit (CRU), ERA-5, and Scientific Information for Land Owners (SILO)) and their corresponding variations on the Standardised Precipitation Index (SPI) at different time scales. Based on the findings, CRU was used to predict meteorological drought using machine learning techniques. The different machine learning models are Support Vector Regression, Random Forest and Artificial Neural Networks. The results suggest SVM to be the best performing model among these models for predicting SPI at short time scales (1 month and 3 month) and ANN to be the best performing model for long-term scales (6 months and 12 months). Such findings depict the capabilities of different models in examining drought characteristics and confirming the use of interpolated climatic grids thereby assisting in regional drought management.

Enhanced robustness in machine learning: Application of an adaptive robust loss function

In the field of machine learning, the selection of a loss function plays a pivotal role in determining the training dynamics and generalization capability of models. Traditional loss functions such as mean square error (MSE) and cross-entropy are often not equipped to handle noisy and anomalous data effectively, which can result in models that perform poorly in practical scenarios. To address these shortcomings, this study introduces a novel adaptive robust loss function, enhanced by a tunable parameter , which allows for flexibility in adjusting the robustness of the function according to the nature of the data being processed. Our research demonstrates that this new loss function significantly improves the performance of linear regression and multilayer perceptron models, particularly in environments laden with noisy and anomalous data. By adapting the parameter , the function can cater to varying levels of data irregularities, thus enhancing the models accuracy and reliability across diverse and complex data environments. This adaptive mechanism not only offers a substantial theoretical contribution to the understanding of robust loss functions but also provides a practical tool for machine learning practitioners to develop models that are resilient to data imperfections. The implications of this research are profound, suggesting a shift towards more adaptive and robust approaches in machine learning model development.

A deep transfer learning model for online monitoring of surface roughness in milling with variable parameters

Surface roughness is crucial for the functional and aesthetic properties of mechanical components and must be carefully controlled during machining. However, predicting it under varying machining parameters is challenging due to limited experimental data and fluctuating factors like tool wear and vibration. This study develops a deep transfer learning model that incorporates the correlation alignment method and tool wear to enhance model generalization and reduce data acquisition costs. It utilizes multi-sensor data and the ResNet18 with a convolutional block attention module (CBAM-ResNet) to extract features with improved generalization and accuracy for monitoring milled surface roughness under varying conditions. The performance of the model is evaluated from different perspectives. First, the proposed model achieves high accuracy with fewer than 500 experimental samples from the target domain by using the CORAL module in the CBAM-ResNet model. This demonstrates the model's strong generalization capability by minimizing second-order statistical discrepancies between different datasets. Second, ablation experiments reveal a significant reduction in test error when incorporating CORAL and tool wear, highlighting their contributions to improved model generalization. Integrating tool wear information significantly reduces test errors across various transfer conditions, as it reflects changes in cutting force, vibration, and built-up edge formation. Third, comparisons with existing deep transfer models further emphasize the advantages of the proposed approach in improving model generalization. In summary, the proposed surface roughness model, which incorporates tool wear and multi-sensor signal features as inputs and employs feature transfer and CBAM-ResNet, demonstrates superior generalization and accuracy across various machining parameters.

A Bearing Fault Diagnosis Model with Convolutional Cross Transformer and ResNet18

Abstract In the industrial field, malfunction of rotating machinery, especially bearings, can cause significant economic losses to enterprises. Addressing the limitations of traditional fault diagnosis methods, such as poor generalization performance and low noise resistance, this paper introduces a fault diagnosis model that parallels the Cross Convolutional Transformer and ResNet18 (CCTAR). The proposed CCTAR utilizes two feature extraction channels, aimed at balancing the extraction of local and global features, and the specially designed convolutional cross-decoding layer has excellent noise resistance, surpassing traditional multi-layer Transformer encoding layers with a single-layer structure. CCTAR achieves commendable recognition accuracy across multiple datasets and maintains high accuracy in noisy environments. Furthermore, transfer learning experiments have demonstrated the proposed model's capability to achieve superior fault diagnosis performance across different working conditions with a limited number of samples, highlighting its practical significance.

Dynamic liquid level prediction for multiple oil wells based on transfer learning and multidimensional feature fusion network

Abstract Accurate prediction of the dynamic liquid level (DLL) in oil wells is crucial for the intelligent optimization of pumping systems. It not only provides real-time insights into the operational conditions of the pumping system but also supports the optimization of operational parameters with data. However, due to the long-term operation of oil wells and their complex internal environments, direct measurement of the DLL is challenging, leading to low reliability of the obtained data. Therefore, this paper conducts an in-depth analysis of the parameters involved in the pumping process, identifies the model's input features, and develops a DLL prediction model for multiple wells based on multidimensional feature fusion (MFF). This model captures the characteristics of DLL changes and the diversity of input features. To address the issues of slow model training and low prediction accuracy caused by insufficient datasets in practical applications, this paper integrates transfer learning techniques and proposes a new model, the DLL model for multiple wells based on Transfer Learning and Multidimensional Feature Fusion (TMFF). Initially, the Euclidean distance and Maximum mean discrepancy methods are employed to verify the feature similarity between the source and target domains, using highly similar DLL data as experimental data. By combining transfer learning techniques with the MFF model, the TMFF model is established. The model's capabilities are validated using field-collected data with broad representativeness. Experimental results demonstrate that the proposed MFF model possesses high accuracy and generalization capability. Additionally, the TMFF model effectively resolves the issue of insufficient data during model training. In summary, the methods proposed in this paper can provide accurate DLL data for practical applications in intelligent oilfields.

An Integrated Machine Learning Workflow to Estimate In Situ Stresses Based on Downhole Sonic Logs and Laboratory Triaxial Ultrasonic Velocity Data

AbstractThe optimum performance of various subsurface operations such as stimulation treatments, wellbore drilling, horizontal well placement, underground mining, and tunneling rely on accurate estimation of the in situ stresses. This study presents an integrated machine learning (ML) workflow for the reliable determination of the in situ stress in subsurface rock formations. The study workflow was completed in three phases. In the first phase, six supervised ML regression techniques were employed to develop the stress prediction models using laboratory true triaxial ultrasonic velocity tests (labTUV) data of subsurface rock samples retrieved from well 16A(78)‐32 located at the Utah FORGE geothermal site. In the second phase, subsurface geological formations were classified into rock facies using an unsupervised K‐means clustering algorithm. Finally, in the third phase, the optimized ML models were employed for predicting the in situ stresses in the corresponding rock facies in the well 16A(78)‐32 using field sonic logs. A comparison of evaluation metrics revealed the superior performance of ANN models for horizontal and vertical stress predictions with root mean squared error of 1.5, 0.6, and 1.7 MPa, and determination coefficient (R2) of 0.98, 0.98, and 0.96, for the testing/validation phases, respectively. The generalization capability of ML models was explored by uncovering the underlying physics. The mathematical expressions of constitutive relations were extracted from three ANN models. Further, a total of five rock facies were identified in the subsurface geological formations. The novel workflow would be capable of delivering reliable in situ stress profiles in subsurface geological formations without performing expensive field tests.