Deep Machine Learning Approaches Research Articles

The antibiotic resistance crisis and the development of new antibiotics.

The Global Burden of Disease report of 2019 estimated 14 million infection-related deaths, making it the second leading cause of death after ischaemic heart disease. Bacterial pathogens accounted for 7.7 million deaths and deaths attributable to bacterial antibiotic resistance amounted to 1.3 million, describing a clear demand for novel antibiotics. Antibiotic development had its golden age in 1930-1960. Following failures in the screening of chemical libraries for novel antibiotics at the beginning of this century, the high cost of launching new antibiotics (estimated at US$ 1.4 billion per registered drug) and difficulties in achieving a return of investment for novel antibiotics, pharmaceutical industry has mostly left the field. The current Lilliput review analyses the question whether scientific or economic hurdles prevented the registration of new antibiotics. Scientifically, substantial progress has been achieved over recent years to define the chemical properties needed to overcome the permeation barrier in Gram-negative pathogens; in extending the chemical space of antibiotic candidates by full modular synthesis of suitable molecules; by extending bioprospecting to previously 'unculturable' bacteria or unusual bacteria; by attacking bacterial targets on the outer bacterial membrane; and by looking for support from structural biology, genomics, molecular genetics, phylogenetic analyses and deep machine learning approaches. However, these research activities were mostly conducted by academic researchers and biotech companies with limited financial resources. It thus seems that the development of new antibiotics, frequently described as the drying of the pipeline, is less limited by lack of scientific insight than by lack of the mobilization of the monetary resources needed to bring these discoveries to the market despite recent financial push and pull efforts of the public sector.

Deep machine learning approaches for battery health monitoring

The performance and longevity of batteries can be measured through battery parameters, like: state of charge, state of health, and remaining useful life. Therefore, accurate estimation of these parameters is essential for developing efficient battery management systems. Recently, a variety of smart model-based and data-driven artificial neural network methods have been evolved for efficient management of battery systems. However, there is space for improving the accuracy and reliability of the proposed methods, particularly in the prediction and forecasting of battery performance metrics. In this study, a deep machine learning based forecasting technique has been proposed to better estimate the battery performance parameters. Available open source data-sets on three distinct battery technologies comprising of battery cycling profiles and environmental conditions, are adopted to construct and validate models for predicting battery health. A common data-set is developed by combining the open source data, which is later supplied as input to the predictive analytics and time series analysis. Encoders and decoders are employed for the extraction of important and relevant feature information from the data-set. Three different network model-based architectures are developed to predict the state of charge of Lithium-ion batteries. The proposed models are observed to accurately capture the dynamic behavior of the battery during discharge, which is confirmed by the analysis of the recorded voltage, current, and temperature data, and the precise estimation of state of charge. Furthermore, a novel forecasting method based on the time-series analysis is also introduced. Mean average error values less than 0.2% are observed when neural basis expansion analysis for interpretable time series architecture in conjunction with encoder–decoder-based feature extraction is employed.

Utilizing deep machine learning approach to disclose hematological indices and biomarkers in asthma-COPD overlap

Utilizing deep machine learning approach to disclose hematological indices and biomarkers in asthma-COPD overlap

The Evaluation of Artificial Intelligence Technology for the Differentiation of Fresh Human Blood Cells From Other Species' Blood in the Investigation of Crime Scenes.

The current study used the deep machine learning approach to differentiate human blood specimens from cow, goat, and chicken blood stains based on cellmorphology. A total of 1,955 known Giemsa-stained digitized images were acquired from the blood of humans, cows, goats, and chickens. To train the deep learning models, the well-known VGG16, Resnet18, and Resnet34 algorithms were used. Based on the image analysis, confusion matrices were generated. Findings showed that the F1 score for the chicken, cow, goat, and human classes were all equal to 1.0 for each of the three algorithms. The Matthews correlation coefficient (MCC) was 1 for chickens, cows, and humans in all three algorithms, while the MCC score was 0.989 for goats by ResNet18, and it was 0.994 for both ResNet34 and VGG16 algorithms. The three algorithms showed 100% sensitivity, specificity, and positive and negative predictive values for the human, cow, and chicken cells. For the goatcells, the data showed 100% sensitivity and negative predictive values with specificity and positive predictive values ranging from 98.5% to 99.6%. These data showed the importance of deep learning as a potential tool for the differentiation of the species of origin of fresh crime scene blood stains.

Prediction of sentinel lymph node metastasis in breast cancer patients based on preoperative features: a deep machine learning approach

Sentinel lymph node (SLN) biopsy is the standard surgical approach to detect lymph node metastasis in breast cancer. Machine learning is a novel tool that provides better accuracy for predicting positive SLN involvement in breast cancer patients. This study obtained data from 2890 surgical cases of breast cancer patients from two referral hospitals in Iran from 2000 to 2021. Patients whose SLN involvement status was identified were included in our study. The dataset consisted of preoperative features, including patient features, gestational factors, laboratory data, and tumoral features. In this study, TabNet, an end-to-end deep learning model, was proposed to predict SLN involvement in breast cancer patients. We compared the accuracy of our model with results from logistic regression analysis. A total of 1832 patients with an average age of 51 ± 12 years were included in our study, of which 697 (25.5%) had SLN involvement. On average, the TabNet model achieved an accuracy of 75%, precision of 81%, specificity of 70%, sensitivity of 87%, and AUC of 0.74, while the logistic model demonstrated an accuracy of 70%, precision of 73%, specificity of 65%, sensitivity of 79%, F1 score of 73%, and AUC of 0.70 in predicting the SLN involvement in patients. Vascular invasion, tumor size, core needle biopsy pathology, age, and FH had the most contributions to the TabNet model. The TabNet model outperformed the logistic regression model in all metrics, indicating that it is more effective in predicting SLN involvement in breast cancer patients based on preoperative data.

Correction: Accelerating materials discovery using integrated deep machine learning approaches

Correction for ‘Accelerating materials discovery using integrated deep machine learning approaches’ by Weiyi Xia et al., J. Mater. Chem. A, 2023, 11, 25973–25982, https://doi.org/10.1039/d3ta03771a.

How are poor sleepers with other clinical conditions affected by maladaptive personality traits? A neural network-based analysis.

Psychopathology research mainly focused on the cross-sectional and longitudinal associations between personality and psychiatric disorders without considering the moment-to-moment dynamics of personality in response to environmental situations. The present study aimed to both cluster a young sample according to three mixed clinical conditions (poor sleep quality, depression, and somatization) and to predict the derived clusters by maladaptive personality traits and sex differences using a deep machine learning approach. A sample of 839 adults aged 18-40 years (64% female) from the west of Iran were clustered according to the mixed clinical conditions using the cluster analysis techniques. An Artificial Neural Network (ANN) modeling is used to predict the derived clusters by maladaptive personality traits and biological gender. A receiver operating characteristic (ROC) curve was used to identify independent variables with high sensitivity specific to the derived clusters. The cluster analysis techniques suggested a fully stable and acceptable four-cluster solution for Depressed Poor Sleepers, Nonclinical Good Sleepers, Subclinical Poor Sleepers, and Clinical Poor Sleepers. The ANN model led to the identification of one hidden layer with two hidden units. The results of Area under the ROC Curve were relatively to completely acceptable, ranging from.726 to.855. Anhedonia, perceptual dysregulation, depressivity, anxiousness, and unusual beliefs are the most valuable traits with importance higher than 70%. The machine learning approach can be well used to predict mixed clinical conditions by maladaptive personality traits. Future research can test the complexity of normal personality traits connected to mixed clinical conditions.

Assessing the impact of borehole field data on AI-based deep learning models for heating and cooling prediction

Sensor readings play a critical role in the prediction performance of AI-based data-driven fault-detection and diagnostics for borehole heat exchangers in heating and cooling systems. The accuracy and algorithm selection of individual deep neural networks are well-investigated research topics but the impact of quality of sensors data on algorithm selection and its further impact on prediction performance has not been investigated and quantified for geothermal heating/cooling systems. The objective of the current work is to investigate the impact of data quality, available as measured time-series data from the borehole field, on the prediction accuracy of the deep neural networks-based models. In the current work four different data sets (D-1, D-2, D-3, D-4) are used for data-driven modeling using four different deep machine learning approaches, i.e., LSTM, BD-LSTM, GRU and CNN. The developed AI models have been trained, validated, and tested using the real-field data sets with same number of independent input parameters i.e., inlet temperature and mass flow rate to predict the fluid outlet temperature. The validated AI models are used to predict the performance of borehole heat exchanger to meet the heating and cooling loads. The prediction results show that the BD-LSTM outperforms other deep neural networks in terms of accuracy. With the data sets D-1, D-2, D-3 and D-4 it was observed that BD-LSTM had lowest MAPE of 2.99 %, 0.22 %, 6.39 % and 2.21 % respectively. Several experiments show that the BD-LSTM and LSTM models are better than the other used approaches in terms of prediction accuracy, but larger number of trainable parameters results in more computation time than CNN and GRU respectively. More data points with higher sampling frequency, results in more accurate prediction as depicted by the results from D-2 and D-4. Although, the results showed that the data collection interval is crucial, but different test runs indicate that avoiding the loss of time-series data during the cleaning process is equally important to accurately capture the Seasonality and Trend of the heating/cooling data. This is realized in case of D-1 where almost 58 % of the values were removed as outliers and these missing values were needed to be approximated which also affected the model prediction accuracy. The presented analysis highlights that the LSTM based DNN model has the capability to predict borehole heat exchanger performance accurately and the predictions are highly dependent on data frequency, the quality as well as data quantity from geothermal installations used to train the model.

Deep learning design of functionally graded porous electrode of proton exchange membrane fuel cells

For the next generation of proton exchange membrane (PEM) fuel cells, the conventional electrode with uniform distribution of functional components is urged to be replaced by functional graded electrode for the prominent performance, efficiency and avoid exacerbated catalyst cost. Due to the complex and non-linear behaviours of PEM fuel cell system, rapid and effective computational model and optimisation algorithm are required to handle such a complex relationship between electrode design parameters and cell performance. In this work, a multi-physics model with multi-directionally graded electrode is developed, in which a deep machine learning approach is embedded, to create a surrogate model for multi-objective optimisation empowered by non-dominated sort genetic algorithm (NSGA-II). A robust prediction deep neural network model with the mean square error lower than 0.01 is obtained from training and then coupling with NSGA-II to evaluate and optimise the fuel cell performances and cost. Remarkably, the Pareto front is successfully defining the trade-off relationship between the objectives where it aids to identify an optimum point where it satisfies the cost effectiveness while maintaining relatively high cell performances. Our work presents a promising strategy to optimise the fuel cell system with underlying interaction and allow rapid and accurate prediction and optimisation.

Deep machine learning for the PANDA software trigger

Deep machine learning methods have been studied for the software trigger of the future PANDA experiment at FAIR, using Monte Carlo simulated data from the GEANT-based detector simulation framework PandaRoot. Ten physics channels that cover the main physics topics, including electromagnetic, exotic, charmonium, open charm, and baryonic reaction channels, have been investigated at four different anti-proton beam momenta. Different classification concepts and network architectures have been studied. Finally a residual convolutional neural network with four residual blocks in a binary classification scheme has been chosen due to its extendability, performance and stability. The presented study represents a feasibility study of a completely software-based trigger system. Compared to a conventional selection method, the deep machine learning approach achieved a significant efficiency gain of up to 200%, while keeping the background reduction factor at the required level of 1/1000. Furthermore, it is shown that the use of additional input variables can improve the data quality for subsequent analysis. This study shows that the PANDA software trigger can benefit greatly from the deep machine learning methods.

A Real-Time Deep Machine Learning Approach for Sudden Tool Failure Prediction and Prevention in Machining Processes.

Tool Condition Monitoring systems are essential to achieve the desired industrial competitive advantage in terms of reducing costs, increasing productivity, improving quality, and preventing machined part damage. A sudden tool failure is analytically unpredictable due to the high dynamics of the machining process in the industrial environment. Therefore, a system for detecting and preventing sudden tool failures was developed for real-time implementation. A discrete wavelet transform lifting scheme (DWT) was developed to extract a time-frequency representation of the AErms signals. A long short-term memory (LSTM) autoencoder was developed to compress and reconstruct the DWT features. The variations between the reconstructed and the original DWT representations due to the induced acoustic emissions (AE) waves during unstable crack propagation were used as a prefailure indicator. Based on the statistics of the LSTM autoencoder training process, a threshold was defined to detect tool prefailure regardless of the cutting conditions. Experimental validation results demonstrated the ability of the developed approach to accurately predict sudden tool failures before they occur and allow enough time to take corrective action to protect the machined part. The developed approach overcomes the limitations of the prefailure detection approach available in the literature in terms of defining a threshold function and sensitivity to chip adhesion-separation phenomenon during the machining of hard-to-cut materials.

Performing Automatic Identification and Staging of Urothelial Carcinoma in Bladder Cancer Patients Using a Hybrid Deep-Machine Learning Approach

Accurate clinical staging of bladder cancer aids in optimizing the process of clinical decision-making, thereby tailoring the effective treatment and management of patients. While several radiomics approaches have been developed to facilitate the process of clinical diagnosis and staging of bladder cancer using grayscale computed tomography (CT) scans, the performances of these models have been low, with little validation and no clear consensus on specific imaging signatures. We propose a hybrid framework comprising pre-trained deep neural networks for feature extraction, in combination with statistical machine learning techniques for classification, which is capable of performing the following classification tasks: (1) bladder cancer tissue vs. normal tissue, (2) muscle-invasive bladder cancer (MIBC) vs. non-muscle-invasive bladder cancer (NMIBC), and (3) post-treatment changes (PTC) vs. MIBC.

Accelerating materials discovery using integrated deep machine learning approaches

Our work introduces an innovative deep machine learning framework to significantly accelerate novel materials discovery, as demonstrated by its application to the La–Si–P system where new ternary and quaternary compounds were successfully identified.

Prediction models for marshall mix parameters using bio-inspired genetic programming and deep machine learning approaches: A comparative study

This research study utilizes four machine learning techniques, i.e., Multi Expression programming (MEP), Artificial Neural Network (ANN), Adaptive Neuro-Fuzzy Inference System (ANFIS), and Ensemble Decision Tree Bagging (DT-Bagging) for the development of new and advanced models for prediction of Marshall Stability (MS), and Marshall Flow (MF) of asphalt mixes. A comprehensive and detailed database of 343 data points was established for both MS and MF. The predicting variables were chosen among the four most influential, and easy-to-determine parameters. The models were trained, tested, validated, and the outcomes of the newly developed models were compared with actual outcomes. The root squared error (RSE), Nash-Sutcliffe efficiency (NSE), mean absolute error (MAE), root mean square error (RMSE), relative root mean square error (RRMSE), regression coefficient (R2), and correlation coefficient (R), were all used to evaluate the performance of models. The sensitivity analysis (SA) revealed that in the case of MS, the rising order of input significance was bulk specific gravity of compacted aggregate, Gmb (38.56 %) > Percentage of Aggregates, Ps (19.84 %) > Bulk Specific Gravity of Aggregate, Gsb (19.43 %) > maximum specific gravity paving mix, Gmm (7.62 %), while in case of MF the order followed was: Ps (36.93 %) > Gsb (14.11 %) > Gmb (10.85 %) > Gmm (10.19 %). The outcomes of parametric analysis (PA) consistency of results in relation to previous research findings. The DT-Bagging model outperformed all other models with values of 0.971 and 0.980 (R), 16.88 and 0.24 (MAE), 28.27 and 0.36 (RMSE), 0.069 and 0.041 (RSE), 0.020 and 0.032 (RRMSE), 0.010 and 0.016 (PI), 0.931 and 0.959 (NSE), for MS and MF, respectively. The results of the comparison analysis showed that ANN, ANFIS, MEP, and DT-Bagging are all effective and reliable approaches for the estimation of MS and MF. The MEP-derived mathematical expressions represent the novelty of MEP and are relatively simple and reliable. Roverall values for MS and MF were in the order of DT-Bagging >MEP >ANFIS >ANN with all values exceeding the permitted range of 0.80 for both MS and MF. Hence, all the modeling approaches showed higher performance, possessed high generalization and predication capabilities, and assess the relative significance of input parameters in the prediction of MS and MF. Hence, the findings of this research study would assist in the safer, faster, and sustainable prediction of MS and MF, from the standpoint of resources and time required to perform the Marshall tests.

Identification of surface defects using deep and transfer learning

The purpose of this study was to employ a previously trained (pre-trained) convolutional neural network called Resnet101 in conjunction with deep machine learning approaches in order to construct an algorithm for classifying cracks in the photos that were evaluated. Adjustments were made to the ultimate layer, which resulted in the fully connected layer being altered. Specifically, the basic 1000-output fully connected layer in Resnet101 was replaced with a binary-classification layer, which consisted of two categories: an image with cracks and an image without cracks. In this study, we investigate whether or not it is possible to use deep neural networks to accomplish the rapid and entirely automated detection of flaws by utilizing analyzed photographs as the data source. The research that was done led to the discovery that a pre-trained convolutional neural network that makes use of support vector machines to train a fully connected layer is quite an efficient option, and that the acquired forecasting algorithm allows the categorization of faults with extremely good accuracy. The proposed classification algorithm is 99 percent efficient. In material inspection tasks, this idea can be used to find cracks and other flaws in the material, such as those that could be found in a number of public structures like buildings, roads, and bridges.

A Hybrid Model to Predict Stock Closing Price Using Novel Features and a Fully Modified Hodrick–Prescott Filter

Forecasting stock market prices is an exciting knowledge area for investors and traders. Successful predictions lead to high financial revenues and prevent investors from market risks. This paper proposes a novel hybrid stock prediction model that improves prediction accuracy. The proposed method consists of three main components, a noise-filtering technique, novel features, and machine learning-based prediction. We used a fully modified Hodrick–Prescott filter to smooth the historical stock price data by removing the cyclic component from the time series. We propose several new features for stock price prediction, including the return of firm, return open price, return close price, change in return open price, change in return close price, and volume per total. We investigate traditional and deep machine learning approaches for prediction. Support vector regression, auto-regressive integrated moving averages, and random forests are used for conventional machine learning. Deep learning techniques comprise long short-term memory and gated recurrent units. We performed several experiments with these machine learning algorithms. Our best model achieved a prediction accuracy of 70.88%, a root-mean-square error of 0.04, and an error rate of 0.1.

Four-Dimension Seismic Analysis in Carbonate: A Closed Loop Study

Four-dimensional seismic analysis is an effective reservoir surveillance tool to track the changes of fluid and pressure phases in the oil and gas reservoirs over time of the baseline and monitoring seismic acquisition. In practice, the 4D seismic signal associated with such changes may be negligible, especially in heterogeneous carbonate reservoirs. Therefore, 4D seismic analysis is a model for integrating various disciplines in the oil and gas industry, such as seismic, petrophysics, reservoir engineering, and production engineering. In this study, we started the 4D seismic workflow with a 1D well-based 4D feasibility study to detect the likelihood of 4D signals before performing 4D seismic co-processing of the baseline and monitoring surveys starting from the seismic field data of both datasets. As part of a full 4D seismic co-processing of the baseline and monitor surveys, 4D seismic metric attributes were analyzed over the survey area to measure the improvement in seismic acquisition repeatability for the scarce 1994 baseline seismic and the 2014 monitor seismic survey. For the monitor survey, a 4D time-trace shift was performed using the baseline survey as a reference to measure the time shifts between the baseline and monitor surveys at 20-year intervals. The 4DFour-dimensional dynamic trace warping was followed by a 4D seismic inversion to compare the 4D difference in the seismic inverted data with the difference in seismic amplitude. The seismic inversion helped overcome noise, multiple contaminations, and differences in dynamic amplitude range between the baseline and monitor seismic surveys. We then examined the relationship between well logs and seismic volumes by predicting a volume of log properties at the well locations of the seismic volume. In this method, we computed a possibly nonlinear operator that can predict well logs based on the properties of adjacent seismic data. We then tested a Deep Feed Forward Neural Network (DFNN) on six wells to adequately train and validate the machine learning approach using the baseline and monitoring seismic inverted data. The objective of trying such a deep machine learning approach was to predict the density and porosity of both the baseline and the monitoring seismic data to validate the accuracy of the 4D seismic inversion and evaluate the changes in reservoir properties over a time-lapse of 20 years of production from 1994 to 2014. Finally, we matched the 4D seismic signal with changes in reservoir production properties, investigating the mechanism underlying the observed 4D signal. It was found that the detectability of 4D signals is primarily related to changes in fluid saturation and pressure changes in the reservoir, which increased from 1994 to 2014. This innovative closed-loop research proved that the low repeatability of seismic acquisition can be compensated by optimal 4D seismic co-processing with a complete integration workflow to assess the reliability of the 4D seismic observed signal.

Deep Learning Applications for Acute Stroke Management.

Brain imaging is essential to the clinical care of patients with stroke, a leading cause of disability and death worldwide. Whereas advanced neuroimaging techniques offer opportunities for aiding acute stroke management, several factors, including time delays, inter-clinician variability, and lack of systemic conglomeration of clinical information, hinder their maximal utility. Recent advances in deep machine learning (DL) offer new strategies for harnessing computational medical image analysis to inform decision making in acute stroke. We examine the current state of the field for DL models in stroke triage. First, we provide a brief, clinical practice-focused primer on DL. Next, we examine real-world examples of DL applications in pixel-wise labeling, volumetric lesion segmentation, stroke detection, and prediction of tissue fate postintervention. We evaluate recent deployments of deep neural networks and their ability to automatically select relevant clinical features for acute decision making, reduce inter-rater variability, and boost reliability in rapid neuroimaging assessments, and integrate neuroimaging with electronic medical record (EMR) data in order to support clinicians in routine and triage stroke management. Ultimately, we aim to provide a framework for critically evaluating existing automated approaches, thus equipping clinicians with the ability to understand and potentially apply DL approaches in order to address challenges in clinical practice. ANN NEUROL 2022;92:574-587.

Multi phenomena melt pool sensor data fusion for enhanced process monitoring of laser powder bed fusion additive manufacturing

Finding actionable trends in laser-based metal additive manufacturing process monitoring data is challenging owing to the diversity and complexity of the underlying physical interactions. A single monitoring solution that captures a particular process phenomenon, such as a photodiode that tracks melt pool intensity, is not alone capable of evaluating process stability or detecting flaw formation with sufficient precision for routine application in industry. In this work, to improve flaw detection performance, we adopted a data fusion approach that captures multiple process phenomena. To demonstrate this, we acquired data from laser powder bed fusion (LPBF) builds of cylindrical specimens produced with different laser spot sizes, emulating defocusing due to process faults such as thermal lensing. The resulting specimens had porosity of varying types and severity, quantified by post-build non-destructive X-ray computed tomography, Archimedes density measurements, and destructive metallographic characterization. During the build, the melt pool state was monitored with two coaxial high-speed video cameras and a temperature field imaging system. Physically intuitive low-level melt pool signatures, such as melt pool temperature, shape and size, and spatter intensity were extracted from this high-dimensional, image-based sensor data. These process signatures were subsequently used as input features in relatively simple machine learning models, such as a support vector machine, which were trained to detect laser defocusing, and in addition, predict porosity type and severity. The results show that the data fusion approach significantly enhanced system performance by reducing the overall false positive rate from ∼ 0.1 to ∼ 0.001 without sacrificing the true positive rate (∼0.90). These results were at par with a black-box, deep machine learning approach (convolutional neural network).

The Importance of Being Interpretable: Toward an Understandable Machine Learning Encoder for Galaxy Cluster Cosmology

Abstract We present a deep machine-learning (ML) approach to constraining cosmological parameters with multiwavelength observations of galaxy clusters. The ML approach has two components: an encoder that builds a compressed representation of each galaxy cluster and a flexible convolutional neural networks to estimate the cosmological model from a cluster sample. It is trained and tested on simulated cluster catalogs built from the Magneticum simulations. From the simulated catalogs, the ML method estimates the amplitude of matter fluctuations, σ 8, at approximately the expected theoretical limit. More importantly, the deep ML approach can be interpreted. We lay out three schemes for interpreting the ML technique: a leave-one-out method for assessing cluster importance, an average saliency for evaluating feature importance, and correlations in the terse layer for understanding whether an ML technique can be safely applied to observational data. These interpretation schemes led to the discovery of a previously unknown self-calibration mode for flux- and volume-limited cluster surveys. We describe this new mode, which uses the amplitude and peak of the cluster mass probability density function as anchors for mass calibration. We introduce the term overspecialized to describe a common pitfall in astronomical applications of ML in which the ML method learns simulation-specific details, and we show how a carefully constructed architecture can be used to check for this source of systematic error.