Dense Neural Network Research Articles

Transformer-Decoder GPT Models for Generating Virtual Screening Libraries of HMG-Coenzyme A Reductase Inhibitors: Effects of Temperature, Prompt Length, and Transfer-Learning Strategies.

Attention-based decoder models were used to generate libraries of novel inhibitors for the HMG-Coenzyme A reductase (HMGCR) enzyme. These deep neural network models were pretrained on previously synthesized drug-like molecules from the ZINC15 database to learn the syntax of SMILES strings and then fine-tuned with a set of ∼1000 molecules that inhibit HMGCR. The number of layers used for pretraining and fine-tuning was varied to find the optimal balance for robust library generation. Virtual screening libraries were also generated with different temperatures and numbers of input tokens (prompt length) to find the most desirable molecular properties. The resulting libraries were screened against several criteria, including IC50 values predicted by a dense neural network (DNN) trained on experimental HMGCR IC50 values, docking scores from AutoDock Vina (via Dockstring), a calculated quantitative estimate of druglikeness, and Tanimoto similarity to known HMGCR inhibitors. It was found that 50/50 or 25/75% pretrained/fine-tuned models with a nonzero temperature and shorter prompt lengths produced the most robust libraries, and the DNN-predicted IC50 values had good correlation with docking scores and statin similarity. 42% of generated molecules were classified as statin-like by k-means clustering, with the rosuvastatin-like group having the lowest IC50 values and lowest docking scores.

Cascadable optical nonlinear activation function based on Ge-Si.

To augment the capabilities of optical computing, specialized nonlinear devices as optical activation functions are crucial for enhancing the complexity of optical neural networks. However, existing optical nonlinear activation function devices often encounter challenges in preparation, compatibility, and multi-layer cascading. Here, we propose a cascadable optical nonlinear activation function architecture based on Ge-Si structured devices. Leveraging dual-source modulation, this architecture achieves cascading and wavelength switching by compensating for loss. Experimental comparisons with traditional Ge-Si devices validate the cascading capability of the new architecture. We first verified the versatility of this activation function in a MNIST task, and then in a multi-layer optical dense neural network designed for complex gesture recognition classification, the proposed architecture improves accuracy by an average of 23% compared to a linear network and 15% compared to a network with a traditional activation function architecture. With its advantages of cascadability and high compatibility, this work underscores the potential of all-optical activation functions for large-scale optical neural network scaling and complex task handling.

Predicting the Physical Properties of Dark Matter Subhalos from Baryonic Parameters Using Machine Learning

Dark matter subhalos play an important role in galaxy formation and evolution. However, accurate prediction of dark matter properties remains a challenge of modern-day astronomy. In recent times, machine learning (ML) tools have shown promising results in solving numerous astrophysical problems. In this paper, we use data from the EAGLE simulations to determine the total mass and the half-mass radius of dark matter subhalos using structural properties of gas, star, black hole, and photometric features using gradient boosted decision trees (GBDT) and dense neural network. GBDT does not require data preprocessing, and results in better performance compared to the neural network. According to GBDT, the most important feature for subhalo radius and mass estimation is gas radius and black hole mass respectively. The all-features combined approach results in the highest test accuracy — Pearson’s correlation coefficient = 0.947 and 0.981, coefficient of determination = 0.898 and 0.962, normalized median absolute deviation = 0.111 and 0.114 for radius and mass respectively. We evaluate our model for masses and redshifts beyond its training range and find that GBDT demonstrates significantly better extrapolation capabilities than the neural network. We also test our model on simulations with different resolutions, and find that the discrepancies lie within 10% if the resolution is changed. This novel study incorporates the structural parameters of gas and black hole to determine the dark matter properties using a ML-based approach. The promising results of this study prove that ML tools can improve our current understanding of dark matter, and answer some of the basic cosmological questions.

Proof of concept study on early forecasting of antimicrobial resistance in hospitalized patients using machine learning and simple bacterial ecology data.

Antibiotic resistance in bacterial pathogens is a major threat to global health, exacerbated by the misuse of antibiotics. In hospital practice, results of bacterial cultures and antibiograms can take several days. Meanwhile, prescribing an empirical antimicrobial treatment is challenging, as clinicians must balance the antibiotic spectrum against the expected probability of susceptibility. We present here a proof of concept study of a machine learning-based system that predicts the probability of antimicrobial susceptibility and explains the contribution of the different cofactors in hospitalized patients, at four different stages prior to the antibiogram (sampling, direct examination, positive culture, and species identification), using only historical bacterial ecology data that can be easily collected from any laboratory information system (LIS) without GDPR restrictions once the data have been anonymised. A comparative analysis of different state-of-the-art machine learning and probabilistic methods was performed using 44,026 instances over 7 years from the Hôpital Européen Marseille, France. Our results show that multilayer dense neural networks and Bayesian models are suitable for early prediction of antibiotic susceptibility, with AUROCs reaching 0.88 at the positive culture stage and 0.92 at the species identification stage, and even 0.82 and 0.92, respectively, for the least frequent situations. Perspectives and potential clinical applications of the system are discussed.

Analyzing the impact of conversation structure on predicting persuasive comments online

The topic of persuasion in online conversations has social, political and security implications; as a consequence, the problem of predicting persuasive comments in online discussions is receiving increasing attention in the literature. Following recent advancements in graph neural networks, we analyze the impact of conversation structure in predicting persuasive comments in online discussions. We evaluate the performance of artificial intelligence models receiving as input graphs constructed on the top of online conversations sourced from the “Change My View” Reddit channel. We experiment with different graph architectures and compare the performance on graph neural networks, as structure-based models, and dense neural networks as baseline models. Experiments are conducted on two tasks: (1) persuasive comment detection, aiming to predict which comments are persuasive, and (2) influence prediction, aiming to predict which users are persuasive. The experimental results show that the role of the conversation structure in predicting persuasiveness is strongly dependent on its graph representation given as input to the graph neural network. In particular, a graph structure linking only comments belonging to the same speaker in the conversation achieves the best performance in both tasks. This structure outperforms both the baseline model, which does not consider any structural information, and structures linking different speakers’ comments with each other. Specifically, the F1 score of the best performing model is 0.58, which represents an improvement of 5.45% over the baseline model (F1 score of 0.55) and 7.41% over the model linking different speakers’ comments (F1 score of 0.54).

Deep Learning–Based Enhanced Optimization for Automated Rice Plant Disease Detection and Classification

ABSTRACTEnsuring global food security necessitates innovative solutions for early detection and precise classification of diseases in staple crops like rice. This study introduces an advanced approach for automated rice plant disease detection and classification by integrating deep learning and metaheuristic optimization techniques. Specifically, a deep dense neural network (DNN) is employed for its capacity to capture intricate patterns in images and extreme learning machine (ELM) for classification. To enhance the optimization process, an innovative variant of the Shuffled Shepherd Optimization (SSO) algorithm, known as Enhanced Artificial Shuffled Shepherd Optimization (EASSO), is introduced. EASSO incorporates adaptive strategies and enhanced exploration–exploitation mechanisms, enabling more efficient convergence during the training of the DNN. The proposed system operates by processing high‐resolution images of rice plants through the DNN, extracting nuanced features indicative of various diseases, including blast, bacterial blight, and brown spots. EASSO optimizes the DNN's parameters, maximizing its accuracy in disease classification. The synergy between DNN and EASSO ensures a robust and adaptive model capable of handling diverse and complex disease patterns. This automated approach significantly reduces the reliance on manual inspection, enabling timely intervention and improving overall agricultural productivity. Experimental results demonstrate the superiority of the DNN‐EASSO framework over traditional methods, showcasing higher accuracy rates and faster convergence. The incorporation of Enhanced Artificial Shuffled Shepherd Optimization enhances the precision and reliability of disease classification, making this integrated system a valuable tool for farmers and agricultural practitioners. This research represents a significant stride toward sustainable agriculture, showcasing the potential of advanced technologies in ensuring food security worldwide.

Retraction Note: Transparency in healthcare and e-commerce: detecting online fake reviews using a dense neural network model with relevance mapping

Retraction Note: Transparency in healthcare and e-commerce: detecting online fake reviews using a dense neural network model with relevance mapping

Estimation of the Global Equivalence Ratio of a Swirl Combustor from a Small Number of Absorption Spectra Using Machine Learning.

A new optical diagnostic method that predicts the global fuel-air equivalence ratio of a swirl combustor using absorption spectra from only three optical paths is proposed here. Under normal operation, the global equivalence ratio and total flow rate determine the temperature and concentration fields of the combustor, which subsequently determine the absorption spectra of any combustion species. Therefore, spectra, as the fingerprint for a produced combustion field, were employed to predict the global equivalence ratio, one of the key operational parameters, in this study. Specifically, absorption spectra of water vapor at wavenumbers around 7444.36, 7185.6, and 6805.6 cm-1 measured at three different downstream locations of the combustor were used to predict the global equivalence ratio. As it is difficult to find analytical relationships between the spectra and produced combustion fields, a predictive model was a data-driven acquisition. The absorption spectra as an input were first feature-extracted through stacked convolutional autoencoders and then a dense neural network was used for regression prediction between the feature scores and the global equivalence ratio. The model could predict the equivalence ratio with an absolute error of ±0.025 with a probability of 96%, and a gradient-weighted regression activation mapping analysis revealed that the model leverages not only the peak intensities but also the variations in the shape of absorption lines for its predictions.

Human movement monitoring system for classification of strength exercises and verification of their execution technique

Human movement analysis is critical to optimizing sports training and influencing exercise intensity and effectiveness. In the age of modern technology, more and more advanced systems are emerging to support coaches and expand the range of analysis performed. This article aims to verify that artificial intelligence, together with machine learning algorithms, can accurately classify exercises in a dynamic gym environment and effectively assess the correctness of their performance. For the initial analysis of movement, the Google MediaPipe Pose model was used, which was responsible for detecting the human silhouette and determining the coordinates of the position of critical joints. Based on these coordinates, the angles between each joint were calculated, and then their sequences were further analyzed. The sequences were analyzed using the following three algorithms: support vector machine (SVM), dense neural network, and LSTM recurrent network. As a result, the system based on recurrent LSTM networks achieved the best prediction efficiency of approximately 98%, enabling accurate exercise classification. Subsequently, verification of the activities' correctness was also carried out, and the system, based on recursive LSTM networks, again achieved the best efficiency, this time equal to 96% on average for all exercises. On this basis, it was concluded that the discussed approach enables practical analysis of human movement, which can significantly improve training methods and facilitate coaching work.

Solvation Enthalpies and Free Energies for Organic Solvents through a Dense Neural Network: A Generalized-Born Approach

A dense artificial neural network, ESE-ΔH-DNN, with two hidden layers for calculating both solvation free energies ΔG°solv and enthalpies ΔH°solv for neutral solutes in organic solvents is proposed. The input features are generalized-Born-type monatomic and pair electrostatic terms, the molecular volume, and atomic surface areas of the solute, as well as five easily available properties of the solvent. ESE-ΔH-DNN is quite accurate for ΔG°solv, with an RMSE (root mean square error) below 0.6 kcal/mol and an MAE (mean absolute error) well below 0.4 kcal/mol. It performs particularly well for alkane, aromatic, ester, and ketone solvents. ESE-ΔH-DNN also exhibits a fairly good accuracy for ΔH°solv prediction, with an RMSE below 1 kcal/mol and an MAE of about 0.6 kcal/mol.

Dimension Reduction of Multidimensional Structured and Unstructured Datasets through Ensemble Learning of Neural Embeddings

Dimension reduction aims to project a high‐dimensional dataset into a low‐dimensional space. It tries to preserve the topological relationships among the original data points and/or induce clusters. NetDRm, an online dimensionality reduction method based on neural ensemble learning that integrates different dimension reduction methods in a synergistic way, is introduced. NetDRm is designed for datasets of multidimensional points that can be either structured (e.g., images) or unstructured (e.g., point clouds, tabular data). It starts by training a collection of deep residual encoders that learn the embeddings induced by multiple dimension reduction methods applied to the input dataset. Subsequently, a dense neural network integrates the generated encoders by emphasizing topological preservation or cluster induction. Experiments conducted on widely used multidimensional datasets (point‐cloud manifolds, image datasets, tabular record datasets) show that the proposed method yields better results in terms of topological preservation ( curves), cluster induction (V measure), and classification accuracy than the most relevant dimension reduction methods.

Convolutional neural network classification of beam profiles from silicon photonics gratings

Convolutional neural network (CNN) models consist of CNN block(s), and dense neural network (DNN) block(s) are used to perform image classification on beam profiles in light beams coupled out from silicon photonics (SiPh) mixed-pitch gratings. The beam profiles are first simulated and segregated into three categories based on their corresponding height above the SiPh gratings. With one CNN block, one DNN block, and 128 nodes in the DNN block, classification accuracy of 98.68% is achieved when classifying 454 beam profile images to their corresponding categories. Expanding the number of CNN blocks, DNN blocks, and nodes, 64 CNN models are constructed, trained, and evaluated. Out of the 64 CNN models, 52 of them achieved classification accuracy of >95%.

A Real-Time Intrusion Detection System for DoS/DDoS Attack Classification in IoT Networks Using KNN-Neural Network Hybrid Technique

As more devices are connected to the Internet through the Internet of Things (IoT), there are huge security challenges. One of the major problems is Distributed Denial of Service (DoS) and Distributed Denial of Service (DDoS) attacks. These attacks floods networks with useless traffic, disrupting IoT services. There is a need for better security measures to handle this. Intrusion Detection Systems (IDS) is used to find suspicious activities, but many of them can't keep up with new types of attacks in real time. This study focuses on creating an efficient real time hybrid framework that uses the Kth Nearest Neighbor (KNN) algorithm and dense neural networks. This proposed framework aims to identify and categorize DoS/DDoS attacks in real time through the utilization of a simulation model and MQTT-IoT-IDS2020 dataset and compared with existing frameworks, our proposed framework excels in accuracy, precision, and recall.

Explainable AI based automated segmentation and multi-stage classification of gastroesophageal reflux using machine learning techniques

Presently, close to two million patients globally succumb to gastrointestinal reflux diseases (GERD). Video endoscopy represents cutting-edge technology in medical imaging, facilitating the diagnosis of various gastrointestinal ailments including stomach ulcers, bleeding, and polyps. However, the abundance of images produced by medical video endoscopy necessitates significant time for doctors to analyze them thoroughly, posing a challenge for manual diagnosis. This challenge has spurred research into computer-aided techniques aimed at diagnosing the plethora of generated images swiftly and accurately. The novelty of the proposed methodology lies in the development of a system tailored for the diagnosis of gastrointestinal diseases. The proposed work used an object detection method called Yolov5 for identifying abnormal region of interest and Deep LabV3+ for segmentation of abnormal regions in GERD. Further, the features are extracted from the segmented image and given as an input to the seven different machine learning classifiers and custom deep neural network model for multi-stage classification of GERD. The DeepLabV3+ attains an excellent segmentation accuracy of 95.2% and an F1 score of 93.3%. The custom dense neural network obtained a classification accuracy of 90.5%. Among the seven different machine learning classifiers, support vector machine (SVM) outperformed with classification accuracy of 87% compared to all other class outperformed combination of object detection, deep learning-based segmentation and machine learning classification enables the timely identification and surveillance of problems associated with GERD for healthcare providers.

[formula omitted][formula omitted]ave [formula omitted]earner: Predicting wave farms power output using effective meta-learner deep gradient boosting model: A case study from Australian coasts

Precise prediction of wave energy is indispensable and holds immense promise as ocean waves have a power capacity of 30–40 kW/m along the coast. Utilising this energy source does not generate harmful emissions, making it a superior substitute for fossil fuel-based energy. The computational expense associated with simulating and computing intricate hydrodynamic interactions in wave farms restricts optimisation methods to a few thousand evaluations and makes a challenging situation for training in deep neural prediction models. To address this issue, we propose a new solution: a Meta-learner gradient boosting method that employs four multi-layer convolutional dense neural network surrogate models combined with an optimised extreme gradient boosting. In order to train and validate the predictive model, we used four wave farm datasets, including the absorbed power outputs and 2D coordinates of wave energy converters (WECs) located along the southern coast of Australia, Adelaide, Sydney, Perth and Tasmania. Furthermore, the capability of the transfer learning strategy is evaluated. The WECs used in this study are of the fully submerged three-tether converter type, similar to the CETO prototype. The effectiveness of the proposed approach is assessed by comparing it with 15 well-established and effective machine learning (ML) methods. The experimental findings indicate that the proposed model is competitive with other ML and deep learning approaches, exhibiting considerable accuracy of 88.8%, 90.0%, 90.3%, and 84.4% in Adelaide, Perth, Sydney and Tasmania and improved robustness in predicting wave farm power output.

Development of a differential treatment selection model for depression on consolidated and transformed clinical trial datasets

Major depressive disorder (MDD) is the leading cause of disability worldwide, yet treatment selection still proceeds via “trial and error”. Given the varied presentation of MDD and heterogeneity of treatment response, the use of machine learning to understand complex, non-linear relationships in data may be key for treatment personalization. Well-organized, structured data from clinical trials with standardized outcome measures is useful for training machine learning models; however, combining data across trials poses numerous challenges. There is also persistent concern that machine learning models can propagate harmful biases. We have created a methodology for organizing and preprocessing depression clinical trial data such that transformed variables harmonized across disparate datasets can be used as input for feature selection. Using Bayesian optimization, we identified an optimal multi-layer dense neural network that used data from 21 clinical and sociodemographic features as input in order to perform differential treatment benefit prediction. With this combined dataset of 5032 individuals and 6 drugs, we created a differential treatment benefit prediction model. Our model generalized well to the held-out test set and produced similar accuracy metrics in the test and validation set with an AUC of 0.7 when predicting binary remission. To address the potential for bias propagation, we used a bias testing performance metric to evaluate the model for harmful biases related to ethnicity, age, or sex. We present a full pipeline from data preprocessing to model validation that was employed to create the first differential treatment benefit prediction model for MDD containing 6 treatment options.

Euclid: Testing photometric selection of emission-line galaxy targets

Multi-object spectroscopic galaxy surveys typically make use of photometric and colour criteria to select their targets. That is not the case of which will use the NISP slitless spectrograph to record spectra for every source over its field of view. Slitless spectroscopy has the advantage of avoiding defining a priori a specific galaxy sample, but at the price of making the selection function harder to quantify. In its Wide Survey was designed to build robust statistical samples of emission-line galaxies with fluxes brighter than $ 2e-16 erg s cm $, using the Halpha -$ N ii right $ complex to measure redshifts within the range $ $. Given the expected signal-to-noise ratio of NISP spectra, at such faint fluxes a significant contamination by incorrectly measured redshifts is expected, either due to misidentification of other emission lines, or to noise fluctuations mistaken as such, with the consequence of reducing the purity of the final samples. This can be significantly ameliorated by exploiting the extensive photometric information to identify emission-line galaxies over the redshift range of interest. Beyond classical multi-band selections in colour space, machine learning techniques provide novel tools to perform this task. Here, we compare and quantify the performance of six such classification algorithms in achieving this goal. We consider the case when only the photometric and morphological measurements are used, and when these are supplemented by the extensive set of ancillary ground-based photometric data, which are part of the overall scientific strategy to perform lensing tomography. The classifiers are trained and tested on two mock galaxy samples, the EL-COSMOS and Euclid Flagship2 catalogues. The best performance is obtained from either a dense neural network or a support vector classifier, with comparable results in terms of the adopted metrics. When training on on-board photometry alone, these are able to remove $87<!PCT!>$ of the sources that are fainter than the nominal flux limit or lie outside the $0.9<z<1.8$ redshift range, a figure that increases to $97<!PCT!>$ when ground-based photometry is included. These results show how by using the photometric information available to it will be possible to efficiently identify and discard spurious interlopers, allowing us to build robust spectroscopic samples for cosmological investigations.

Neural networks for rapid phase quantification of cultural heritage X-ray powder diffraction data

Recent developments in synchrotron radiation facilities have increased the amount of data generated during acquisitions considerably, requiring fast and efficient data processing techniques. Here, the application of dense neural networks (DNNs) to data treatment of X-ray diffraction computed tomography (XRD-CT) experiments is presented. Processing involves mapping the phases in a tomographic slice by predicting the phase fraction in each individual pixel. DNNs were trained on sets of calculated XRD patterns generated using a Python algorithm developed in-house. An initial Rietveld refinement of the tomographic slice sum pattern provides additional information (peak widths and integrated intensities for each phase) to improve the generation of simulated patterns and make them closer to real data. A grid search was used to optimize the network architecture and demonstrated that a single fully connected dense layer was sufficient to accurately determine phase proportions. This DNN was used on the XRD-CT acquisition of a mock-up and a historical sample of highly heterogeneous multi-layered decoration of a late medieval statue, called `applied brocade'. The phase maps predicted by the DNN were in good agreement with other methods, such as non-negative matrix factorization and serial Rietveld refinements performed with TOPAS, and outperformed them in terms of speed and efficiency. The method was evaluated by regenerating experimental patterns from predictions and using the R-weighted profile as the agreement factor. This assessment allowed us to confirm the accuracy of the results.

Advanced Machine Learning Techniques for Corrosion Rate Estimation and Prediction in Industrial Cooling Water Pipelines.

This paper presents the results of a study on data preprocessing and modeling for predicting corrosion in water pipelines of a steel industrial plant. The use case is a cooling circuit consisting of both direct and indirect cooling. In the direct cooling circuit, water comes into direct contact with the product, whereas in the indirect one, it does not. In this study, advanced machine learning techniques, such as extreme gradient boosting and deep neural networks, have been employed for two distinct applications. Firstly, a virtual sensor was created to estimate the corrosion rate based on influencing process variables, such as pH and temperature. Secondly, a predictive tool was designed to foresee the future evolution of the corrosion rate, considering past values of both influencing variables and the corrosion rate. The results show that the most suitable algorithm for the virtual sensor approach is the dense neural network, with MAPE values of (25 ± 4)% and (11 ± 4)% for the direct and indirect circuits, respectively. In contrast, different results are obtained for the two circuits when following the predictive tool approach. For the primary circuit, the convolutional neural network yields the best results, with MAPE = 4% on the testing set, whereas for the secondary circuit, the LSTM recurrent network shows the highest prediction accuracy, with MAPE = 9%. In general, models employing temporal windows have emerged as more suitable for corrosion prediction, with model performance significantly improving with a larger dataset.

NEural Engine for Discovering Luminous Events (NEEDLE): identifying rare transient candidates in real time from host galaxy images

ABSTRACT Known for their efficiency in analysing large data sets, machine learning-based classifiers have been widely used in wide-field sky survey pipelines. The upcoming Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will generate millions of real-time alerts every night, enabling the discovery of large samples of rare events. Identifying such objects soon after explosion will be essential to study their evolution. Using ∼5400 transients from the Zwicky Transient Facility (ZTF) Bright Transient Survey as training and test data, we develop NEEDLE (NEural Engine for Discovering Luminous Events), a novel hybrid (convolutional neural network + dense neural network) classifier to select for two rare classes with strong environmental preferences: superluminous supernovae (SLSNe) preferring dwarf galaxies, and tidal disruption events (TDEs) occurring in the centres of nucleated galaxies. The input data includes (i) cutouts of the detection and reference images, (ii) photometric information contained directly in the alert packets, and (iii) host galaxy magnitudes from Pan-STARRS (Panoramic Survey Telescope and Rapid Response System). Despite having only a few tens of examples of the rare classes, our average (best) completeness on an unseen test set reaches 73 per cent (86 per cent) for SLSNe and 80 per cent (87 per cent) for TDEs. While very encouraging for completeness, this may still result in relatively low purity for the rare transients, given the large class imbalance in real surveys. However, the goal of NEEDLE is to find good candidates for spectroscopic classification, rather than to select pure photometric samples. Our system will be deployed as an annotator on the UK alert broker, Lasair, to provide predictions of real-time alerts from ZTF and LSST to the community.