Neural Network Library Research Articles

Deep neural networks integrating genomics and histopathological images for predicting stages and survival time-to-event in colon cancer.

There exists an unexplained diverse variation within the predefined colon cancer stages using only features from either genomics or histopathological whole slide images as prognostic factors. Unraveling this variation will bring about improved staging and treatment outcomes. Hence, motivated by the advancement of Deep Neural Network (DNN) libraries and complementary factors within some genomics datasets, we aggregate atypia patterns in histopathological images with diverse carcinogenic expression from mRNA, miRNA and DNA methylation as an integrative input source into a deep neural network for colon cancer stages classification, and samples stratification into low or high-risk survival groups. The genomics-only and integrated input features return Area Under Curve-Receiver Operating Characteristic curve (AUC-ROC) of 0.97 compared with AUC-ROC of 0.78 obtained when only image features are used for the stage's classification. A further analysis of prediction accuracy using the confusion matrix shows that the integrated features have a weakly improved accuracy of 0.08% more than the accuracy obtained with genomics features. Also, the extracted features were used to split the patients into low or high-risk survival groups. Among the 2,700 fused features, 1,836 (68%) features showed statistically significant survival probability differences in aggregating samples into either low or high between the two risk survival groups. Availability and Implementation: https://github.com/Ogundipe-L/EDCNN.

ПРИМЕНЕНИЕ НЕЙРОННЫХ СЕТЕЙ ДЛЯ АНАЛИЗА ТЕХНИЧЕСКИХ ДАННЫХ

The article discusses the application of neural networks using Keras, which is an open neural network library written in the Python language. This library is aimed at operational work with deep learning net-works, while being designed to be compact, modular and extensible

Detection and Classification of Fabric Defects Using Deep Learning Algorithms

The textile industry primarily relies on fabric as a crucial raw material, the production of which involves multiple complex stages. Due to the multitude and complexity of these stages, fabric defects can frequently occur. With the modern fabric production process being nearly fully automated, and given the variety of potential defects, detecting errors on fabrics has become increasingly challenging. The rapid pace of production and the substantial market share of the sector mean that relying on human inspection for error detection can lead to significant time losses and can reduce the accuracy of defect detection to around 60%. Consequently, recent years have seen a shift towards the development of intelligent systems for fabric defect detection in parallel with technological advancements. With the rapid progression of artificial intelligence, the application of image processing techniques has commenced in this field. This study has developed a real-time defect detection system for fabrics using deep learning techniques. Initially, a network model was created using an open-source neural network library, CNN, achieving 89% accuracy. Subsequent implementations using the VGG16 and InceptionV3 architectures reached accuracies of 89% and 86%, respectively. To further improve the study, fabrics were classified into two categories: defective and non-defective, and the pre-trained Convolutional Neural Networks model ResNet50-v2 was employed as a feature extractor. This approach yielded an approximate accuracy of 95%.

Particle Size Analysis of Nanopowders Using Neural Networks and Electron Microscopy

The purpose of the research is to develop an application capable of automatically determining the particle size distribution of nanopowder using neural network technology in order to simplify the process of preparing documentation during its manufacture.Methods. To determine the physical properties of nanopowders during their fabrication, it is necessary to analyze the particle size distribution. A methodology for determining the size distribution of nanopowder particles based on light neural networks is proposed. Images obtained by electron microscopy are used for processing, which allows to speed up the preparation of manufactured powders for sale. The dataset collected for training contains real images of samples of different powders, augmented data and generated images. The Python language, LabVIEW graphical programming environment, YOLO convolutional neural network and various Python language libraries were used in the development.Results. The study resulted in a model trained on the collected dataset that is capable of recognizing particles in images. A software interface was created to work with the model to analyze nanopowder samples.Conclusion. The developed application allows to automatically determine the size of each powder particle on the basis of the obtained images, as well as to build graphs of their size distribution. This greatly simplifies the work of nanopowder producers and facilitates the preparation of the necessary documentation for the produced product.

Intelligent photovoltaic system to maximize the capture of solar energy

The large consumption of electricity worldwide has an impact on the environment which can be said to alter climate change, the degradation of the ozone layer and acid rain. A house has an average daily consumption of 270kWh, this is why solar panels are very useful and can help to have a renewable energy at low costs. To create an intelligent photovoltaic system, different electronic sensors can be applied to follow the sunlight through a series of instructions in some programming software. The article proposes to prototype an intelligent photovoltaic system, based on artificial intelligence with a neural network library "propet" having a positive impact on the optimization of power generation by allowing a more accurate tracking of the sun and a greater collection of photovoltaic energy throughout the day. performing an integration between Arduino and machine learning algorithms such as artificial neural networks in prediction of time series. Different practical experiments were performed to illustrate the effectiveness of the proposed method.

Development of an ANN Model for RGB Color Classification using the Dataset Extracted from a Fabricated Colorimeter

Codes of red, green, and blue data (RGB) extracted from a lab-fabricated colorimeter device were used to build a proposed classifier with the objective of classifying colors of objects based on defined categories of fundamental colors. Primary, secondary, and tertiary colors namely red, green, orange, yellow, pink, purple, blue, brown, grey, white, and black, were employed in machine learning (ML) by applying an artificial neural network (ANN) algorithm using Python. The classifier, which was based on the ANN algorithm, required a definition of the mentioned eleven colors in the form of RGB codes in order to acquire the capability of classification. The software's capacity to forecast the color of the code that belongs to an object under detection is one of the results of the proposed classifier. The work demanded the collection of about 5000 color codes which in turn were subjected to algorithms for training and testing. The open-source platform TensorFlow for ML and the open-source neural network library Keras were used to construct the algorithm for the study. The results showed an acceptable efficiency of the built classifier represented by an accuracy of 90% which can be considered applicable, especially after some improvements in the future to makes it more effective as a trusted colorimeter.

Modeling and optimization of strategies for making individual decisions in multi-agent socio-economic systems with the use of machine learning

This article presents a new approach to modeling and optimizing individual decision-making strategies in multi-agent socio-economic systems (MSES). This approach is based on the synthesis of agent-based modeling methods, machine learning and genetic optimization algorithms. A procedure for the synthesis and training of artificial neural networks (ANNs) that simulate the functionality of MSES and provide an approximation of the values of its objective characteristics has been developed. The feature of the two-step procedure is the combined use of particle swarm optimization methods (to determine the optimal values of hyperparameters) and the Adam machine learning algorithm (to compute weight coefficients of the ANN). The use of such ANN-based surrogate models in parallel multi-agent real-coded genetic algorithms (MA-RCGA) makes it possible to raise substantially the time-efficiency of the evolutionary search for optimal solutions. We have conducted numerical experiments that confirm a significant improvement in the performance of MA-RCGA, which periodically uses the ANN-based surrogate-model to approximate the values of the objective and fitness functions. A software framework has been designed that consists of the original (reference) agent-based model of trade interactions, the ANN-based surrogate model and the MA-RCGA genetic algorithm. At the same time, the software libraries FLAME GPU, OpenNN (Open Neural Networks Library), etc., agent-based modeling and machine learning methods are used. The system we developed can be used by responsible managers.

Enhanced Recurrent Neural Network (RNN) For Heart Disease Risk Prediction Using Framingham Datasets

Heart disease is one of the leading causes of death globally, which takes 17.9 million lives each year. The existing heart disease prediction techniques have a gap that does not consider the smoking attributes from the heart disease data. So, the accuracy is based on the limited number of medical data and the deep learning model. The existing deep learning models which use the Recurrent Neural Network (RNN) for heart disease prediction consume more processing and analysing time, mainly due to the delay of data retrieval. This delay causes the prediction process to become slower and leads to a moderate prediction only. The backpropagation of the RNN with an update gate and internal memory to carry the updated data cause a minor data glitch that leads to lower accuracy. Therefore, an efficient heart disease prediction model is very crucial to provide early detection among patients. This research proposes a Heart Disease Risk Prediction Model (HDRPM) with an enhanced RNN to improve the prediction accuracy using Framingham heart disease datasets. The specificity and sensitivity are imposed to improve the quality of the predictions. Sensitivity measure is used for detecting patients with heart disease perfectly and specificity measure is used for detecting patients without the disease perfectly. Besides the accuracy and quality of the prediction problem, the imbalance of minority classes in the dataset occurred in most deep learning prediction fields. This research aims to improve the quality of imbalanced Framingham datasets using Synthetic Minority Over-sampling Technique (SMOTe), which will synthetic instances in a small class to be equalized. The existing RNN model faces vanishing gradients that impede the learning of long data sequences. These gradients that carry information in the RNN cells will become smaller gradually till it minimises the parameter updates and leads to poor learning. For this purpose, the presence of multiple Gated Recurrent Unit (GRU) is used to overcome the vanishing gradients and ensure the hidden layers. The neurons of RNN rapidly cater for the essential information during the training and validation phase of the HDRPM. The integration of multiple GRU with the RNN, operating on the Tensorflow as back-end and Keras as the core for the neural network library has increased the performance of the proposed model. The proposed model provides up to 98.78%, the highest accuracy achieved compared to related previous work, which is a quantum neural network model with 98.57. This HDRPM is expected to significantly contribute to early detection of heart disease patients.

Improving Credit Risk Assessment through Deep Learning-based Consumer Loan Default Prediction Model

This study aims to enhance credit risk identification, improve loan borrower review efficiency, and increase default prediction accuracy rate using data mining and machine learning techniques. The study also employs deep learning to develop a consumer loan default prediction model that minimizes credit risks and ensures consistent development. The researchers collected data from a survey of 1000 participants, stratified into local and foreign banks, and selected the top 11 banks based on turnover and customer volume. To construct the machine learning model, Keras, a neural network library that runs on TensorFlow, was utilized. The model predicts loan applicant default likelihood. The study's practical implications demonstrate a noteworthy success rate of customer default prediction, which can significantly benefit banks. The model was evaluated on a test set of 250 records and achieved a test set accuracy of 95.2%, correctly predicting the default state of 238 out of 250 respondents.

ОЦЕНКА ПРЕДИКТОРНОЙ ВОЗМОЖНОСТИ ИСКУССТВЕННО ИНТЕЛЛЕКТУАЛЬНЫХ МЕТОДОВ В ПРОГНОЗИРОВАНИИ КАРДИОВАСКУЛЯРНЫХ СОБЫТИЙ

Introduction. Predicting the risk of cardiovascular diseases and adverse outcomes are a promising direction in medicine. Using machine learning methods to process large databases to identify new predictors and establish more complex and deeper interactions between them creates other predictive capabilities in a personalized approach to risk assessment. The new approach to improving forecasting is the ability to use artificial intelligence techniques by creating a combination of sophisticated mathematical models and algorithms under certain conditions of computational power and improved database quality. Aim. To estimate the accuracy and reliability of models based on machine learning for solving problems of cardiovascular disease risk prediction. Materials and Methods. Data from a one-stage anonymous survey of volunteers in Primorsky Krai were collected in a multicenter observational study «Epidemiology of Cardiovascular Diseases in the Regions of the Russian Federation». A total of 2 131 participants took part in the study. The main and additional predictors of cardiovascular diseases were selected for further analysis and inclusion in prediction models. Models were created and evaluated using statistical and neural network libraries of the Python programming language. The quality of the models was determined by calculating the total area under the receiver operating characteristic curve. Results and discussion. Compared to the established risk prediction algorithm by calculating the area under the receiver operating characteristic curve, machine-learning algorithms improved the prediction: random forest +1.7%, logistic regression +3.2%, neural networks +3.6%. The algorithm with the highest performance (neural networks) predicted 320 cases (sensitivity 70.2%) and 1217 non-cases (specificity 74.7%), correctly predicting 248 (+7.6%) more patients who developed cardiovascular disease compared to the established absolute cardiovascular risk calculation scale Systematic Coronary Risk Evaluation. Conclusion. The use of neural networks with multilayer perceptron, as one of the methods of layer-by-layer construction of machine learning algorithm, is the best option for creating a prognostic model and currently provides the most reliable results.

Speedup deep learning models on GPU by taking advantage of efficient unstructured pruning and bit-width reduction

This work is focused on the UnSparse-Opt framework for the efficient unstructured pruning and quantisation of feedforward neural networks and the improvement of their efficiency on graphic processing units (GPU) by using a direct sparse algorithm. The Nvidia deep neural network (cuDnn) library is the most effective implementation of deep learning (DL) algorithms for GPUs. One of the most common techniques for improving the efficiency of Convolutional Neural Network (CNN) models is weight pruning and quantisation. There are two main types of pruning: structural and non-structural. The first enables much easier acceleration on many type of accelerators, but with structural it is difficult to achieve a sparsity level and accuracy as high as that obtained with the non-structural version. Non-structural pruning with retraining can generate the weight tensors up to ∼90% or more of sparsity in some deep CNN models. In this article, the pruning algorithm is presented which achieve high sparsity levels without drop in accuracy. In the next stage, the linear and non-linear quantisation is adapted for further reductions in time and memory footprint. Additionally, this work presents real CNN models pruned with high sparsities in which some subset of layers can have comparable or better efficiency than cuDnn by using a direct sparse method. Finally, it shows sparse CNN-based architectures with reduced precision which can be more efficient than CuDnn library.

Underground Garage Patrol Based on Road Marking Recognition by Keras and Tensorflow

The purpose of this study was to design an unmanned patrol service in combination with artificial intelligence technology to solve the problem of underground vehicle patrol. This design used the Raspberry Pi development board, L298N driver chip, Raspberry Pi camera, and other major hardware equipment to transform the remote control car. This design used Python as the programming language. By writing Python code, the car could be driven under the control of the computer keyboard and the camera was turned on for data collection. The Keras neural network library was used to quickly build a neural network model, the collected data was used to train the model, and the model was finally generated. The model was placed in the TensorFlow system for processing, and the car could travel in a preset track for unmanned driving.

Creation of a library of images of cable line insulation breakdown for neural network training based on data from the study of wave processes in the digital twin of the cable line

The aim of the work is to create a library of images of cable line insulation breakdown for training a neural network of a functional unit integrated into a high voltage test facility based on data from the study of wave processes in the digital twin of a cable line. The goal is achieved by determining the characteristic properties of the wave process in a cable line when tested with high direct voltage of negative polarity during field experiments and verifying these results on a mathematical model. The calculated data array on the verified model is the main volume of primary information for creating a library of images of insulation breakdown. The most important and significant results are the creation of a library of images of insulation breakdown and the adaptation of the algorithm for approximating input data with images from the neural network library, the integration of artificial intelligence into the process of determining the location of defective insulation at the test stage. The significance of the results is to reduce the methodological and instrumental error in determining the damage zone, since all stages of data collection and analysis are performed by a functional module built into the test unit, in addition, the calculation is performed automatically, which minimizes the role of anthropogenic factors on the result and reduces the requirements for the personnel of the electrical laboratory, the introduction of such equipment will reduce complete the time of elimination of the accident and increase the speed of restoration work on the route of the line.

Artificial intelligence-enabled rapid and symptom-based medication recommendation system (COV-MED) for the COVID-19 patients

In a general COVID-19 population in Cox’s Bazar, Bangladesh, we developed a medication recommendation system based on clinical information from the electronic medical record (EMR). Our goal was also to enable deep learning (DL) strategies to quickly assist physicians and COVID-19 patients by recommending necessary medications. The general demographic data, clinical symptoms, basic clinical tests, and drug information of 8953 patients were used to create a dataset. The learning model in this COVID-MED model was created using Keras (an open-source artificial neural network library) to solve regression problems. In this study, a sequential model was adopted. In order to improve the prediction capability and achieve global minima quickly and smoothly, the COVID-MED model incorporates an adaptive optimizer dubbed Adam. The model calculated a mean absolute error of 0.0037, a mean squared error of 0.000035, and a root mean squared error of 0.0059. The model predicts the output medications, such as injections or other oral medications, with around 99% accuracy. These findings show that medication can be predicted using information from the EMR. Similar models allow for patient-specific decision support to help prevent medication errors in diseases other than COVID-19.

Patient Prediction Through Convolutional Neural Networks

Abstract This paper presents a methodology for predicting the lung diseases of patients through medical images using the Convolutional neural network (CNN). The importance of this work comes from the current SARS-CoV-2 pandemic simulation where with the presented method in this work, pneumonia infection from healthy situation can be diagnosed using the X-ray images. For validating the presented method, various X-ray images are employed in the Python coding environment where various libraries are used: TensorFlow for tensor operations, Scikit-learn for machine learning (ML), Keras for artificial neural network (ANN), matplotlib and seaborn libraries to perform exploratory data analysis on the data set and to evaluate the results visually. The practical simulation results reveal 91% accuracy, 90% precision, and 96% sensitivity making prediction between diseases.

Analysis of the Application Efficiency of TensorFlow and PyTorch in Convolutional Neural Network.

In this paper, we present an analysis of important aspects that arise during the development of neural network applications. Our aim is to determine if the choice of library can impact the system's overall performance, either during training or design, and to extract a set of criteria that could be used to highlight the advantages and disadvantages of each library under consideration. To do so, we first extracted the previously mentioned aspects by comparing two of the most popular neural network libraries-PyTorch and TensorFlow-and then we performed an analysis on the obtained results, with the intent of determining if our initial hypothesis was correct. In the end, the results of the analysis are gathered, and an overall picture of what tasks are better suited for what library is presented.

Forecasting of Total Column Ozone using Regression Analysis and LSTM-RNN Machine Learning Approach

Objectives: Atmospheric Ozone plays an important role in global climate change, human health and environmental conditions. Accurate and timely prediction of variation in Total Column Ozone (TCO) concentration is very important for both climatology and environment. In this context, the present study aims to utilize the advancement of Artificial Intelligence in Machine learning and a conventional method to develop models for the prediction of TCO concentration using historical time series data over a tropical region, India. Methods: Long Short-Term Memory (LSTM) deep learning networks are very useful in classifying, processing and making predictions based on time series data. In this work, Multiple Linear Regression (MLR) model, Long Short-Term Memory (LSTM) models are developed for forecasting TCO over a tropical region. The Statistical Parameters such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) were used to analyze and evaluate the performance of the proposed models. Findings: In the present study the TCO concentration varied between 217.4 DU and 288.6 DU. The performances of MLR and LSTM model were compared. From the results, it is evident that the values of RMSE, MAE and MAPE of the MLR model are 3.807924, 2.93766, 1.15 respectively whereas the values for LSTM model are 3.074492, 2.574, 1.01 respectively. LSTM was found to be the accurate model for the study data set and the predicted TCO concentration has a very good correlation with the actual observations and it does not rely on the meteorological parameters. Novelty: The LSTM models are usually employed in the field of machine translation, speech recognition and for the prediction of traffic levels, stock levels and air pollutants. This work utilizes the fast numerical machine learning computing library Tensorflow and a high level neural network library Keras that runs on top of Tensorflow to develop a LSTM network model for the prediction of the concentration levels of TCO. Keywords: Total Column Ozone; Regression; Forecasting; Climate Change; Long Short-Term Memory

The applications of deep learning algorithms on in silico druggable proteins identification

IntroductionThe top priority in drug development is to identify novel and effective drug targets. In vitro assays are frequently used for this purpose; however, traditional experimental approaches are insufficient for large-scale exploration of novel drug targets, as they are expensive, time-consuming and laborious. Therefore, computational methods have emerged in recent decades as an alternative to aid experimental drug discovery studies by developing sophisticated predictive models to estimate unknown drugs/compounds and their targets. The recent success of deep learning (DL) techniques in machine learning and artificial intelligence has further attracted a great deal of attention in the biomedicine field, including computational drug discovery. ObjectivesThis study focuses on the practical applications of deep learning algorithms for predicting druggable proteins and proposes a powerful predictor for fast and accurate identification of potential drug targets. MethodsUsing a gold-standard dataset, we explored several typical protein features and different deep learning algorithms and evaluated their performance in a comprehensive way. We provide an overview of the entire experimental process, including protein features and descriptors, neural network architectures, libraries and toolkits for deep learning modelling, performance evaluation metrics, model interpretation and visualization. ResultsExperimental results show that the hybrid model (architecture: CNN-RNN (BiLSTM) + DNN; feature: dictionary encoding + DC_TC_CTD) performed better than the other models on the benchmark dataset. This hybrid model was able to achieve 90.0% accuracy and 0.800 MCC on the test dataset and 84.8% and 0.703 on a nonredundant independent test dataset, which is comparable to those of existing methods. ConclusionWe developed the first deep learning-based classifier for fast and accurate identification of potential druggable proteins. We hope that this study will be helpful for future researchers who would like to use deep learning techniques to develop relevant predictive models.

Neural Network Assisted Pathology Case Identification

BackgroundTraditionally, cases for cohort selection and quality assurance purposes are identified through structured query language (SQL) searches matching specific keywords. Recently, several neural network-based natural language processing (NLP) pipelines have emerged as an accurate alternative/complementary method for case retrieval. MethodsThe diagnosis section of 1000 pathology reports with the terms “colon” and “carcinoma” were retrieved from our laboratory information system through a SQL query. Each of the reports were labeled as either positive or negative, where cases are considered positive if the case was a primary adenocarcinoma of the colon. Negative cases comprised adenocarcinoma from other sites, metastatic adenocarcinomas, benign conditions, rectal cancers, and other cases that do not fit in the primary colonic adenocarcinoma category. The 1000 cases were randomly separated into training, validation, and holdout sets. A convolutional neural network (CNN) model built using Keras (a neural network library) was trained to identify positive cases, and the model was applied to the holdout set to predict the category for each case. ResultsThe CNN model classified 141 out of 149 primary colonic adenocarcinoma cases, and 43 out of 51 negative cases correctly, achieving an accuracy of 92% and area under the ROC curve (AUC) of 0.957. ConclusionTrained convolutional neural network models by itself, or as an adjunct to keyword and pattern-based text extraction methods may be used to search for pathology cases of interest with high accuracy.

Modelling an Adaptive Learning System Using Artificial Intelligence

The goal of this paper is to use artificial intelligence to build and evaluate an adaptive learning system where we adopt the basic approaches of spiking neural networks as well as artificial neural networks. Spiking neural networks receive increasing attention due to their advantages over traditional artificial neural networks. They have proven to be energy efficient, biological plausible, and up to 105 times faster if they are simulated on analogue traditional learning systems. Artificial neural network libraries use computational graphs as a pervasive representation, however, spiking models remain heterogeneous and difficult to train. Using the artificial intelligence deductive method, the paper posits two hypotheses that examines whether 1) there exists a common representation for both neural networks paradigms for tutorial mentoring, and whether 2) spiking and non-spiking models can learn a simple recognition task for learning activities for adaptive learning. The first hypothesis is confirmed by specifying and implementing a domain-specific language that generates semantically similar spiking and non-spiking neural networks for tutorial mentoring. Through three classification experiments, the second hypothesis is shown to hold for non-spiking models, but cannot be proven for the spiking models. The paper contributes three findings: 1) a domain-specific language for modelling neural network topologies in adaptive tutorial mentoring for students, 2) a preliminary model for generalizable learning through back-propagation in spiking neural networks for learning activities for students also represented in results section, and 3) a method for transferring optimised non-spiking parameters to spiking neural networks has also been developed for adaptive learning system. The latter contribution is promising because the vast machine learning literature can spill-over to the emerging field of spiking neural networks and adaptive learning computing. Future work includes improving the back-propagation model, exploring time-dependent models for learning, and adding support for adaptive learning systems.