Neural Deep Learning Research Articles

Artificial Intelligence, Machine Learning, and Intelligent Decision Support Systems: Iterative “Learning” SQGbased procedures for Distributed Models’ Linkage

In this paper we discuss the on-going joint work contributing to the IIASA (International Institute for Applied Systems Analysis, Laxenburg, Austria) and National Academy of Science of Ukraine projects on “Modeling and management of dynamic stochastic interdependent systems for food-water-energy-health security nexus” (see [1-2] and references therein). The project develops methodological and modeling tools aiming to create Intelligent multimodel Decision Support System (IDSS) and Platform (IDSP), which can integrate national Food, Water, Energy, Social models with the models operating at the global scale (e.g., IIASA GLOBIOM and MESSAGE), in some cases ‘downscaling’ the results of the latter to a national level. Data harmonization procedures rely on new type non-smooth stochastic optimization and stochastic quasigradient (SQG) [3-4] methods for robust of-line and on-line decisions involving large-scale machine learning and Artificial Intelligence (AI) problems in particular, Deep Learning (DL) including deep neural learning or deep artificial neural network (ANN). Among the methodological aims of the project is the development of “Models’ Linkage” algorithms which are in the core of the IDSS as they enable distributed models’ linkage and data integration into one system on a platform [5-8]. The linkage algorithms solve the problem of linking distributed models, e.g., sectorial and/or regional, into an inter-sectorial inter-regional integrated models. The linkage problem can be viewed as a general endogenous reinforced learning problem of how software agents (models) take decisions in order to maximize the “cumulative reward". Based on novel ideas of systems’ linkage under asymmetric information and other uncertainties, nested strategic-operational and local-global models are being developed and used in combination with, in general, non-Bayesian probabilistic downscaling procedures. In this paper we illustrate the importance of the iterative “learning” solution algorithms based on stochastic quasigradient (SQG) procedures for robust of-line and on-line decisions involving large-scale Machine Learning, Big Data analysis, Distributed Models Linkage, and robust decision-making problems. Advanced robust statistical analysis and machine learning models of, in general, nonstationary stochastic optimization allow to account for potential distributional shifts, heavy tails, and nonstationarities in data streams that can mislead traditional statistical and machine learning models, in particular, deep neural learning or deep artificial neural network (ANN). Proposed models and methods rely on probabilistic and non-probabilistic (explicitly given or simulated) distributions combining measures of chances, experts’ beliefs and similarity measures (for example, compressed form of the kernel estimators). For highly nonconvex models such as the deep ANN network, the SQGs allow to avoid local solutions. In cases of nonstationary data, the SQGs allow for sequential revisions and adaptation of parameters to the changing environment, possibly, based on of-line adaptive simulations. The non-smooth STO approaches and SQG-based iterative solution procedures are illustrated with examples of robust estimation, models’ linkage, machine learning, adaptive Monte Carlo optimization for cat risks (floods, earthquakes, etc.) modeling and management

Shift invariant deep convolution neural learning for resource efficient healthcare data transmission in WSN

Wireless Sensor Network (WSN) comprises many small sensors nodes incorporated by IoT which plays vital role in several applications. IoT incorporates physical devices into a network structure that includes software, sensors to collect data from the surrounding environment, and sinks for data transmission. To increase network lifespan during data transmission, a resource-efficient routing system is needed. Different academics have been working on improving routing techniques recently to increase effectiveness and route discovery. The key issue affecting the performance of WSNs is an efficient resource usage of routing. In order to overcome this problem, a new technique called Hierarchical Point wise Mutual Informative Clustering-based Shift Invariant Deep Convolutive Neural Learning (HPMIC-SIDCNL) technique is introduced in WSN. The HPMIC-SIDCNL technique uses the Shift Invariant Convolutive Deep neural learning concept to learn the given input with help of several layers such as input, three hidden layers, and output layer. Initially, the IoT devices are used in sensor nodes for sensing and collecting patient data. In the input layer, sensor nodes are considered as input. Then input is transmitted into first hidden layer where resource of the sensor nodes such as energy and bandwidth are measured. Then estimated resources are transmitted into second hidden layer where the clustering process is carried out using the Hierarchical IBM Point wise Mutual Informative Clustering technique. The Hierarchical IBM clustering uses the Point wise Mutual Information for collection of sensor nodes depend on energy and bandwidth. In the third hidden layer, the cluster is selected on higher residual energy and minimum bandwidth for improving the data transmission and reduces the delay. The cluster head receives the patient data after it has been sent by the source node in that particular cluster. In order to perform data transmission from source to sink node at output layer, cluster then locates the closest cluster head. In this manner, cluster heads are used to execute resource-efficient healthcare data transmission from source to sink node, minimizing the latency. The obtained findings show that the suggested HPMIC-SIDCNL technique outperforms in terms of high delivery ratio with little packet loss and delay. The advantage of the proposed HPMIC-SIDCNL technique, it is simple to understand because it represents a solution to a given problem step by step and is not dependent on any programming language. Since everyone can understand it, even those without programming experience, it is simple.

Explainable AI in Scene Understanding for Autonomous Vehicles in Unstructured Traffic Environments on Indian Roads Using the Inception U-Net Model with Grad-CAM Visualization

The intelligent transportation system, especially autonomous vehicles, has seen a lot of interest among researchers owing to the tremendous work in modern artificial intelligence (AI) techniques, especially deep neural learning. As a result of increased road accidents over the last few decades, significant industries are moving to design and develop autonomous vehicles. Understanding the surrounding environment is essential for understanding the behavior of nearby vehicles to enable the safe navigation of autonomous vehicles in crowded traffic environments. Several datasets are available for autonomous vehicles focusing only on structured driving environments. To develop an intelligent vehicle that drives in real-world traffic environments, which are unstructured by nature, there should be an availability of a dataset for an autonomous vehicle that focuses on unstructured traffic environments. Indian Driving Lite dataset (IDD-Lite), focused on an unstructured driving environment, was released as an online competition in NCPPRIPG 2019. This study proposed an explainable inception-based U-Net model with Grad-CAM visualization for semantic segmentation that combines an inception-based module as an encoder for automatic extraction of features and passes to a decoder for the reconstruction of the segmentation feature map. The black-box nature of deep neural networks failed to build trust within consumers. Grad-CAM is used to interpret the deep-learning-based inception U-Net model to increase consumer trust. The proposed inception U-net with Grad-CAM model achieves 0.622 intersection over union (IoU) on the Indian Driving Dataset (IDD-Lite), outperforming the state-of-the-art (SOTA) deep neural-network-based segmentation models.

NeuroCyberGuard: Developing a Robust Cybersecurity Defense System through Deep Neural Learning-Based Mathematical Modeling

Abstract Due to the increasing sophistication and frequency of cyber threats, cyber security has become a pressing issue in today’s interconnected society. The traditional security methods have proven insufficient to tackle the evolving threat landscape, leading to the need for more advanced and effective defence mechanisms. The emerging field of deep neural learning has shown promise in cybersecurity, as it can learn complex patterns and behaviors in data and provide more accurate and efficient detection and prediction capabilities. This research aims to develop a robust cybersecurity defense system, named NeuroCyberGuard, by using deep neural learning-based mathematical modelling. The proposed approach leverages the power of deep neural networks to analyze large amounts of network traffic data, identify potential threats, and provide real-time alerts and responses to prevent security breaches. The mathematical modelling component enables the system to optimize its performance and adapt to changing network conditions and attack patterns. The system will be evaluated using a variety of benchmark datasets and real-world scenarios to validate its effectiveness and efficiency. The expected outcome of this research is a state-of-the-art cybersecurity defense system that provides comprehensive protection against cyber threats and enhances the security posture of organizations.

Coffee Leaf Diseases Classification and the Effect of Fine-tuning on Deep Convolutional Neural Networks

This research proposes a method for the automatic diagnosis and classification of leaf diseases in Kenyan Arabica coffee leaves. We trained Deep Convolution Neural Learning models on the JMuBEN2 obtained from Mendeley data public access to determine whether a particular leaf image contains Phoma, Cercospora, or Rust. The proposed models for this work were the ResNet50, Densenet-121, and VGG19 architectures, all are well-known classification models. They were trained using transfer learning and fine-tuning and their respective outputs were compared based on these methods of training. After training the dataset using the aforementioned models, the Densenet-121 model was superior to the others gaining an accuracy of 95.44% after transfer learning and 99.36% after fine-tuning the model.

Deep neural learning based optimization for automated high performance antenna designs

The present paper introduces an optimization-oriented method here practiced for designing high performance single antennas in a fully automated environment. The proposed method comprises two sequential major steps. The first one devotes configuring the shape of antenna and determining the feeding point by employing the bottom-up optimization (BUO) method. In this algorithm, the number of microstrip transmission lines (TLs) used to model the radiator is increased consecutively and the shape of the antenna is revised up to finding the initial satisfying results. Secondly, for determining the best design parameters of the configured antenna shape in the first step (i.e., width and length of TLs), deep neural network (DNN) that is based on Thompson sampling efficient multi-objective optimization (TSEMO) is applied. The recommended optimization method is successfully attracted as a problem solver for designers to tackle the subject for antenna design such as the complexity and large dimensions of structures. Hence, the main advantage of the implemented optimization method in this article is to noticeably decrease the required designer’s involvement automatically generating valid layouts. For validating the suggested method, two wideband antennas are designed, prototyped and subjected to experiment. The first optimized antenna covers the frequency band 8.8–10.1 GHz (13.75 % bandwidth) characterized by a maximum gain of 7.13 dB while the second one covers the frequency band 11.3–13.16 GHz (15.2 %) which exhibits a maximum gain of 7.8 dB.

Tversky Similarity based Deep Neural Learning Classification for Engineering Alloys

Integrated Computational Materials Engineering (ICME) is an environment friendly technique used for performing cloth discovery and design. Computational methods introduced a new deep studying classification approach to display screen the candidate cloth designs. During the product designing stage, the ingredients are customised to meet particular needs. In ICME processes, there is always a degree of uncertainty in the process, structure, and property components. Uncertainties may be quantified, reduced, and propagated via structure–property links using the Tversky Similarity based Deep Neural Learning Classification (TS-DNLC) Method. In TS-DNLC Method, number of compound data are considered as input and given to the input layer. An input compound data is given to hidden layer 1. In that layer, regression is employed for performing the compound data analysis with structure–property linkages. After that, the regression coefficient value is sent to the hidden layer 2. In that layer, Tversky similarity function is used to identify the similarity between the regression coefficient value of training compound data and threshold value. Tversky similarity value varies from 0 to 1 and the results are transmitted to the output layer. By this way, TS-DNLC Method improves the performance of structure–property linkages. The computational cost of proposed TS-DNLC Method is higher than conventional uncertainty quantification.

Recurrent deep neural learning classification algorithm based high level synthesis in VLSI circuit with runtime adaptability

Electronics industry has attained huge development in last few decades due to the rapid increase in system design applications. With the growth of very large scale integration (VLSI) design, integrated circuits (ICs) are employed in many applications. VLSI design comprises many steps like system-level design, high-level synthesis (HLS), logic design, test generation, and physical design. HLS interprets behavior description and create digital hardware that executes the behavior. But, the power-process-voltage-temperature (PPVT) variation can causee many issues and reduce the performance of VLSI design circuits. In order to address these problems, Recurrent Deep Neural Learning Classification based High Level Synthesis (RDNLC-HLS) Model is designed for better runtime adaptability with minimal time consumption. VLSI circuits are designed with the behavioral input and the output performance is measured at runtime. The behavioral description of the circuit is taken as input. Then, source code compilation process translates high level specification into Intermediate Representation (IR) and converts to control/data flow graph (CDFG). CDFG reveals data and control dependencies between operations. The proposed Recurrent Deep Neural Learning Classification based High Level Synthesis (RDNLC-HLS) Model is designed for providing better runtime adaptability with minimal time consumption. Finally, Register Transfer Level Generation is carried out to yield better runtime adaptability with minimal time. Simulation results on ISCAS’89 Benchmark Dataset, shows that the RDNLC-HLS model increases the FUSA with minimal error rate and CAT.

Development and Validation of an Insulin Resistance Model for a Population with Chronic Kidney Disease Using a Machine Learning Approach.

Background: Chronic kidney disease (CKD) is a complex syndrome without a definitive treatment. For these patients, insulin resistance (IR) is associated with worse renal and patient outcomes. Until now, no predictive model using machine learning (ML) has been reported on IR in CKD patients. Methods: The CKD population studied was based on results from the National Health and Nutrition Examination Survey (NHANES) of the USA from 1999 to 2012. The homeostasis model assessment of IR (HOMA-IR) was used to assess insulin resistance. We began the model building process via the ML algorithm (random forest (RF), eXtreme Gradient Boosting (XGboost), logistic regression algorithms, and deep neural learning (DNN)). We compared different receiver operating characteristic (ROC) curves from different algorithms. Finally, we used SHAP values (SHapley Additive exPlanations) to explain how the different ML models worked. Results: In this study population, 71,916 participants were enrolled. Finally, we analyzed 1,229 of these participants. Their data were segregated into the IR group (HOMA IR > 3, n = 572) or non-IR group (HOMR IR ≤ 3, n = 657). In the validation group, RF had a higher accuracy (0.77), specificity (0.81), PPV (0.77), and NPV (0.77). In the test group, XGboost had a higher AUC of ROC (0.78). In addition, XGBoost also had a higher accuracy (0.7) and NPV (0.71). RF had a higher accuracy (0.7), specificity (0.78), and PPV (0.7). In the RF algorithm, the body mass index had a much larger impact on IR (0.1654), followed by triglyceride (0.0117), the daily calorie intake (0.0602), blood HDL value (0.0587), and age (0.0446). As for the SHAP value, in the RF algorithm, almost all features were well separated to show a positive or negative association with IR. Conclusion: This was the first study using ML to predict IR in patients with CKD. Our results showed that the RF algorithm had the best AUC of ROC and the best SHAP value differentiation. This was also the first study that included both macronutrients and micronutrients. We concluded that ML algorithms, particularly RF, can help determine risk factors and predict IR in patients with CKD.

Kushner-Stratonovich Dice Segmented Curvelet (KSDSC) deep convolutional neural learning for heart disease prediction

Data mining is commonly used method for processing large amount of data in heart disease prediction. Many heart disease prediction researches are carried out by different authors. But the accuracy level was not improved. In order to address these issues, Kushner-Stratonovich Dice Segmented Haar Wavelet (KSDSC) Deep Convolutional Neural Learning Model is introduced. The main aim of KSDSC Model is to perform efficient heart disease prediction with five layers, namely one input layer, three hidden layers and one output layer. Initially, ultrasound images are collected as an input at input layer. An image pre-processing is carried out using Kushner-Stratonovich Filter to eliminate the noisy pixels from US image and transmitted to hidden layer 2. Sørensen–Dice Image Segmentation Process partitions the preprocessed image into number of divisions in hidden layer 2. After that in hidden layer 3, the multiple features are extracted using Curvelet transform from segmented image and sent to the output layer. The output layer uses softmax yolov4 darknet53 activation function to match extracted features for heart disease prediction. Experimental analysis is performed on parameters such as prediction accuracy, false positive rate, and prediction time with respect to a number of US images.

On some similarities and differences between deep neural networks and kernel learning machines

On some similarities and differences between deep neural networks and kernel learning machines

Survey on reinforcement learning for language processing

In recent years some researchers have explored the use of reinforcement learning (RL) algorithms as key components in the solution of various natural language processing (NLP) tasks. For instance, some of these algorithms leveraging deep neural learning have found their way into conversational systems. This paper reviews the state of the art of RL methods for their possible use for different problems of NLP, focusing primarily on conversational systems, mainly due to their growing relevance. We provide detailed descriptions of the problems as well as discussions of why RL is well-suited to solve them. Also, we analyze the advantages and limitations of these methods. Finally, we elaborate on promising research directions in NLP that might benefit from RL.

Exponential kernelized feature map Theil-Sen regression-based deep belief neural learning classifier for drift detection with data stream

Exponential kernelized feature map Theil-Sen regression-based deep belief neural learning classifier for drift detection with data stream

RETRACTED ARTICLE: Deep convolution neural networks learned image classification for early cancer detection using lightweight

Essentially, cancer refers to the formation of abnormal cells in any section or area of the body. The goal of early cancer detection is to identify patients who are showing signs early on in order to maximise their chances of a successful therapy. Early detection and treatment of cancer reduce the disease mortality. A wide range of image processing and machine learning techniques have been presented for the identification of cancer. There was no improvement in detection accuracy or efficiency with existing systems. To overcome these problems, we present the Least Mean Square Filterative Ricker Wavelet Transform-based Deep Convolutional Neural Learning Classifier Model (L-DCNLC). By using a fully connected max pooling deep convolutional network with higher accuracy and reduced time consumption, the L-DCNLC Model aims to identify cancer early. There are three hidden layers and one output layer in the fully linked max pooling deep convolutional network. The number of patient photographs in the database is used as input in the L-DCNLC Model's input layer. Following that, preprocessing is performed in hidden layer 1 to perform denoising in order to improve image quality. This is accomplished by employing the Least Mean Square Weiner Filtering process, which has a higher peak signal-to-noise ratio. The Continuous Ricker Wavelet Transform is then used to extract the image features from the preprocessed image in hidden layer 2. Finally, the classification process is performed in hidden layer 3 using the Kulczynksi Similarity Coefficient to detect the cancerous image by comparing the testing and extracted features. Thus, the cancerous image is detected and displayed with a low error rate in the output layer. The brain cancer and lung cancer datasets are analysed to determine peak signal to noise ratios (PSNR), cancer detection accuracy, and cancer detection time. The evaluation results indicate that the L-DCNLC Model improves accuracy and PSNR while requiring less computational time than previous works.

Biserial targeted feature projection based radial kernel regressive deep belief neural learning for covid-19 prediction.

Coronavirus disease 2019 (COVID-19) is a highly infectious viral disease caused by the novel SARS-CoV-2 virus. Different prediction techniques have been developed to predict the coronavirus disease’s existence in patients. However, the accurate prediction was not improved and time consumption was not minimized. In order to address these existing problems, a novel technique called Biserial Targeted Feature Projection-based Radial Kernel Regressive Deep Belief Neural Learning (BTFP-RKRDBNL) is introduced to perform accurate disease prediction with lesser time consumption. The BTFP-RKRDBNL techniques perform disease prediction with the help of different layers such as two visible layers namely input and layer and two hidden layers. Initially, the features and data are collected from the dataset and transmitted to the input layer. The Point Biserial Correlative Target feature projection is used to select relevant features and other irrelevant features are removed with minimizing the disease prediction time. Then the relevant features are sent to the hidden layer 2. Next, Radial Kernel Regression is applied to analyze the training features and testing disease features to identify the disease with higher accuracy and a lesser false positive rate. Experimental analysis is planned to measure the prediction accuracy, sensitivity, and specificity, and prediction time for different numbers of patients. The result illustrates that the method increases the prediction accuracy, sensitivity, and specificity by 10, 6, and 21% and reduces the prediction time by 10% as compared to state-of-the-art works.

Electric Profiling Based Fully Recurrent Deep Neural Learning Classification for Groundwater Quality Prediction

Groundwater is present under Earth surface within soil pore spaces and rock formation. It is recharged via surface and typically discharged. Water pollution affects the quality of water and troubles human health, economic growth as well as social wealth. The groundwater quality identification is essential to maintain freshwater resources for sustainable development. But, the existing methods failed to improve the groundwater quality and minimize time consumption. To address these problems, an Electric Profiling Ground Water Identification based Fully Recurrent Deep Neural Learning Classification (EPGWI-FRDNLC) Method is designed to achieve efficient quality analytics by higher accuracy and minimum time consumption. In EPGWI-FRDNLC Method, electric profiling process is carried out for ground water identification. After that, a fully recurrent deep neural learning classification process is carried out for ground water quality prediction analytics. Fully recurrent deep neural learning classification process includes more than three layers for performing the ground water quality analysis. In EPGWI-FRDNLC Method Model, a lot of data were measured for input and given to the input layer. After that, input data were given to hidden layer 1. In that layer, softmax regression is used for performing the input parameter analysis like temperature, pH, turbidity, salinity, nitrates and phosphates. Then, the regression coefficient value is transferred to hidden layer 2. Tanimato similarity function is employed for identifying the similarity between the regression coefficient value of training data and threshold value. Tanimato similarity value ranges from 0 to 1 and the results are sent to the output layer. By this way, EPGWI-FRDNLC Method improves the ground water quality prediction analytics. Experimental evaluation of EPGWI-FRDNLC Method was performed with various metrics by an amount of data.

Aging Facial Recognition for Feature Extraction Using Adaptive Fully Recurrent Deep Neural Learning

Abstract The face recognition (FR) process identifies or confirms a person with the help of images and videos. The traditional aging FR (AFR) method encounters numerous issues as a result of age-related biological transformation. An adaptive fully recurrent deep neural learning (AFRDNL) technique is presented in this research to improve FR accuracy (FRA) with minimal time complexity (TC). Feature extraction and classification are two processes included in the proposed technique. The AFRDNL technique gathers facial images from the database and trains the face images with three layers. To learn the facial features, three hidden layers are employed in the AFRDNL technique. Finally, the AFRDNL technique utilizes the Gaussian activation function for matching extracted testing features with earlier saved training features. This process is repeated until a minimal training error with a higher FRA is achieved. Experimental evaluation is carried out with three image datasets like Face and Gesture Recognition Research Network, MORPH and Cross-Age Labeled faces in the Wild (CALFW) datasets using different metrics, such as FRA, false-positive rate (FPR), TC and F1score. The experimental results confirm that the AFRDNL technique effectively increases the FRA and F1score and minimizes the FPR and TC more effectively than the state-of-the-art methods.

A Machine-Learning Framework to Improve Wi-Fi Based Indoorpositioning

The indoor positioning system comprises portable wireless devices that aid in finding the location of people or objects within the buildings. Identification of the items is through the capacity level of the signal received from various access points (i.e., Wi-Fi routers). The positioning of the devices utilizing some algorithms has drawn more attention from the researchers. Yet, the designed algorithm still has problems for accurate floor planning. So, the accuracy of position estimation with minimum error is made possible by introducing Gaussian Distributive Feature Embedding based Deep Recurrent Perceptive Neural Learning (GDFE-DRPNL), a novel framework. Novel features from the dataset are through two processing stages dimensionality reduction and position estimation. Initially, the essential elements selection using the Gaussian Distributive Feature Embedding technique is the novel framework. The feature reduction process aims to reduce the time consumption and overhead for estimating the location of various devices. In the next stage, employ Deep Recurrent multilayer Perceptive Neural Learning to evaluate the device position with dimensionality reduced features. The proposed Deep-learning approach accurately learns the quality and the signal strength data with multiple layers by applying Deming Regressive Trilateral Positioning Model. As a result, the GDFE-DRPNL framework increases the positioning accuracy and minimizes the error rate. The experimental assessments with various factors such as positioning accuracy minimized by 70% and 60%, computation time minimized by 45% and 55% as well as overhead by 11% and 23% compared with PFRL and two-dimensional localization algorithm. Through the experiment and after analyzing the data, verify that the proposed GDFE-DRPNL algorithm in this paper is better than the previous methods.

Revealing the Distributional Vulnerability of Discriminators by Implicit Generators.

In deep neural learning, a discriminator trained on in-distribution (ID) samples may make high-confidence predictions on out-of-distribution (OOD) samples. This triggers a significant matter for robust, trustworthy and safe deep learning. The issue is primarily caused by the limited ID samples observable in training the discriminator when OOD samples are unavailable. We propose a general approach for fine-tuning discriminators by implicit generators (FIG). FIG is grounded on information theory and applicable to standard discriminators without retraining. It improves the ability of a standard discriminator in distinguishing ID and OOD samples by generating and penalizing its specific OOD samples. According to the Shannon entropy, an energy-based implicit generator is inferred from a discriminator without extra training costs. Then, a Langevin dynamic sampler draws specific OOD samples for the implicit generator. Lastly, we design a regularizer fitting the design principle of the implicit generator to induce high entropy on those generated OOD samples. The experiments on different networks and datasets demonstrate that FIG achieves the state-of-the-art OOD detection performance.

A Deep Siamese Neural Network N-Shot Learning Approach for the Recognition of Chinese Calligraphy

A Deep Siamese Neural Network N-Shot Learning Approach for the Recognition of Chinese Calligraphy