Deep Learning Artificial Neural Network Research Articles

Remembrance of things perceived: Adding thalamocortical function to artificial neural networks.

Recent research has illuminated the complexity and importance of the thalamocortical system but it has been difficult to identify what computational functions it performs. Meanwhile, deep-learning artificial neural networks (ANNs) based on bio-inspired models of purely cortical circuits have achieved surprising success solving sophisticated cognitive problems associated historically with human intelligence. Nevertheless, the limitations and shortcomings of artificial intelligence (AI) based on such ANNs are becoming increasingly clear. This review considers how the addition of thalamocortical connectivity and its putative functions related to cortical attention might address some of those shortcomings. Such bio-inspired models are now providing both testable theories of biological cognition and improved AI technology, much of which is happening outside the usual academic venues.

Automatic Generation of Literary Sentences in French

In this paper, we describe a model for the automatic generation of literary sentences in French. Although there has been much recent effort directed towards automatic text generation in general, the generation of creative, literary sentences that is not restricted to a specific genre, which we approached in this work, is a difficult task that is not commonly treated in the scientific literature. In particular, our present model has not been previously applied to the generation of sentences in the French language. Our model was based on algorithms that we previously used to generate sentences in Spanish and Portuguese and on a new corpus, which we constructed and present here, consisting of literary texts in French, called MegaLitefr. Our automatic text generation algorithm combines language models, shallow parsing, the canned text method, and deep learning artificial neural networks. We also present a manual evaluation protocol that we propose and implemented to assess the quality of the artificial sentences generated by our algorithm, by testing if they fulfil four simple criteria. We obtained encouraging results from the evaluators for most of the desired features of our artificially generated sentences.

Prognostic Prediction of Cancer Based on Radiomics Features of Diagnostic Imaging: The Performance of Machine Learning Strategies.

Tumor phenotypes can be characterized by radiomics features extracted from images. However, the prediction accuracy is challenged by difficulties such as small sample size and data imbalance. The purpose of the study was to evaluate the performance of machine learning strategies for the prediction of cancer prognosis. A total of 422 patients diagnosed with non-small cell lung carcinoma (NSCLC) were selected from The Cancer Imaging Archive (TCIA). The gross tumor volume (GTV) of each case was delineated from the respective CT images for radiomic features extraction. The samples were divided into 4 groups with survival endpoints of 1 year, 3 years, 5 years, and 7 years. The radiomic image features were analyzed with 6 different machine learning methods: decision tree (DT), boosted tree (BT), random forests (RF), support vector machine (SVM), generalized linear model (GLM), and deep learning artificial neural networks (DL-ANNs) with 70:30 cross-validation. The overall average prediction performance of the BT, RF, DT, SVM, GLM and DL-ANNs was AUC with 0.912, 0.938, 0.793, 0.746, 0.789 and 0.705 respectively. The RF and BT gave the best and second performance in the prediction. The DL-ANN did not show obvious advantage in predicting prognostic outcomes. Deep learning artificial neural networks did not show a significant improvement than traditional machine learning methods such as random forest and boosted trees. On the whole, the accurate outcome prediction using radiomics serves as a supportive reference for formulating treatment strategy for cancer patients.

An Intrusion Detection Mechanism for MANETS based on Deep Learning Artificial Neural Networks (ANNS)

Mobile Ad-hoc Network (MANET) is a distributed, decentralized network of wireless portable nodes connecting directly without any fixed communication base station or centralized administration. Nodes in MANET move continuously in random directions and follow an arbitrary manner, which presents numerous challenges to these networks and make them more susceptible to different security threats. Due to this decentralized nature of their overall architecture, combined with the limitation of hardware resources, those infrastructure-less networks are more susceptible to different security attacks such as black hole attack, network partition, node selfishness, and Denial of Service (DoS) attacks. This work aims to present, investigate, and design an intrusion detection predictive technique for Mobile Ad hoc networks using deep learning artificial neural networks (ANNs). A simulation-based evaluation and a deep ANNs modelling for detecting and isolating a Denial of Service (DoS) attack are presented to improve the overall security level of Mobile ad hoc networks.

Analysis of Factors Affecting Electric Power Quality: PLS-SEM and Deep Learning Neural Network Analysis

The world today is increasingly dependent directly or indirectly on the power system. Ensuring the quality of power supplied to electrical equipment is essential. The national regulatory framework is for harmonic mitigation in the global power system. This paper discusses the relationship between Efficiency (E), Security (S), and Reliability (R) for Electric Power Quality (EPQ). We measure the harmonic mitigation regulations listed in the IEEE 519 standard. To evaluate the proposed E, S, and R constructs and their relationship to EPQ, a multi-planning approach the method of Partial Least Squares-Structural Equation Modeling (PLS-SEM) and Deep Learning Artificial Neural Network (ANN) analysis were performed. In it, deep Learning Artificial Neural Network (ANN) was performed to complement the PLS-SEM findings and higher prediction accuracy. The study shows that the aspects of efficiency (E), security (S), and reliability (R) have a significant relationship with Electric Power Quality (EPQ). Another result of the study indicates that science, technology, engineering and math (STEM) resource conditions have a significant and positive impact on EPQ.

Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

The muscular activities gathered by real-time myoelectric interfaces of surface electromyography (sEMG) can be used to develop myoelectric prosthetic hands for physically disabled people. However, the acquired myoelectric signals must be accurately classified in real time to properly control the operation of the external devices. In this study, we propose methods for detecting and classifying muscular activities using sEMG signals. These methods include outlier removal, data manipulation, data preprocessing, dimensionality reduction, and classification. We use the Ninapro database 1 (DB1) containing sEMG signals from 27 intact subjects while performing 53 hand movements repeatedly. We apply the Principle Component Analysis (PCA), Independent Component Analysis (ICA), and t-distributed Stochastic Neighbor Embedding (t-SNE) feature extraction methods for dimensionality reduction. Five machine learning (ML) algorithms and deep learning artificial neural networks (ANN) are applied for the classification of muscular movements. It is observed that for the recognition of 53 muscular movements of 27 subjects with preprocessed raw data, ANN obtains the highest accuracy of 93.92% for inter-subject and 97.73% for intra-subject movement recognition. Among the ML algorithms, K-Nearest Neighbors (KNN) performs the best with both t-SNE features and the preprocessed raw data in least computational time. With the preprocessed raw data, KNN obtains 93.174% and 97.458% for inter-subject and intra-subject movement classification, respectively while with the t-SNE features, KNN obtains 89.844% accuracy for inter-subject and 95.04% accuracy for intra-subject in reduced computational time.

Evaluating and Modeling the Degradation of PLA/PHB Fabrics in Marine Water.

Developing degradable bio-plastics has been considered feasible to lessen marine plastic pollution. However, unanimity is still elusive regarding the actual degradability of bio-plastics such as polylactide (PLA) and poly(hydroxybutyrate) (PHB). Thus, herein, we studied the degradability of fabrics made from PLA/PHB blends in marine seawater. The dry-mass percentage of the PLA/PHB fabrics decreased progressively from 100% to 85~90% after eight weeks of immersion. Two environmental aging parameters (UV irradiation and aerating) were also confirmed to accelerate the abiotic hydrolysis of the incubated fabrics. The variation in the molecular structure of the PLA/PHB polymers after the degradation process was investigated by electrospray ionization mass spectrometry (ESI-MS). However, the hydrolysis degradability of bulky PLA/PHB blends, which were used to produce such PLA/PHB fabrics, was negligible under identical conditions. There was no mass loss in these solid PLA/PHB plastics except for a decrease in their tensile strength. Finally, a deep learning artificial neural network model was proposed to model and predict the nonlinear abiotic hydrolysis behavior of PLA/PHB fabrics. The degradability of PLA/PHB fabrics in marine water under the synergistic destructive effects of seawater, UV, and dissolved oxygen provides a pathway for more sustainable textile fibers and apparel products.

Prediction of hydrogen concentration responsible for hydrogen-induced mechanical failure in martensitic high-strength steels

Hydrogen, at critical concentrations, responsible for hydrogen-induced mechanical property degradation cannot yet be estimated beforehand and can only be measured experimentally upon fracture with specific specimen sizes. In this work, we develop two deep learning artificial neural network (ANN) models with the ability to predict hydrogen concentration responsible for early mechanical failure in martensitic ultra-high-strength steels. This family of steels is represented by four different steels encompassing different chemical compositions and heat treatments. The mechanical properties of these steels with varying size and morphology of prior austenitic grains in as-supplied state and after hydrogen-induced failure together with their corresponding hydrogen charging conditions were used as inputs. The feed forward back propagation models with network topologies of 12-7-5-3-2-1 (I) and 14-7-5-3-2-1 (II) were validated and tested with unfamiliar data inputs. The models I and II show good hydrogen concentration prediction capabilities with mean absolute errors of 0.28, and 0.33 wt.ppm at test datasets, respectively. A linear correlation of 80% and 77%, between the experimentally measured and ANN predicted hydrogen concentrations, was obtained for Model I and II respectively. This shows that for this family of steels, the estimation of hydrogen concentration versus property degradation is a feasible approach for material safety analysis.

X-ray dose profiles using artificial neural networks

This paper introduces a novel computational method to simulate and predict radiation dose profiles in a water phantom irradiated by X-rays of 6 and 15 MV at different depths and field sizes using Artificial Neural Networks within the error margin required by the code of practice 398 of the International Atomic Energy Agency (IAEA). Our method uses deep-learning Artificial Neural Networks as an alternative to the Monte Carlo methods usually used nowadays. It reproduces the radiation dose profiles for X-rays of 6 and 15 MV data reported in the British Journal of Radiology (Aird, 1996). Even more, our method reproduces data from other sources with acceptable errors. These simulations pave the way to enhance radiotherapy techniques in planning patient doses and calibrating ionizing radiation measurement instruments used in the fight against cancer.

A new recurrent pi‐sigma artificial neural network inspired by exponential smoothing feedback mechanism

AbstractPi‐sigma artificial neural networks have very good performance for forecasting problems because of their highly nonlinear model structure. Some time series can be forecasted better with the combination of simple and highly nonlinear structures. In this study, the architecture of the pi‐sigma artificial neural network is modified by an inspiring exponential smoothing feedback mechanism. The architecture of the proposed neural network automatically balances simple and complex nonlinear model structures. Moreover, a training algorithm is created by using the sine cosine algorithm. The training algorithm has some solutions for overfitting and local optimum problems. The main contribution of the study is to propose a new hybrid recurrent neural network and its training algorithm. In the analysis of the new network's performance, the randomly selected eight subseries of the Financial Times Stock Exchange 100 Index daily time series between 2014 and 2018 are used. The performance of the proposed method is compared with popular deep learning artificial neural networks, pi‐sigma artificial neural networks, and classical exponential smoothing methods. The statistical results show that the proposed method can produce better forecasting results than the others.

An Integrated Seamless Control Strategy for Distributed Generators Based on a Deep Learning Artificial Neural Network

One of the challenges of inverter-based distributed generators (DGs) is to keep the voltage and frequency at their specified limits during transitions between grid-connected and islanded modes of operation. This paper presents an integrated seamless control strategy for inverter-based DGs to ensure smooth transitions between the different modes of operation. The proposed strategy is based on a deep learning neural network (DL-ANN) Proportional-Integral- Derivative (PID) controller to regulate the terminal voltage of the DG interface system. A feed-forward loop is integrated with the proposed strategy to mitigate grid harmonics by controlling the DG inverter to feed the harmonics components of non-linear loads without exceeding its capacity. Results are provided to evaluate the dynamic performance of the proposed unified control strategy under different disturbances. Finally, to demonstrate the superiority of the DL-ANN controller, a comparison is carried out with the conventional Proportional-Integral (PI) controller and the set-membership affine projection adaptive (SMAPA)-based PI controller.

Employment of instrumented vehicles to identify real-time snowy weather conditions on freeways using supervised machine learning techniques – A naturalistic driving study

This study introduces the idea of using vehicles as weather sensors to identify real-time weather on freeways in the same context as Road Weather Information System (RWIS) but in a continuous, trajectory-level, and for road segments allocated in the vehicles paths. The study developed a novel approach to detect snowy and clear weather conditions by utilizing real-time data collected from vehicles' external sensors and CANbus. The proposed approach used time series datasets from the SHRP2 Naturalistic Driving Study (NDS), collected during normal driving conditions on freeways. Trips occurring in snowy weather alongside matched trips in clear weather were segmented into time- and distance-based segments such as a one-minute, one-mile, and half a mile. Three assemblies of the input data are considered in the modeling step: data collected from external sensors, CANbus data, and these two data combined. Data analysis was implemented using the Deep Learning Artificial Neural Network, Decision Tree, Random Forest, and Gradient Boosted Trees models. The results indicate that using different segmentation levels provides decent results in detecting snowy weather. The accuracy in estimating the real-time snowy weather was in ranges of 80% to 85%, 71% to 79%, and 73% to 83% for the one-minute, one-mile, and half mile segmentation types, respectively. The GBT model performed the best among all models based on the area under the Receiver Operating Characteristics (ROC) curve, the highest cumulative percentage in estimating the snowy weather using the lower portion of the population, and the highest overall accuracy. Results indicated that an accuracy of 83% in estimating snowy weather conditions could be accomplished using the data collected from external sensors only without accessing CANbus data.

The Development of Spatial Attention U‐Net for the Recovery of Ionospheric Measurements and Extraction of Ionospheric Parameters

AbstractWe train a deep learning artificial neural network model, Spatial Attention U‐Net, to recover useful ionospheric signals from noisy ionogram data measured by Hualien's Vertical Incidence Pulsed Ionospheric Radar. Our results show that the model can well identify F2 layer ordinary and extraordinary modes (F2o and F2x) and the combined signals of the E layer (ordinary and extraordinary modes and sporadic Es). The model is also capable of identifying some signals that were not labeled. The performance of the model can be significantly degraded by insufficient number of samples in the data set. From the recovered signals, we determine the critical frequencies of F2o and F2x and the intersection frequency between the two signals. The difference between the two critical frequencies is peaking at 0.63 MHz, with the uncertainty being 0.18 MHz.

Detection of fruit maturity stage and yield estimation in wild blueberry using deep learning convolutional neural networks

This study looked at the development of six deep learning artificial neural network models for detecting ripeness stage in wild blueberries, along with developing models for yield estimation. The six networks used were YOLOv3, YOLOv3-SPP, YOLOv3-Tiny, YOLOv4, YOLOv4-Small and YOLOv4-Tiny. Both 3-class (green berries, red berries, blue berries) and 2-class (unripe berries, ripe berries) models were developed with YOLOv4 performing the best with mean average precisions of 79.79% and 88.12% respectively. This result was further supported by YOLOv4 achieving the highest F1 score of 0.82. YOLOv4-Tiny performed the best from a computational load perspective having a mean inference time of 7.8 ms and a mean memory usage of 1.63 GB for single 1280 × 736 pixel images. Only minor differences in the accuracy of the nonlinear regression yield prediction models were detected, with YOLOv4-Small performing the best with a mean absolute error of 24.1%. Despite this error, the results are encouraging, and this novel approach to yield estimation in wild blueberries will aid growers in making better, more localized, management decisions, improving yields and ultimately increasing profits by better understanding their fields ripening characteristics.

Advanced calibration of mortality prediction on cardiovascular disease using feature-based artificial neural network

Cardiovascular (CVD) is the leading cause of death worldwide and a significant public health concern. Therefore, its mortality prediction is crucial to many existing treatment guidelines. Medical claims data can be used to accurately foresee the health outcomes of patients contracting to a variety of diseases. Many machine learning algorithms, especially deep learning artificial neural networks, can predict mortality rate among patients with CVD using clinical data. Calibration of probabilistic prediction is essential for precise medical interventions as it indicates how well a model’s output matches the probability of the event. However, deep learning neural networks are poorly calibrated. Through experiments, we observe that feature representation is an important factor influencing calibration. This paper proposes a novel feature-based deep learning neural network framework to predict the mortality rate among patients with CVD. Our focus is to present a comprehensive study to achieve advanced performance calibration of mortality prediction on CVD in leveraging deep learning architecture and feature representations. Our study demonstrates that the proposed feature-based neural network framework integrated with Principal Component Analysis or Autoencoders significantly reduces training time and boosts calibration, making model updates in clinical context more flexible and decision-making in medical prevention more reliable.

Fuzzy Deep Daily Nutrients Requirements Representation

The optimal regime models implement parameters presented by nominal values, intervals, fuzzy models, intuitionistic models. Unfortunately, these models are restrictive and ignore a significant portion of the knowledge contained in the specifications. To overcome this problem, we propose an optimal system that implements deep learning artificial neural networks and fuzzy genetic algorithms for the first time in the literature. The deep neural network extracts the information, the neural network units memorize this information, genetic algorithms select the best architecture of the auto-encoder basing on new regulation function, and fuzzy logic allows some flexibility for our system. First, we collect the expert's nutrients recommendations from different expert research papers. These recommendations are, then, represented in terms of trapezoidal numbers by adopting appropriate rules that encourage the consumption of the favorable nutrients and limit consumption of the unfavorable nutrients in daily diets. Then, we generate large data sets basing on the trapezoidal representation. To transform the expert's recommendations into significant crisp values, we call the auto-encoder neural network, and we propose an original regulation term that controls all the auto-encoder units. To select the best auto-encoder architecture, we use the fuzzy genetic algorithm basing on a simple fuzzy rule to determine the crossover percent, the mutation percent, and the population size at each iteration. Compared to the random systems, the proposed method has shown a great capacity to generalize its experience to unseen recommendations. In a clinical setting, our system can be used by a dietician to accurately determine the daily nutrient requirements of a given individual.

Comparison of sex determination using three methods applied to the greater sciatic notch of os coxae in a Thai population: Dry bone morphology, 2-dimensional photograph morphometry, and deep learning artificial neural network.

The os coxa is commonly used for sex and age estimation with a high degree of accuracy. Our study aimed to compare the accuracy among three methods, which include a deep learning approach to increase the accuracy of sex prediction. A total sample of 250 left os coxae from a Thai population was divided into a 'training' set of 200 samples and a 'test' set of 50 samples. The age of the samples ranged from 26 to 94 years. Three methods of sex determination were assessed in this experiment: a dry bone method, an image-based method and deep learning method. The intra- and inter-observer reliabilities were also assessed in the dry bone and photo methods. Our results showed that the accuracies were 80.65%, 90.3%, and 91.95% for the dry bone, image-based, and deep learning methods, respectively. The greater sciatic notch shape was wide and symmetrical in females while narrow and asymmetrical in males. The intra- and inter-observer agreements were moderate to almost perfect level (Kappa = 0.67-0.93, ICC = 0.74-0.94). Conclusion: The image-based and deep learning methods were efficient in sex determination. However, the deep learning technique performed the best among the three methods due to its high accuracy and rapid analysis. In this study, deep learning technology was found to be a viable option for remote consultations regarding sex determination in the Thai population.

Construction and implementation of music recommendation model utilising deep learning artificial neural network and mobile edge computing

The purpose is to better develop China's music industry, facilitate users' online music works queries, and encourage the development of excellent music works. The music recommendation system is analysed, and a new music recommendation system is constructed based on the combination of the hybrid Deep Learning (DL) Artificial Neural Network (ANN) and Mobile Edge Computing (MEC) technologies. First, the principle of MEC is analysed. Then, the combination of Deep Learning Artificial Neural Network (DLANN) technology and MEC technology is discussed. Afterward, DLANN and MEC are fused to implement the music recommendation model, and the proposed model is evaluated through experiment. The results show that the combination of DLANN and MEC technologies can increase the DL efficiency of the computer, the storage capacity, and the overall efficiency of the servers. This proves the feasibility of the proposed music recommendation system. The user satisfaction of the proposed music recommendation system exceeds all mainstream music recommendation algorithms on the market. Thus, the research can provide a reference for the improvement of the music recommendation algorithm in the future and is of great significance. The proposed music recommendation system outperforms most of the current music recommendation systems on the market.

Construction and implementation of music recommendation model utilising deep learning artificial neural network and mobile edge computing

Construction and implementation of music recommendation model utilising deep learning artificial neural network and mobile edge computing

New Directions for Skincare Monitoring: An NFC-Based Battery-Free Approach Combined With Deep Learning Techniques

Skincare monitoring has always been of paramount importance in the field of dermatology. In this study, we developed a smart skincare device that can harvest energy from a near field communication (NFC)-based smartphone and allow the adoption of a battery-free design approach. This device consists of two integrated sensors: one for the measurement of skin moisture and another for that of ultraviolet (UV) radiations. We conducted a series of experimental tests on different subjects in indoor and outdoor environments (8 and 6, respectively). Their skin moisture and temperature were measured parallelly to the ultraviolet A (UVA) and ultraviolet B (UVB) radiations from the sun. Later, the 6 channel sensing outputs obtained from the sensors (including ambient humidity and temperature) were input into a deep learning artificial neural network (ANN) model, which was used to predict the corresponding outputs and calculate the respective mean square error (MSE). The ultraviolet index (UVI) outputs were classified (using the same ANN model) into “less harmful”, “moderate harmful” and “burn”. The overall classification accuracy was 99.8%: the best performance achieved by using an ANN model. Notably, our skincare device is enclosed in a 3D flexible design print and is smart, battery-free, equipped with an Android application interface and more convenient to transport than other commercially available devices.