Deep Neural Network Techniques Research Articles

Attention-based aspect sentiment classification using enhanced learning through cnn-Bilstm networks

Deep neural networks (dnn) techniques for aspect-based sentiment classification have been widely studied. The success of these methods depends largely on training data which are often inadequate because of the rigor involved in manually tagging large collection of opinionated texts. Attempts have been made to transfer knowledge from document-level to aspect-level sentiment task. However, the success of this approach is also dependent on the model because aspect sentiment data like other type of texts contain complex semantic features. In this paper, we present an attention-based deep learning technique which jointly learns on document and aspect-level sentiment data and which also transfers learning from the document-level data to aspect-level sentiment classification. It basically consists of a convolutional layer and a bidirectional long short-term memory (Bilstm) layer. The first variant of our technique uses convolutional neural network (cnn) to extract high-level semantic features. The output of the feature extraction is then fed into the Bilstm layer which captures the contextual feature representation of the texts. The second variant applies the Bilstm layer directly on the input data. In both variants, the output hidden representation is passed to an output layer using softmax activation function for sentiment polarity classification. We evaluate our model on four standard benchmark datasets which shows the effectiveness of our approach with improvements over baselines. We also conduct ablation studies to show the effect of the different document-level weights on the learning techniques.

Relevance Feedback and Deep Neural Network-Based Semantic Method for Query Expansion

Machine learning techniques have been widely used in almost every area of arts, science, and technology for the last two decades. Document analysis and query expansion also use machine learning techniques at a broad scale for information retrieval tasks. The state-of-the-art models like the Bo1 model, Bo2 model, KL divergence model, and chi-square model are probabilistic, and they work on DFR-based retrieval models. These models are much focused on term frequency and do not care about the semantic relationship among the terms. The proposed model applies the semantic method to find the semantic similarity among the terms to expand the query. The proposed method uses the relevance feedback method that selects a user-assisted most relevant document from top “ k ” initially retrieved documents and then applies deep neural network technique to select the most informative terms related to original query terms. The results are evaluated at FIRE 2011 ad hoc English test collection. The mean average precision of the proposed method is 0.3568. The proposed method also compares the state-of-the-art models. The proposed model observed 19.77% and 8.05% improvement on the mean average precision (MAP) parameter with respect to the original query and Bo1 model, respectively.

Simplified Image Classification for Nigeria's Agricultural Produce through Deep Neural Network Techniques

Simplified Image Classification for Nigeria's Agricultural Produce through Deep Neural Network Techniques

The Use of an Incremental Learning Algorithm for Diagnosing COVID-19 from Chest X-ray Images

The new Coronavirus or simply Covid-19 causes an acute deadly disease. It has spread rapidly across the world, which has caused serious consequences for health professionals and researchers. This is due to many reasons including the lack of vaccine, shortage of testing kits and resources. Therefore, the main purpose of this study is to present an inexpensive alternative diagnostic tool for the detection of Covid-19 infection by using chest radiographs and Deep Convolutional Neural Network (DCNN) technique. In this paper, we have proposed a reliable and economical solution to detect COVID-19. This will be achieved by using X-rays of patients and an Incremental-DCNN (I-DCNN) based on ResNet-101 architecture. The datasets used in this study were collected from publicly available chest radiographs on medical repositories. The proposed I-DCNN method will help in diagnosing the positive Covid-19 patient by utilising three chest X-ray imagery groups, these will be: Covid-19, viral pneumonia, and healthy cases. Furthermore, the main contribution of this paper resides on the use of incremental learning in order to accommodate the detection system. This has high computational energy requirements, time consuming challenges, while working with large-scale and regularly evolving images. The incremental learning process will allow the recognition system to learn new datasets, while keeping the convolutional layers learned previously. The overall Covid-19 detection rate obtained using the proposed I-DCNN was of 98.70\% which undeniably can contribute effectively to the detection of COVID-19 infection.

Extra Proximal-Gradient Network with Learned Regularization for Image Compressive Sensing Reconstruction.

Learned optimization algorithms are promising approaches to inverse problems by leveraging advanced numerical optimization schemes and deep neural network techniques in machine learning. In this paper, we propose a novel deep neural network architecture imitating an extra proximal gradient algorithm to solve a general class of inverse problems with a focus on applications in image reconstruction. The proposed network features learned regularization that incorporates adaptive sparsification mappings, robust shrinkage selections, and nonlocal operators to improve solution quality. Numerical results demonstrate the improved efficiency and accuracy of the proposed network over several state-of-the-art methods on a variety of test problems.

A Deep Neural Network Technique for Detecting Real-Time Drifted Twitter Spam

The social network is considered a part of most user’s lives as it contains more than a billion users, which makes it a source for spammers to spread their harmful activities. Most of the recent research focuses on detecting spammers using statistical features. However, such statistical features are changed over time, and spammers can defeat all detection systems by changing their behavior and using text paraphrasing. Therefore, we propose a novel technique for spam detection using deep neural network. We combine the tweet level detection with statistical feature detection and group their results over meta-classifier to build a robust technique. Moreover, we embed our technique with initial text paraphrasing for each detected tweet spam. We train our model using different datasets: random, continuous, balanced, and imbalanced. The obtained experimental results showed that our model has promising results in terms of accuracy, precision, and time, which make it applicable to be used in social networks.

A Deep Neural Network-Based Yoruba Intelligent Chatbot System

Two Artificial Intelligence software systems, Bot and Chatbot have recently debuted on the internet. This initiate a communication between the user and a virtual agent. The modeling and performance in deep learning (DL) computation for an Assistant Conversational Agent are presented in this research (Chatbot). The deep neural network (DNN) technique is used to respond to a large number of tokens in an input sentence with more appropriate dialogue. The model was created to do Yoruba-to-Yoruba translations. The major goal of this project is to improve the model's perplexity and learning rate, as well as to find a blue score for translation in the same language. Kares is used to run the experiments, which is written in Python. The data was trained using a deep learning-based algorithm. With the use of training examples, a collection of Yoruba phrases with various intentions was produced. The results demonstrate that the system can communicate in basic Yoruba terms and that it would be able to learn simple Yoruba words. The study result when evaluated showed that the system had 80% accuracy rate. Keywords: Chatbot, Natural Language Processing, Deep Learning, Artificial Neural Network, Yoruba Language

Cloud-based intelligent monitoring system to implement mask violation detection and alert simulation

The importance of wearing a mask in public places came to light when the COVID-19 pandemic has started due to the coronavirus. To strictly control the spread of the virus, wearing a mask is mandatory to avoid getting the virus through others or spreading the virus to others if we are carrying it. Since it’s not possible to check each individual in public places whether he/she is wearing a mask, this paper proposed a face mask detection using Deep Learning (DL) and Convolutional Neural Network (CNN) techniques. A cloud-based approach that adopted DL is used to identify the persons violating the rules. The dataset used in the work is collected from various studies, such as Prajnasb/observations and Kaggle’s Face Mask Detection Dataset that contains images of people wearing and not wearing masks. The faces in the images will be detected and cropped with the help of a trained face detector which will be used for checking whether the face in the image is wearing a mask or not. Face mask detection is done with the help of CNN. The input image is fed into the CNN and the output is binary format, whether person wearing or not wearing a mask. The work uses Max Pooling and Average Pooling layers of CNN. The outcome of the work shows that the proposed method achieves 98 % of accuracy using Max Pooling which is better than the currently available works.

Hyper Spectral Fruit Image Classification for Deep Learning Approaches and Neural Network Techniques

In the field of agro-business technology, computerization contributes to productivity, monetary turnover of events along local viability. The interest in tariffs in addition to the consistency analysis is influenced by the mix of leafy foods. The most tangible aspect of the food derived from the earth is the implementation that influences the need for, the customer’s desires as well as the judgment of the market. Although people may plan and assess, time-concentrated, complex, subjective, costly, and handily influenced by environmental variables is problematic. Subsequently, a shrewd natural product evaluation system is needed. Deep learning has achieved remarkable milestones in the field of conventional computers. In this article, we use deep learning techniques on the topic of hyperspectral image exploration. Unlike traditional machine vision exercises, the only thing to do with a gander is the spatial setting; our proposed solution would use both the spatial setting and the phantom relationship to enhance the hyperspectral image grouping. In clear words, we endorse four new deep learning models, in particular the 3D Convolutionary Neural Network (3D-CNN) and the Repetitive 3D Convolutionary Neural Network (R-3D-CNN) for hyperspectral image recognition.

Incorporation of machine learning and deep neural network approaches into a remote sensing-integrated crop model for the simulation of rice growth

Machine learning (ML) and deep neural network (DNN) techniques are promising tools. These can advance mathematical crop modelling methodologies that can integrate these schemes into a process-based crop model capable of reproducing or simulating crop growth. In this study, an innovative hybrid approach for estimating the leaf area index (LAI) of paddy rice using climate data was developed using ML and DNN regression methodologies. First, we investigated suitable ML regressors to explore the LAI estimation of rice based on the relationship between the LAI and three climate factors in two administrative rice-growing regions of South Korea. We found that of the 10 ML regressors explored, the random forest regressor was the most effective LAI estimator, and it even outperformed the DNN regressor, with model efficiencies of 0.88 in Cheorwon and 0.82 in Paju. In addition, we demonstrated that it would be feasible to simulate the LAI using climate factors based on the integration of the ML and DNN regressors in a process-based crop model. Therefore, we assume that the advancements presented in this study can enhance crop growth and productivity monitoring practices by incorporating a crop model with ML and DNN plans.

Artifical intelligence with optimal deep learning enabled automated retinal fundus image classification model

AbstractDiabetic retinopathy (DR) and age related macular degeneration (AMD) becomes widespread microvascular illness among diabetic patients. Traditional retinal fundus image classification requires visual inspection by the professionals, which is time consuming and requires expert's knowledge. Earlier identification of retinal diseases is essential to delay or avoid vision deterioration and vision loss. The recently developed artificial intelligence (AI) and deep learning (DL) models can be employed for accurate retinal image classification. With this motivation, this study designs a new artificial intelligence with optimal deep convolutional neural network (AI‐ODCNN) technique for retinal fundus image classification. Primarily, the proposed model uses the Gaussian Blur based noise removal and contrast enhancement technique (CLAHE) based contrast enhancement technique to pre‐process the retinal fundus image. In addition, morphology and contour based image segmentation is performed. Moreover, the deep CNN with RMSProp Optimizer is employed for retinal fundus image classification. A wide range of simulations was performed on the automated retinal image analysis and structured analysis of the retina and the outcomes are examined with respect to various measures. The simulation outcomes ensured the better performance of the proposed approach related to other recent algorithms with maximum accuracy of 96.47%.

Study on Poultry Pose Estimation Based on Multi-Parts Detection.

Simple SummaryPoultry farming is an important part of China’s agriculture system. The automatic estimation of poultry posture can help to analyze the movement, behavior, and even health of poultry. In this study, a poultry pose-estimation system was designed, which realized the automatic pose estimation of a single broiler chicken using a multi-part detection method. The experimental results show that this method can obtain better pose-estimation results for a single broiler chicken with respect to precision, recall, and F1 score. The pose-estimation system designed in this study provides a new means to provide help for poultry pose/behavior researchers in the future.Poultry pose estimation is a prerequisite for evaluating abnormal behavior and disease prediction in poultry. Accurate pose-estimation enables poultry producers to better manage their poultry. Because chickens are group-fed, how to achieve automatic poultry pose recognition has become a problematic point for accurate monitoring in large-scale farms. To this end, based on computer vision technology, this paper uses a deep neural network (DNN) technique to estimate the posture of a single broiler chicken. This method compared the pose detection results with the Single Shot MultiBox Detector (SSD) algorithm, You Only Look Once (YOLOV3) algorithm, RetinaNet algorithm, and Faster_R-CNN algorithm. Preliminary tests show that the method proposed in this paper achieves a 0.0128 standard deviation of precision and 0.9218 ± 0.0048 of confidence (95%) and a 0.0266 standard deviation of recall and 0.8996 ± 0.0099 of confidence (95%). By successfully estimating the pose of broiler chickens, it is possible to facilitate the detection of abnormal behavior of poultry. Furthermore, the method can be further improved to increase the overall success rate of verification.

Smart Road Damage Detection and Warning using Machine Learning

We present a neural network topology, as well as training and prediction algorithms, in this research. To create a safe road environment, we present a deep neural network technique to detect road surface deterioration conditions. For training and testing, we provide an image dataset as input. Various sorts of road anxiety are depicted in the photographs. The suggested approach is compared to a variety of deep learning models from different disciplines. The findings of this study are expected to play a significant role in guaranteeing safe driving in the future by effectively detecting poor road conditions.

FC048: New Tool to Predict the Clinical Course and Renal Failure in Patients with Immunoglobulin a Nephropathy

Abstract BACKGROUND AND AIMS Idiopathic Immunoglobulin A nephropathy (IgAN) is the most common biopsy-proven glomerulonephritis in the world. Approximately 40% of IgAN patients reach renal failure (RF) 20 years after their kidney biopsy. The high prevalence of RF shows that IgAN has a significant economic impact in the countries because renal replacement therapy is costly. Moreover, the disease's onset in the second and third decades of life represents a social challenge because patients are typically very active and highly productive in the workplace. This challenge is one more reason to move on the prediction of the clinical course and RF in IgAN patients at the time of the kidney biopsy and during the follow-up. We developed an artificial neural network (ANN) tool (DialCheck 1.0) based on seven variables and the histological score of the kidney biopsy to predict RF in IgAN patients at the time of kidney biopsy [1]. METHOD We have recently developed a new tool which consists of a set of ANN-powered models that combine temporally accurate observations for fine-grained features and leverage state-of-the-art deep neural network techniques to forecast the patient's clinical evolution. RESULTS A cohort of 948 IgAN patients, of whom the clinical course of the disease was known, was used to develop the new tool that predicts the dynamics of age and disease laboratory parameters (serum creatinine, daily proteinuria), blood pressure, histological score of the kidney biopsy and the RF. The system was designed to help the physicians give a broader spectrum of information regarding the patient and the potential clinical development of IgAN and outcomes. In detail, the model computes a latent dynamic representation of the patient and predicts a prospective clinical picture of the patient and the probability of RF. The tool with an accuracy of >80% will be tested in an independent retrospective cohort of 454 IgAN patients and in a prospective multicenter randomized clinical study in which 426 IgAN patients will be enrolled in two different groups based on the type of kidney lesions (active and chronic renal lesions). CONCLUSION We have a new tool (DialCheck 2.0), based on ANN, that may predict the outcome and the RF in IgAN patients. Moreover, it may help the physician to analyze the long-term response to therapy.

Synthetic to Realistic Imbalanced Domain Adaption for Urban Scene Perception

Deep neural networks (DNNs) technique has achieved impressive performance on semantic segmentation,while its training process requires a large amount of pixel-wise labelled data. Domain adaptation,as a promising solution,can break the restriction by training the model on synthetic data,and generalising it in real-world data. However,there is still a lack of attention paid to the imbalance problems on semantic segmentation adaptation,including the imbalance problem between i) source and target data,ii) different classes. To solve these problems,a progressive hierarchical feature alignment method is proposed in this paper. To alleviate the data imbalance problem,the network is progressively trained by the data from multisource domains,so as to obtain domain-invariant features. To address the class imbalance problem,the features are aligned hierarchically across domains. According to the experimental results,our method shows the competitive adapted segmentation performance on three benchmark datasets.

Configuration of the Deep Neural Network Hyperparameters for the Hypsometric Modeling of the Guazuma crinita Mart. in the Peruvian Amazon

The Guazuma crinita Mart. is a dominant species of great economic importance for the inhabitants of the Peruvian Amazon, standing out for its rapid growth and being harvested at an early age. Understanding its vertical growth is a challenge that researchers have continued to study using different hypsometric modeling techniques. Currently, machine learning techniques, especially artificial neural networks, have revolutionized modeling for forest management, obtaining more accurate predictions; it is because we understand that it is of the utmost importance to adapt, evaluate and apply these methods in this species for large areas. The objective of this study was to build and evaluate the efficiency of the use of a deep neural network for the prediction of the total height of Guazuma crinita Mart. from a large-scale continuous forest inventory. To do this, we explore different configurations of the hidden layer hyperparameters and define the variables according to the function HT = f(x) where HT is the total height as the output variable and x is the input variable(s). Under this criterion, we established three HT relationships: based on the diameter at breast height (DBH), (i) HT = f(DBH); based on DBH and Age, (ii) HT = f(DBH, Age) and based on DBH, Age and Agroclimatic variables, (iii) HT = f(DBH, Age, Agroclimatology), respectively. In total, 24 different configuration models were established for each function, concluding that the deep artificial neural network technique presents a satisfactory performance for the predictions of the total height of Guazuma crinita Mart. for modeling large areas, being the function based on DBH, Age and agroclimatic variables, with a performance validation of RMSE = 0.70, MAE = 0.50, bias% = −0.09 and VAR = 0.49, showed better accuracy than the others.

Answering medical questions in Chinese using automatically mined knowledge and deep neural networks: an end-to-end solution

BackgroundMedical information has rapidly increased on the internet and has become one of the main targets of search engine use. However, medical information on the internet is subject to the problems of quality and accessibility, so ordinary users are unable to obtain answers to their medical questions conveniently. As a solution, researchers build medical question answering (QA) systems. However, research on medical QA in the Chinese language lags behind work on English-based systems. This lag is mainly due to the difficulty of constructing a high-quality knowledge base and the underutilization of medical corpora in the Chinese language.ResultsThis study developed an end-to-end solution to implement a medical QA system for the Chinese language with low cost and time. First, we created a high-quality medical knowledge graph from hospital data (electronic health/medical records) in a nearly automatic manner that trained a supervised model based on data labeled using bootstrapping techniques. Then, we designed a QA system based on a memory-based neural network and attention mechanism. Finally, we trained the system to generate answers from the knowledge base and a QA corpus on the internet.ConclusionsBootstrapping and deep neural network techniques can construct a knowledge graph from electronic health/medical records with satisfactory precision and coverage. Our proposed context bridge mechanisms perform training with a variety of language features. Our QA system can achieve state-of-the-art quality in answering medical questions with constrained topics. As we evaluated, complex Chinese language processing techniques, such as segmentation and parsing, were not necessary for practice and complex architectures were not necessary to build the QA system. Lastly, we created an application using our method for internet QA usage.

Mitigating tunnel-induced damages using deep neural networks

For intelligent risk prediction and optimization in the shield tunnel excavation under uncertainty, this paper proposes a data-driven decision support framework based on the integration of deep neural network and gradient descent technique (DNN-GDO). More specifically, the DNN architecture for modeling non-linear relationships between influential factors and potential risk is superior in concurrently extracting and learning features from multi-variable inputs, while the GDO technique as a solution for the multi-objective optimization problem can be directly applied in the established DNN model to provide suggestions for reducing or preventing potential risks in the tunneling environment. To reveal the effectiveness of the developed approach, it is performed in a practical tunnel construction case in the Wuhan metro system, China. Sixteen decision variables belong to four categories of tunnel design, geology, the tunnel boring machine (TBM) operation, and surrounding building are taken into account, which are closely relevant to two significant risks named the accumulative ground settlement and building tilt rate. It has been found that the end-to-end learning pipeline based on DNN can yield high accuracy in the multi-output regression task, resulting in the mean-square error (MSE) of 0.737 and 0.002 for the two target risks, respectively. GDO is proven useful in not only finding the optimal solutions for effective risk control but also uncovering the key features for a better explanation of the optimization process. It has been proved that optimal solutions derived from GDO can flexibly guide to adjust input variables of interest to achieve the minimal risk degree. Besides, cutter rotating speed and ground volume of TBM along with the relative horizontal distance of buildings turn out to be the top 3 critical features contributing more to the tunnel-induced risk mitigation, which deserve more attention. In brief, the novel DNN-GDO decision tool supports to formulate meaningful strategies objectively and efficiently for tunneling safety assurance, which no longer largely relies on time-consuming and laborious manual assessment.

Comparison of Fine-Tuned Deep Convolutional Neural Networks for the Automated Classification of Lung Cancer Cytology Images with Integration of Additional Classifiers.

Objective:It is essential to accurately diagnose and classify histological subtypes into adenocarcinoma (ADC), squamous cell carcinoma (SCC), and small cell lung carcinoma (SCLC) for the appropriate treatment of lung cancer patients. However, improving the accuracy and stability of diagnosis is challenging, especially for non-small cell carcinomas. The purpose of this study was to compare multiple deep convolutional neural network (DCNN) technique with subsequent additional classifiers in terms of accuracy and characteristics in each histology. Methods:Lung cancer cytological images were classified into ADC, SCC, and SCLC with four fine-tuned DCNN models consisting of AlexNet, GoogLeNet (Inception V3), VGG16 and ResNet50 pretrained by natural images in ImageNet database. For more precise classification, the figures of 3 histological probabilities were further applied to subsequent machine learning classifiers using Naïve Bayes (NB), Support vector machine (SVM), Random forest (RF), and Neural network (NN). Results:The classification accuracies of the AlexNet, GoogLeNet, VGG16 and ResNet50 were 74.0%, 66.8%, 76.8% and 74.0%, respectively. Well differentiated typical morphologies were tended to be correctly judged by all four architectures. However, poorly differentiated non-small cell carcinomas lacking typical structures were inclined to be misrecognized in some DCNNs. Regarding the histological types, ADC were best judged by AlexNet and SCC by VGG16. Subsequent machine learning classifiers of NB, SVV, RF, and NN improved overall accuracies to 75.1%, 77.5%, 78.2%, and 78.9%, respectively. Conclusion:Fine-tuning DCNNs in combination with additional classifiers improved classification of cytological diagnosis of lung cancer, although classification bias could be indicated among DCNN architectures.

Improving connectivity and accelerating multiscale topology optimization using deep neural network techniques

Improving connectivity and accelerating multiscale topology optimization using deep neural network techniques