Neural Network-based Deep Learning Research Articles

Application of Deep Learning Technology in the Recommendation System of Constitutional and Constitutional Cases

Due to the huge system in the field of constitution, the content of cases is complicated, and the number of documents related to constitutional cases is huge. It is incredibly tough to find the information that suits your demands in such a large volume of constitutional documents. The artificial neural network-based deep learning method has made significant progress. Deep learning has been used in the disciplines of computer image and video processing, as well as natural language processing, and has outperformed typical machine learning methods. To that purpose, the content-based recommendation algorithm in this work is improved using deep learning approaches. This paper primarily covers the following topics: (1) the current research status of deep learning technology in recommender systems is reviewed, and the theoretical knowledge of deep neural networks is introduced. (2) A text classification model OCCNN based on character-level convolutional neural network is proposed to solve the problem of document classification of massive cases. A semantic similarity calculation model TF-W2V based on word frequency and word vector is also proposed to solve the problem of semantic similarity calculation of cases. (3) The two methods are tested, and the experimental results show that the accuracy is much higher than that of the conventional method. The model can recommend cases that meet the needs of users according to the requirements put forward by users in the process of handling cases.

A One-Dimensional Convolutional Neural Network (1D-CNN) Based Deep Learning System for Network Intrusion Detection

The connectivity of devices through the internet plays a remarkable role in our daily lives. Many network-based applications are utilized in different domains, e.g., health care, smart environments, and businesses. These applications offer a wide range of services and provide services to large groups. Therefore, the safety of network-based applications has always been an area of research interest for academia and industry alike. The evolution of deep learning has enabled us to explore new areas of research. Hackers make use of the vulnerabilities in networks and attempt to gain access to confidential systems and information. This information and access to systems can be very harmful and portray losses beyond comprehension. Therefore, detection of these network intrusions is of the utmost importance. Deep learning based techniques require minimal inputs while exploring every possible feature set in the network. Thus, in this paper, we present a one-dimensional convolutional neural network-based deep learning architecture for the detection of network intrusions. In this research, we detect four different types of network intrusions, i.e., DoS Hulk, DDoS, and DoS Goldeneye which belong to the active attack category, and PortScan, which falls in the passive attack category. For this purpose, we used the benchmark CICIDS2017 dataset for conducting the experiments and achieved an accuracy of 98.96% as demonstrated in the experimental results.

Artificial intelligence inferred microstructural properties from voltage–capacity curves

The quantification of microstructural properties to optimize battery design and performance, to maintain product quality, or to track the degradation of LIBs remains expensive and slow when performed through currently used characterization approaches. In this paper, a convolution neural network-based deep learning approach (CNN) is reported to infer electrode microstructural properties from the inexpensive, easy to measure cell voltage versus capacity data. The developed framework combines two CNN models to balance the bias and variance of the overall predictions. As an example application, the method was demonstrated against porous electrode theory-generated voltage versus capacity plots. For the graphite|LiMn_2O_4 chemistry, each voltage curve was parameterized as a function of the cathode microstructure tortuosity and area density, delivering CNN predictions of Bruggeman’s exponent and shape factor with 0.97 R^2 score within 2 s each, enabling to distinguish between different types of particle morphologies, anisotropies, and particle alignments. The developed neural network model can readily accelerate the processing-properties-performance and degradation characteristics of the existing and emerging LIB chemistries.

Detection of features from the internet of things customer attitudes in the hotel industry using a deep neural network model

Tourism and the hotel business have benefited greatly from the use of digital social networking. Using social big data research, the application of deep learning seems to have been beneficial in a marketing strategies and customer preference estimate. Recognizing human psychology, which is critical to industrial success, has benefited greatly from digital technology and social media. The Internet of Things (IoT) provides a chance for a hotel sector to improve the customer experience although lowering operational expenses. The ratings are determined by the following factors: Value, Apartment, Location, Hygiene, Front Office, Facilities, Professional Service, Internet, and Parking. Traditional techniques which anticipate hotel evaluations through minimal precision add difficulty to the rating assessment. As a result, efficient deep learning algorithms are employed to evaluate reviews designed to help consumers in selecting better hotels. To predict qualities, multiple classification techniques, including convolutional neural network-based deep learning (CNN-DL) and support vector machine (SVM) network-based deep learning, were used in this research. The system examines system efficiency by using the TripAdvisor website, this is a well American database. The research results reveal that the CNN-DL method outperforms another method in terms of classification efficiency and failure rate.The graphical results could also be utilized to enhance the effectiveness of the suggested model and offer insights into response tactics, demonstrating the study's academic and conceptual achievements. Although it is feasible to conclude from such research that the possibility of IoT within the hotel industry has not yet been fully investigated, as researchers commonly speculate on using IoT for implementations that might quickly be of involvement to a hotel sector, but refuse to recognize that possibility as a massive market.

Efficient self-attention mechanism and structural distilling model for Alzheimer’s disease diagnosis

Structural magnetic resonance imaging (sMRI) is commonly used for the identification of Alzheimer’s disease because of its keen insight into atrophy-induced changes in brain structure. Current mainstream convolutional neural network-based deep learning methods ignore the long-term dependencies between voxels; thus, it is challenging to learn the global features of sMRI data. In this study, an advanced deep learning architecture called Brain Informer (BraInf) was developed based on an efficient self-attention mechanism. The proposed model integrates representation learning, feature distilling, and classifier modeling into a unified framework. First, the proposed model uses a multihead ProbSparse self-attention block for representation learning. This self-attention mechanism selects the first ⌊lnN⌋ elements that can represent the overall features from the perspective of probability sparsity, which significantly reduces computational cost. Subsequently, a structural distilling block is proposed that applies the concept of patch merging to the distilling operation. The block reduces the size of the three-dimensional tensor and further lowers the memory cost while preserving the original data as much as possible. Thus, there was a significant improvement in the space complexity. Finally, the feature vector was projected into the classification target space for disease prediction. The effectiveness of the proposed model was validated using the Alzheimer’s Disease Neuroimaging Initiative dataset. The model achieved 97.97% and 91.89% accuracy on Alzheimer’s disease and mild cognitive impairment classification tasks, respectively. The experimental results also demonstrate that the proposed framework outperforms several state-of-the-art methods.

Abstract 3875: De novo spatial proteomic profiling of immune synapses using machine learning-guided microscoop

Abstract The spatial proteome of the immune synapse (IS) between a target cell and a lymphocyte is fundamentally important to understand the mechanism of cell-mediated immunity for both immuno-oncology and therapeutic applications. In this research, we used Microscoop, a fully-automatic microscope system integrated with a machine learning-based algorithm, to best determine ISs for proteomic mapping. We used Raji B cells as antigen-presenting cells (APCs) and induced the formation of ISs by incubating with Jurkat T cells. Multiple IS images were applied to train our algorithm using convolution neural network-based deep learning. A sequential process including fluorescence imaging, deep learning-enabled pattern generation, and photochemical labeling was implemented to achieve spatial-specific biotinylation of the IS proteins. Moreover, Microscoop is capable to repeat the process automatically on thousands of fields of view to label sufficient immune-synaptic proteins with biotin-tag for further enrichment allowing protein identification with mass spectrometry. We have successfully labeled and isolated proteins from spatially reorganized interfaces between T cells and APCs. Following MS-based proteomic analysis, several hundreds of proteins were identified, including the proteins known to be specifically associated with T-cell receptor (TCR) activation such as LCK, one of the major factors involved in TCR signaling at ISs. More interestingly, we identified several proteins novel for ISs, including proteins involved in phosphatidylinositol signaling. Our data showcases the capability of subcellular de novo spatial proteomics of the Microscoop technology, revealing novel factors responsible for initiating the immune response of a lymphocyte and shedding light on immune checkpoint signaling and tumor immunotherapy. Citation Format: You-Pi Liu, Weng Man Chong, Harry Huang, Yi-De Chen, Chia-Wen Chung, Hsiao-Jen Chang, Chih-Wei Chang, Jung-Chi Liao. De novo spatial proteomic profiling of immune synapses using machine learning-guided microscoop [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 3875.

The Application of RBF Neural Network Model Based on Deep Learning for Flower Pattern Design in Art Teaching.

The rapid growth of artificial intelligence technology has been deployed in art teaching and learning. Radial basis function (RBF) networks have a completely different design compared to most neural network architectures. Most neural networks consist of multiple layers that can introduce nonlinearity by repetitive application of nonlinear activation functions. In this research, people will study the application of the RBF neural network model based on deep learning in flower pattern design in art teaching. The image classification process is finding and labeling groups of pixels or vectors inside an image based on rules. Deep learning is a type of machine learning that uses artificial neural networks to replicate the structure and function of the human brain. The proposed model uses the RBF neural network-based deep learning model in flower pattern design in art teaching and provides efficient results.

Feasibility of Markerless Motion Capture for Three-Dimensional Gait Assessment in Community Settings.

Three-dimensional (3D) kinematic analysis of gait holds potential as a digital biomarker to identify neuropathologies, monitor disease progression, and provide a high-resolution outcome measure to monitor neurorehabilitation efficacy by characterizing the mechanisms underlying gait impairments. There is a need for 3D motion capture technologies accessible to community, clinical, and rehabilitation settings. Image-based markerless motion capture (MLMC) using neural network-based deep learning algorithms shows promise as an accessible technology in these settings. In this study, we assessed the feasibility of implementing 3D MLMC technology outside the traditional laboratory environment to evaluate its potential as a tool for outcomes assessment in neurorehabilitation. A sample population of 166 individuals aged 9–87 years (mean 43.7, S.D. 20.4) of varied health history were evaluated at six different locations in the community over a 3-month period. Participants walked overground at self-selected (SS) and fastest comfortable (FC) speeds. Feasibility measures considered the expansion, implementation, and practicality of this MLMC system. A subset of the sample population (46 individuals) walked over a pressure-sensitive walkway (PSW) concurrently with MLMC to assess agreement of the spatiotemporal gait parameters measured between the two systems. Twelve spatiotemporal parameters were compared using mean differences, Bland-Altman analysis, and intraclass correlation coefficients for agreement (ICC2,1) and consistency (ICC3,1). All measures showed good to excellent agreement between MLMC and the PSW system with cadence, speed, step length, step time, stride length, and stride time showing strong similarity. Furthermore, this information can inform the development of rehabilitation strategies targeting gait dysfunction. These first experiments provide evidence for feasibility of using MLMC in community and clinical practice environments to acquire robust 3D kinematic data from a diverse population. This foundational work enables future investigation with MLMC especially its use as a digital biomarker of disease progression and rehabilitation outcome.

Deep Learning-Based Automated Detection of Arterial Vessel Wall and Plaque on Magnetic Resonance Vessel Wall Images.

PurposeTo develop and evaluate an automatic segmentation method of arterial vessel walls and plaques, which is beneficial for facilitating the arterial morphological quantification in magnetic resonance vessel wall imaging (MRVWI).MethodsMRVWI images acquired from 124 patients with atherosclerotic plaques were included. A convolutional neural network-based deep learning model, namely VWISegNet, was used to extract the features from MRVWI images and calculate the category of each pixel to facilitate the segmentation of vessel wall. Two-dimensional (2D) cross-sectional slices reconstructed from all plaques and 7 main arterial segments of 115 patients were used to build and optimize the deep learning model. The model performance was evaluated on the remaining nine-patient test set using the Dice similarity coefficient (DSC) and average surface distance (ASD).ResultsThe proposed automatic segmentation method demonstrated satisfactory agreement with the manual method, with DSCs of 93.8% for lumen contours and 86.0% for outer wall contours, which were higher than those obtained from the traditional U-Net, Attention U-Net, and Inception U-Net on the same nine-subject test set. And all the ASD values were less than 0.198 mm. The Bland–Altman plots and scatter plots also showed that there was a good agreement between the methods. All intraclass correlation coefficient values between the automatic method and manual method were greater than 0.780, and greater than that between two manual reads.ConclusionThe proposed deep learning-based automatic segmentation method achieved good consistency with the manual methods in the segmentation of arterial vessel wall and plaque and is even more accurate than manual results, hence improved the convenience of arterial morphological quantification.

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

Objective. Real-time imaging is highly desirable in image-guided radiotherapy, as it provides instantaneous knowledge of patients’ anatomy and motion during treatments and enables online treatment adaptation to achieve the highest tumor targeting accuracy. Due to extremely limited acquisition time, only one or few x-ray projections can be acquired for real-time imaging, which poses a substantial challenge to localize the tumor from the scarce projections. For liver radiotherapy, such a challenge is further exacerbated by the diminished contrast between the tumor and the surrounding normal liver tissues. Here, we propose a framework combining graph neural network-based deep learning and biomechanical modeling to track liver tumor in real-time from a single onboard x-ray projection. Approach. Liver tumor tracking is achieved in two steps. First, a deep learning network is developed to predict the liver surface deformation using image features learned from the x-ray projection. Second, the intra-liver deformation is estimated through biomechanical modeling, using the liver surface deformation as the boundary condition to solve tumor motion by finite element analysis. The accuracy of the proposed framework was evaluated using a dataset of 10 patients with liver cancer. Main results. The results show accurate liver surface registration from the graph neural network-based deep learning model, which translates into accurate, fiducial-less liver tumor localization after biomechanical modeling (<1.2 (±1.2) mm average localization error). Significance. The method demonstrates its potentiality towards intra-treatment and real-time 3D liver tumor monitoring and localization. It could be applied to facilitate 4D dose accumulation, multi-leaf collimator tracking and real-time plan adaptation. The method can be adapted to other anatomical sites as well.

A Deep Learning Approach for Gait Event Detection from a Single Shank-Worn IMU: Validation in Healthy and Neurological Cohorts.

Many algorithms use 3D accelerometer and/or gyroscope data from inertial measurement unit (IMU) sensors to detect gait events (i.e., initial and final foot contact). However, these algorithms often require knowledge about sensor orientation and use empirically derived thresholds. As alignment cannot always be controlled for in ambulatory assessments, methods are needed that require little knowledge on sensor location and orientation, e.g., a convolutional neural network-based deep learning model. Therefore, 157 participants from healthy and neurologically diseased cohorts walked 5 m distances at slow, preferred, and fast walking speed, while data were collected from IMUs on the left and right ankle and shank. Gait events were detected and stride parameters were extracted using a deep learning model and an optoelectronic motion capture (OMC) system for reference. The deep learning model consisted of convolutional layers using dilated convolutions, followed by two independent fully connected layers to predict whether a time step corresponded to the event of initial contact (IC) or final contact (FC), respectively. Results showed a high detection rate for both initial and final contacts across sensor locations (recall , precision ). Time agreement was excellent as witnessed from the median time error (0.005 s) and corresponding inter-quartile range (0.020 s). The extracted stride-specific parameters were in good agreement with parameters derived from the OMC system (maximum mean difference 0.003 s and corresponding maximum limits of agreement (−0.049 s, 0.051 s) for a confidence level). Thus, the deep learning approach was considered a valid approach for detecting gait events and extracting stride-specific parameters with little knowledge on exact IMU location and orientation in conditions with and without walking pathologies due to neurological diseases.

Leveraging data analytics to understand the relationship between restaurants’ safety violations and COVID-19 transmission

This paper leverages natural language processing, spatial analysis, and statistical analysis to examine the relationship between restaurants’ safety violations and COVID-19 cases. We used location-based consumers’ complaints data during the early stage of business reopening in Florida, USA. First, statistical analysis was conducted to examine the correlation between restaurants’ safety violations and COVID-19 transmission. Second, a neural network-based deep learning model was developed to perform topic modeling based on consumers’ complaints. Third, spatial modeling of the complaints’ geographic distributions was performed to identify the hotspots of consumers’ complaints and COVID-19 cases. The results reveal a positive relationship between consumers’ complaints about restaurants’ safety violations and COVID-19 cases. In particular, consumers’ complaints about personal protection measures had the highest correlation with COVID-19 cases, followed by environmental safety measures. Our analytical methods and findings shed light on customers’ behavioral shifts and hospitality businesses’ adaptive practices during a pandemic.

Machine learning-based classification of woodland bitter vine (Mikania micrantha Kunth)

The woodland invasive species bitter vine (Mikania micrantha Kunth) is widely distributed in southern Asia. Since it adversely affects vegetation growth and ecological functions, it is imperative to develop an automated detection method to investigate the species’ expansion in large areas. We obtained training and test samples in the Zengcheng woodland, Guangzhou Province, China, and used aerial imagery to classify bitter vine using machine learning algorithms. The results suggested that visual interpretation was suitable for bitter vine classification but had slightly lower accuracy than methods using field-based training samples. The random forest (RF) algorithm proved most efficient for large-scale detection with the highest overall classification accuracy (99.62% for the test set and 99.92% for the training set) and Kappa coefficient (0.9878 for the test set and 0.9975 for the training set). Although the backpropagation neural network-based deep learning (DL) approach showed uncertainties and errors in this study, a more advanced DL method could be suitable for this task but requires further research. Bitter vine occurs predominantly in disturbed areas with low vegetation cover. Therefore, monitoring, silvicultural measures, and manual removal are necessary. Since bitter vine is highly distinguishable in images, future studies should further investigate supervised classification methods to map and monitor this species.

A feasibility study for in vivo treatment verification of IMRT using Monte Carlo dose calculation and deep learning-based modelling of EPID detector response

BackgroundThis paper describes the development of a predicted electronic portal imaging device (EPID) transmission image (TI) using Monte Carlo (MC) and deep learning (DL). The measured and predicted TI were compared for two-dimensional in vivo radiotherapy treatment verification.MethodsThe plan CT was pre-processed and combined with solid water and then imported into PRIMO. The MC method was used to calculate the dose distribution of the combined CT. The U-net neural network-based deep learning model was trained to predict EPID TI based on the dose distribution of solid water calculated by PRIMO. The predicted TI was compared with the measured TI for two-dimensional in vivo treatment verification.ResultsThe EPID TI of 1500 IMRT fields were acquired, among which 1200, 150, and 150 fields were used as the training set, the validation set, and the test set, respectively. A comparison of the predicted and measured TI was carried out using global gamma analyses of 3%/3 mm and 2%/2 mm (5% threshold) to validate the model's accuracy. The gamma pass rates were greater than 96.7% and 92.3%, and the mean gamma values were 0.21 and 0.32, respectively.ConclusionsOur method facilitates the modelling process more easily and increases the calculation accuracy when using the MC algorithm to simulate the EPID response, and has potential to be used for in vivo treatment verification in the clinic.

Rupture risk prediction of cerebral aneurysms using a novel convolutional neural network-based deep learning model

BackgroundCerebral aneurysms should be treated before rupture because ruptured aneurysms result in serious disability. Therefore, accurate prediction of rupture risk is important and has been estimated using various hemodynamic factors.ObjectiveTo...

Artificial Neuron Networks Enabled Identification and Characterizations of 2D Materials and van der Waals Heterostructures.

Two-dimensional (2D) materials and their in-plane and out-of-plane (i.e., van der Waals, vdW) heterostructures are promising building blocks for next-generation electronic and optoelectronic devices. Since the performance of the devices is strongly dependent on the crystalline quality of the materials and the interface characteristics of the heterostructures, a fast and nondestructive method for distinguishing and characterizing various 2D building blocks is desirable to promote the device integrations. In this work, based on the color space information on 2D materials' optical microscopy images, an artificial neural network-based deep learning algorithm is developed and applied to identify eight kinds of 2D materials with accuracy well above 90% and a mean value of 96%. More importantly, this data-driven method enables two interesting functionalities: (1) resolving the interface distribution of chemical vapor deposition (CVD) grown in-plane and vdW heterostructures and (2) identifying defect concentrations of CVD-grown 2D semiconductors. The two functionalities can be utilized to quickly identify sample quality and optimize synthesis parameters in the future. Our work improves the characterization efficiency of atomically thin materials and is therefore valuable for their research and applications.

Machine Translation System Using Deep Learning for English to Urdu.

Machine translation is an ongoing field of research from the last decades. The main aim of machine translation is to remove the language barrier. Earlier research in this field started with the direct word-to-word replacement of source language by the target language. Later on, with the advancement in computer and communication technology, there was a paradigm shift to data-driven models like statistical and neural machine translation approaches. In this paper, we have used a neural network-based deep learning technique for English to Urdu languages. Parallel corpus sizes of around 30923 sentences are used. The corpus contains sentences from English-Urdu parallel corpus, news, and sentences which are frequently used in day-to-day life. The corpus contains 542810 English tokens and 540924 Urdu tokens, and the proposed system is trained and tested using 70 : 30 criteria. In order to evaluate the efficiency of the proposed system, several automatic evaluation metrics are used, and the model output is also compared with the output from Google Translator. The proposed model has an average BLEU score of 45.83.

Convolutional Neural Network-Based Intelligent Decision-Making for Automated Vehicles

The decision-making module provides proper driving maneuvers for Automated vehicles (AVs), which is essential to the deployment of AVs. Current research works attempted to tackle this issue by optimizing some specific significant objectives or modeling separate typical maneuvers. However, highly dynamic and complicated traffic flows bring huge challenges to this module. This paper develops a convolutional neural network-based (CNN) deep learning framework to learn proper driving maneuvers from bulks of the collected traffic data. The proposed method firstly covers the dynamic traffic information through constructing virtual traffic images, namely dynamic motion images. Those images contain important motion relationships between the ego car and its neighbors. Then, the proposed CNN learning model is trained to capture inherent features from those motion images and to learn to make proper driving behaviors. We employ the learning framework to learn typical highway driving scenarios. Training and test results validate the effectiveness of our proposed CNN-based learning decision-making approach and suggest that deep-learning-based methods have dramatic potential to promote the intelligent driving capability of AVs and thus accelerate the launch of AVs.

Real‐world multimodal lifelog dataset for human behavior study

To understand the multilateral characteristics of human behavior and physiological markers related to physical, emotional, and environmental states, extensive lifelog data collection in a real-world environment is essential. Here, we propose a data collection method using multimodal mobile sensing and present a long-term dataset from 22 subjects and 616 days of experimental sessions. The dataset contains over 10 000 hours of data, including physiological, data such as photoplethysmography, electrodermal activity, and skin temperature in addition to the multivariate behavioral data. Furthermore, it consists of 10 372 user labels with emotional states and 590 days of sleep quality data. To demonstrate feasibility, human activity recognition was applied on the sensor data using a convolutional neural network-based deep learning model with 92.78% recognition accuracy. From the activity recognition result, we extracted the daily behavior pattern and discovered five representative models by applying spectral clustering. This demonstrates that the dataset contributed toward understanding human behavior using multimodal data accumulated throughout daily lives under natural conditions.

Hybrid gated recurrent unit and convolutional neural network-based deep learning architecture-based visibility improvement scheme for improving fog-degraded images

Hybrid gated recurrent unit and convolutional neural network-based deep learning architecture-based visibility improvement scheme for improving fog-degraded images