Variety Of Deep Learning Models Research Articles

Efficient EEG Feature Learning Model Combining Random Convolutional Kernel with Wavelet Scattering for Seizure Detection.

Automatic seizure detection has significant value in epilepsy diagnosis and treatment. Although a variety of deep learning models have been proposed to automatically learn electroencephalography (EEG) features for seizure detection, the generalization performance and computational burden of such deep models remain the bottleneck of practical application. In this study, a novel lightweight model based on random convolutional kernel transform (ROCKET) is developed for EEG feature learning for seizure detection. Specifically, random convolutional kernels are embedded into the structure of a wavelet scattering network instead of original wavelet transform convolutions. Then the significant EEG features are selected from the scattering coefficients and convolutional outputs by analysis of variance (ANOVA) and minimum redundancy-maximum relevance (MRMR) methods. This model not only preserves the merits of the fast-training process from ROCKET, but also provides insight into seizure detection by retaining only the helpful channels. The extreme gradient boosting (XGboost) classifier was combined with this EEG feature learning model to build a comprehensive seizure detection system that achieved promising epoch-based results, with over 90% of both sensitivity and specificity on the scalp and intracranial EEG databases. The experimental comparisons showed that the proposed method outperformed other state-of-the-art methods for cross-patient and patient-specific seizure detection.

Automatic rating of incomplete hippocampal inversions evaluated across multiple cohorts

Incomplete Hippocampal Inversion (IHI), sometimes called hippocampal malrotation, is an atypical anatomical pattern of the hippocampus found in about 20% of the general population. IHI can be visually assessed on coronal slices of T1 weighted MR images, using a composite score that combines four anatomical criteria. IHI has been associated with several brain disorders (epilepsy, schizophrenia). However, these studies were based on small samples. Furthermore, the factors (genetic or environmental) that contribute to the genesis of IHI are largely unknown. Large-scale studies are thus needed to further understand IHI and their potential relationships to neurological and psychiatric disorders. However, visual evaluation is long and tedious, justifying the need for an automatic method. In this paper, we propose, for the first time, to automatically rate IHI. We proceed by predicting four anatomical criteria, which are then summed up to form the IHI score, providing the advantage of an interpretable score. We provided an extensive experimental investigation of different machine learning methods and training strategies. We performed automatic rating using a variety of deep learning models (”conv5-FC3”, ResNet and ”SECNN”) as well as a ridge regression. We studied the generalization of our models using different cohorts and performed multi-cohort learning. We relied on a large population of 2,008 participants from the IMAGEN study, 993 and 403 participants from the QTIM and QTAB studies as well as 985 subjects from the UKBiobank. We showed that deep learning models outperformed a ridge regression. We demonstrated that the performances of the ”conv5-FC3” network were at least as good as more complex networks while maintaining a low complexity and computation time. We showed that training on a single cohort may lack in variability while training on several cohorts improves generalization (acceptable performances on all tested cohorts including some that are not included in training). The trained models will be made publicly available should the manuscript be accepted.

Review of researches and technologies applicable to digitalization of the process of assessing the exterior of meat and dairy animals

To increase the efficiency of livestock farming, scientists are developing information and communication technologies aimed at digitalizing the process of assessing the exterior of animals. This review should improve understanding of the development steps of systems applicable to the digitalization of animal conformation assessment using computer vision and deep learning neural networks. The search focused on several topics: computer vision systems; training datasets; image acquisition systems; deep learning models; neural networks for training; performance parameters and system evaluation. Machine vision is an innovative solution by combining sensors and neural networks, providing a non-contact way to assess livestock conditions as cameras can replace human observation. Two approaches are used to obtain three-dimensional images for digitalization tasks in animal husbandry: shooting animals using one 3D camera fixed in one place, and shooting from different points using several 3D cameras that record images of animals and individual parts of their bodies, such like an udder. The features extracted from the images, called dorsal features, are used as input to the models. The reviewed publications used a variety of deep learning models, including CNN, DNN, R-CNN, and SSD, depending on the task. Similarly, neural networks such as EfficientNet, ShapeNet, DeepLabCut and RefineDet have been mainly used for animal health monitoring, while GoogleNet, AlexNet, NasNet, CapsNet, LeNet and ERFNet are mainly used for identification purposes.

A Method for Specifying Yoga Poses Based on Deep Learning, Utilizing OpenCV and Media Pipe Technologies

Yoga is a years discipline that calls for physical postures, mental focus, and deep breathing. Yoga practice can enhance stamina, power, serenity, flexibility, and well‐being. Yoga is currently a well-liked type of exercise worldwide. The foundation of yoga is good posture. Even though yoga offers many health advantages, poor posture can lead to issues including muscle sprains and pains. People have become more interested in working online than in person during the last few years. People who are accustomed to internet life and find it difficult to find the time to visit yoga studios benefit from our strategy. Using the web cameras in our system, the model categorizes the yoga poses, and the image is used as input. However, the media pipe library first skeletonizes that image. Utilizing a variety of deep learning models, the input obtained from the yoga postures is improved to improve the asana. The algorithms like VGG16 (Visual Geometric Group), VGG19, Convo2d, CNN.

An Emotion-Aware Multitask Approach to Fake News and Rumor Detection Using Transfer Learning

Social networking sites, blogs, and online articles are instant sources of news for internet users globally. However, in the absence of strict regulations mandating the genuineness of every text on social media, it is probable that some of these texts are fake news or rumors. Their deceptive nature and ability to propagate instantly can have an adverse effect on society. This necessitates the need for more effective detection of fake news and rumors on the web. In this work, we annotate four fake news detection (EFN) and rumor detection datasets with their emotion class labels using transfer learning. We show the correlation between the legitimacy of a text with its intrinsic emotion for fake news and rumor detection, and prove that even within the same emotion class, fake and real news are often represented differently, which can be used for improved feature extraction. Based on this, we propose a multitask framework for fake news and rumor detection, predicting both the emotion and legitimacy of the text. We train a variety of deep learning models in single-task (STL) and multitask settings for a more comprehensive comparison. We further analyze the performance of our multitask approach for EFN in cross-domain settings to verify its efficacy for better generalization across datasets, and to verify that emotions act as a domain-independent feature. Experimental results verify that our multitask models consistently outperform their STL counterparts in terms of accuracy, precision, recall, and F1 score, both for in-domain and cross-domain settings. We also qualitatively analyze the difference in performance in STL and multitask learning (MTL) models.

A classification method for soybean leaf diseases based on an improved ConvNeXt model

Deep learning technologies have enabled the development of a variety of deep learning models that can be used to detect plant leaf diseases. However, their use in the identification of soybean leaf diseases is currently limited and mostly based on machine learning methods. In this investigation an enhanced deep learning network model was developed to recognize soybean leaf diseases more accurately. The improved network model consists of three parts: feature extraction, attention calculation, and classification. The dataset used was first diversified through data augmentation operations such as random masking to enhance network robustness. An attention module was then used to generate feature maps at various depths. This increased the network’s focus on discriminative features, reduced background noise, and enabled the use of the LeakyReLu activation function in the attention module to prevent situations in which neurons fail to learn when the input is negative. Finally, the extracted features were then integrated using a fully connected layer, and the predicted disease category inferred to improve the classification accuracy of soybean leaf diseases. The average recognition accuracy of the improved network model for soybean leaf diseases was 85.42% both higher than the six deep learning comparison models (ConvNeXt (66.41%), ResNet50 (72.22%), Swin Transformer (77.00%), MobileNetV3 (67.27%), ShuffleNetV2 (59.89%), and SqueezeNet (72.92%)), thus proving the effectiveness of the improved method.The model proposed in this paper was also tested on the grapevine leaf dataset, and the performance ability of the improved network model remained due to other common network models, and overall the proposed network model was very effective in leaf disease identification.

Artificial intelligence with temporal features outperforms machine learning in predicting diabetes.

Diabetes mellitus type 2 is increasingly being called a modern preventable pandemic, as even with excellent available treatments, the rate of complications of diabetes is rapidly increasing. Predicting diabetes and identifying it in its early stages could make it easier to prevent, allowing enough time to implement therapies before it gets out of control. Leveraging longitudinal electronic medical record (EMR) data with deep learning has great potential for diabetes prediction. This paper examines the predictive competency of deep learning models in contrast to state-of-the-art machine learning models to incorporate the time dimension of risk. The proposed research investigates a variety of deep learning models and features for predicting diabetes. Model performance was appraised and compared in relation to predominant features, risk factors, training data density and visit history. The framework was implemented on the longitudinal EMR records of over 19K patients extracted from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN). Empirical findings demonstrate that deep learning models consistently outperform other state-of-the-art competitors with prediction accuracy of above 91%, without overfitting. Fasting blood sugar, hemoglobin A1c and body mass index are the key predictors of future onset of diabetes. Overweight, middle aged patients and patients with hypertension are more vulnerable to developing diabetes, consistent with what is already known. Model performance improves as training data density or the visit history of a patient increases. This study confirms the ability of the LSTM deep learning model to incorporate the time dimension of risk in its predictive capabilities.

Study on segmentation of blasting fragment images from open-pit mine based on U-CARFnet.

Bench blasting is the primary means of production in open-pit metal mines. The size of the resulting rock mass after blasting has a significant impact on production cost. Currently, the ore fragment size is obtained mainly through manual measurement or estimation with the naked eye, which is inefficient and inaccurate. This study proposes the U-CARFnet and U-Net models for segmenting blasting fragment images from open-pit mines based on an attention mechanism, residual learning module, and focal loss function. It compares this technique with traditional image segmentation ones and a variety of deep learning models to verify the efficacy of the proposed model. Experimental results show that the accuracy of the U-CARFnet model proposed in this paper reaches 97.11% in the performance evaluation, which shows better performance than the traditional image segmentation method. In this study, the U-CARFnet model is used in the application, and a superior performance is obtained, with an average segmentation error of 5.46%. The proposed approach provides an effective technique for statistically analyzing images of mine rock.

Targeted adversarial attacks on wind power forecasts

In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a long short-term memory (LSTM) network for predicting the power generation of individual wind farms and a convolutional neural network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model’s performance, as well as the extent to which the attacker’s goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.

Deep Learning-Based Yoga Posture Specification Using OpenCV and Media Pipe

Yoga is a years discipline that calls for physical postures, mental focus, and deep breathing. Yoga practice can enhance stamina, power, serenity, flexibility, and wellbeing. Yoga is currently a well-liked type of exercise worldwide. The foundation of yoga is good posture. Even though yoga offers many health advantages, poor posture can lead to issues including muscle sprains and pains. People have become more interested in working online than in person during the last few years. People who are accustomed to internet life and find it difficult to find the time to visit yoga studios benefit from our strategy. Using the web cameras in our system, the model categorizes the yoga poses, and the image is used as input. However, the media pipe library first skeletonizes that image. Utilizing a variety of deep learning models, the input obtained from the yoga postures is improved to improve the asana. On non-skeleton photos, VGG16, InceptionV3, NASNetMobile, YogaConvo2d, and also InceptionResNetV2 came in the order of highest validation accuracy. The proposed model YogaConvo2d with skeletal pictures, which is followed by VGG16, reports validation accuracy in contrast, NASNetMobile, InceptionV3, and InceptionResNetV2.

Enhancing Retinal Scan Classification: A Comparative Study of Transfer Learning and Ensemble Techniques

Ophthalmic diseases are a significant health concern globally, causing visual impairment and blindness in millions of people, particularly in dispersed populations. Among these diseases, retinal fundus diseases are a leading cause of irreversible vision loss, and early diagnosis and treatment can prevent this outcome. Retinal fundus scans have become an indispensable tool for doctors to diagnose multiple ocular diseases simultaneously. In this paper, the results of a variety of deep learning models (DenseNet-201, ResNet125V2, XceptionNet, EfficientNet-B7, MobileNetV2, and EfficientNetV2M) and ensemble learning approaches are presented, which can accurately detect 20 common fundus diseases by analyzing retinal fundus scan images. The proposed model is able to achieve a remarkable accuracy of 96.98% for risk classification and 76.92% for multi-disease detection, demonstrating its potential for use in clinical settings. By utilizing the proposed model, doctors can provide swift and accurate diagnoses to patients, improving their chances of receiving timely treatment and preserving their vision.

Spatiotemporal Learning of Dynamic Positron Emission Tomography Data Improves Diagnostic Accuracy in Breast Cancer

Positron emission tomography (PET) is a noninvasive imaging technology able to assess the metabolic or functional state of healthy and/or pathological tissues. In clinical practice, PET data are usually acquired statically and normalized for the evaluation of the standardized uptake value (SUV). In contrast, dynamic PET acquisitions provide information about radiotracer delivery to tissue, its interaction with the target and its physiological washout. The shape of the time activity curves (TACs) embeds tissue-specific biochemical properties. Conventionally, TACs are employed along with information about blood plasma activity concentration, i.e., the arterial input function, and tracer-specific compartmental models to obtain a full quantitative analysis of PET data. This method’s primary disadvantage is the requirement for invasive arterial blood sample collection throughout the whole PET scan. In this study, we employ a variety of deep learning models to illustrate the diagnostic potential of dynamic PET acquisitions of varying lengths for discriminating breast cancer lesions in the absence of arterial blood sampling compared to static PET only. Our findings demonstrate that the use of TACs, even in the absence of arterial blood sampling and even when using only a share of all timeframes available, outperforms the discriminative ability of conventional SUV analysis.

Explainability techniques applied to road traffic forecasting using Graph Neural Network models

In recent years, several new Artificial Intelligence methods have been developed to make models more explainable and interpretable. The techniques essentially deal with the implementation of transparency and traceability of black box machine learning methods. Black box refers to the inability to explain why the model turns the input into the output, which may be problematic in some fields. To overcome this problem, our approach provides a comprehensive combination of predictive and explainability techniques. Firstly, we compared statistical regression, classic machine learning and deep learning models, reaching the conclusion that models based on deep learning exhibit greater accuracy. Of the great variety of deep learning models, the best predictive model in spatio-temporal traffic datasets was found to be the Adaptive Graph Convolutional Recurrent Network. Regarding the explainability technique, GraphMask shows a notably higher fidelity metric than other methods. The integration of both techniques was tested by means of experimental results, concluding that our approach improves deep learning model accuracy, making such models more transparent and interpretable. It allows us to discard up to 95% of the nodes used, facilitating an analysis of its behavior and thus improving the understanding of the model.

A General Survey on Attention Mechanisms in Deep Learning

Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework consisting of a general attention model, uniform notation, and a comprehensive taxonomy of attention mechanisms. Furthermore, the various measures for evaluating attention models are reviewed, and methods to characterize the structure of attention models based on the proposed framework are discussed. Last, future work in the field of attention models is considered.

Stacking ensemble approach for DDoS attack detection in software-defined cyber–physical systems

Distributed Denial of Service (DDoS) attacks are common and increasing in frequency. It renders the service inaccessible to legitimate users and degrades network performance. A complex structural environment called a Cyber–Physical System (CPS) is created by combining computation, connectivity, and physical parameters. Software-Defined Networking (SDN) is an emerging architecture that separates the data plane from the network plane. A central logic control resides in the control plane, making SDN vulnerable to DDoS attacks. SDN design ideas are broadened and applied to create software-defined Cyber–Physical Systems. Deep learning powers many artificial intelligence apps and services, enhancing automation by performing cognitive tasks without human intervention. It can perform feature extraction and classification on both small and large datasets. This paper presents a variety of Deep Learning models for efficiently detecting DDoS attacks in the SD-CPS framework through a scalable and adaptable SDN-based architecture. We could determine which Deep Learning techniques work best under various attack scenarios by examining multiple Deep Learning techniques. The Deep Learning models performed above 99% accuracy in classifying binary and multiclass data over unknown traffic when tested on two recent security datasets, the SDN-specific dataset and the CICDDoS2019 dataset.

Individual Beef Cattle Identification Using Muzzle Images and Deep Learning Techniques.

Simple SummaryThe ability to identify individual animals has gained great interest in beef feedlots to allow for animal tracking and all applications for precision management of individuals. This study assessed the feasibility and performance of a total of 59 deep learning models in identifying individual cattle with muzzle images. The best identification accuracy was 98.7%, and the fastest processing speed was 28.3 ms/image. A dataset containing 268 US feedlot cattle and 4923 muzzle images was published along with this article. This study demonstrates the great potential of using deep learning techniques to identify individual cattle using muzzle images and to support precision beef cattle management.Individual feedlot beef cattle identification represents a critical component in cattle traceability in the supply food chain. It also provides insights into tracking disease trajectories, ascertaining ownership, and managing cattle production and distribution. Animal biometric solutions, e.g., identifying cattle muzzle patterns (unique features comparable to human fingerprints), may offer noninvasive and unique methods for cattle identification and tracking, but need validation with advancement in machine learning modeling. The objectives of this research were to (1) collect and publish a high-quality dataset for beef cattle muzzle images, and (2) evaluate and benchmark the performance of recognizing individual beef cattle with a variety of deep learning models. A total of 4923 muzzle images for 268 US feedlot finishing cattle (>12 images per animal on average) were taken with a mirrorless digital camera and processed to form the dataset. A total of 59 deep learning image classification models were comparatively evaluated for identifying individual cattle. The best accuracy for identifying the 268 cattle was 98.7%, and the fastest processing speed was 28.3 ms/image. Weighted cross-entropy loss function and data augmentation can increase the identification accuracy of individual cattle with fewer muzzle images for model development. In conclusion, this study demonstrates the great potential of deep learning applications for individual cattle identification and is favorable for precision livestock management. Scholars are encouraged to utilize the published dataset to develop better models tailored for the beef cattle industry.

Smart Road Damage Detection and Warning using Machine Learning

We present a neural network topology, as well as training and prediction algorithms, in this research. To create a safe road environment, we present a deep neural network technique to detect road surface deterioration conditions. For training and testing, we provide an image dataset as input. Various sorts of road anxiety are depicted in the photographs. The suggested approach is compared to a variety of deep learning models from different disciplines. The findings of this study are expected to play a significant role in guaranteeing safe driving in the future by effectively detecting poor road conditions.

SDDA: a method of surface defect data augmentation of hot-rolled strip steel

The detection of surface defects in hot-rolled strip steel is an important step in the production process of hot-rolled strip steel, and it is closely related to the final quality. At present, many researchers have conducted many in-depth studies on the classification of strip steel surface defects and achieved fruitful results. However, current research has always focused on modifying the model; this approach has limited improvement in classification accuracy and often brings a reduction in model efficiency. A data augmentation method named SDDA, which can perform multiscale transformations on the images input to the deep learning models, is proposed and validated on X-SDD dataset by MixNet model. The test results show that after the SDDA data augmentation, the accuracy of the model reached 96.81%, which is 1.71% higher than the previous best result; meanwhile, since the improvement occurs on the data side, it only leads to an increase in training time and does not increase the inference time of the model. Through comparison experiments, it can be found that our data augmentation approach provides significant improvements in multiple metrics such as accuracy and F1 for a variety of deep learning models.

Using deep learning techniques, a framework for estimating the nutritional value of food in real time

Dietary disorders have increased dramatically in recent decades as a result of poor eating habits. Mobile-based dietary evaluation systems that can take real-time pictures of meals and assess their nutritional content might be very helpful in changing dietary habits and, as a result, leading to a healthier lifestyle. By categorizing the input image of food, this article offers a unique approach for automatically estimating food properties such as components and nutritional value. For reliable food recognition, we use a variety of deep learning models. Aside from image analysis, features and components are computed using semantically relevant phrases extracted from a vast corpus of literature acquired over the Internet. We performed testing on a dataset of 100 classes with an average of 1000 images per class, and we were able to get a top 1 classification rate of up to 85%. As an extension of the benchmark dataset Food-101, a sub-continental food dataset has been created. According to the findings, our proposed technique is equally effective on the core Food-101 dataset and its expansion for sub-continental foods. The proposed solution is implemented as a mobile app for the healthcare business.

Accelerating Deep Learning Using Interconnect-Aware UCX Communication for MPI Collectives

Deep learning workloads on modern multi-graphics processing unit (GPU) nodes are highly dependent on intranode interconnects, such as NVLink and PCIe, for high-performance communication. In this article, we take on the challenge to design an interconnect-aware multipath GPU-to-GPU communication using unified communication X (UCX) to utilize all available bandwidth for both NVLink-based systems and those that use a mixture of NVLink and PCIe. Our proposed multipath data transfer mechanism pipelines and stripes the message across multiple intrasocket communication channels and memory regions to achieve 1.84× higher bandwidth for Open message passing interface (MPI) on NVLink-based systems and 1.23× on NVLink and PCIe systems. We then utilize this mechanism to propose a three-stage hierarchical, pipelined MPI_Allreduce design as well as a flat pipelined two-stage algorithm for two different node topologies. For large messages, our proposed algorithms achieve a high speedup when compared to other MPI implementations. We also observe significant speedup for the proposed MPI_Allreduce with Horovod + TensorFlow with a variety of deep learning models.