Deep Learning Architecture Research Articles

Time-Series Embeddings from Language Models: A Tool for Wind Direction Nowcasting

Wind direction nowcasting is crucial in various sectors, particularly for ensuring aviation operations and safety. In this context, the TELMo (Time-series Embeddings from Language Models) model, a sophisticated deep learning architecture, has been introduced in this work for enhanced wind-direction nowcasting. Developed by using three years of data from multiple stations in the complex terrain of an international airport, TELMo incorporates the horizontal u (east–west) and v (north–south) wind components to significantly reduce forecasting errors. On a day with high wind direction variability, TELMo achieved mean absolute error values of 5.66 for 2-min, 10.59 for 10-min, and 14.79 for 20-min forecasts, processed within a swift 9-ms/step timeframe. Standard degree-based analysis, in comparison, yielded lower performance, emphasizing the effectiveness of the u and v components. In contrast, a Vanilla neural network, representing a shallow-learning approach, underperformed in all analyses, highlighting the superiority of deep learning methodologies in wind direction nowcasting. TELMo is an efficient model, capable of accurately forecasting wind direction for air traffic operations, with an error less than 20° in 97.49% of the predictions, aligning with recommended international thresholds. This model design enables its applicability across various geographical locations, making it a versatile tool in global aviation meteorology.

Malaria cell identification using improved machine learning and modified deep learning architecture

Malaria continues to be a serious problem for public health because of its occurrence in tropical and subtropical areas with inadequate healthcare systems and few resources. For prompt intervention and treatment of malaria, effective and precise diagnosis is essential. Professional pathologists examine blood smear films by hand to get a microscopic diagnosis and another way they will do a rapid antigen malaria test which produces the result of 50% accuracy. Convolutional neural network (CNN) is a type of deep learning (DL) model that has been effectively used for a variety of image recognition applications. Our suggested approach uses, improved machine learning (IML) methods like support vector machine (SVM)+principal component analysis (PCA) fit, SVM+t-distributed stochastic neighbor embedding (t-SNE) fit, and CNN architecture with an accuracy of 86.23%, 88.27%, and 97.16% accuracy respectively, to combine feature extraction, data augmentation, and modify the layers by including the SVM algorithm in the final layer of the CNN architecture. The proposed method will significantly reduce pathologists' burden by automating the identification of malaria and improving diagnosis accuracy in resourceconstrained contexts

Improving Sentiment Analysis With Neural Networks

Abstract This paper investigates the effectiveness of sentiment analysis (SA) methods, ranging from rule-based approaches to deep learning architectures, in analysing textual data. The study focuses on three Python libraries: TextBlob, VADER, and Flair, evaluating their accuracy on a public dataset of Twitter posts. Additionally, custom neural network architectures are developed to optimize sentiment classification. Results indicate that while rule-based libraries offer simplicity, deep learning-based libraries show promise for higher accuracy. The customized LSTM models, particularly LSTM2 with architectural adjustments and regularization techniques, demonstrate improved performance over baseline models with classification accuracy as high as 76.3%.

Enhanced deep learning based decision support system for kidney tumour detection

This study presents a high-accuracy deep learning-based decision support system for kidney cancer detection. The research utilizes a relatively large dataset of 10,000 CT images, including both healthy and tumour-detected kidney scans. After data preprocessing and optimization, various deep learning models were evaluated, with DenseNet-201 emerging as the top performer, achieving an accuracy of 99.75 %. The study compares multiple deep learning architectures, including AlexNet, EfficientNet, Darknet-53, Xception, and DenseNet-201, across different learning rates. Performance metrics such as accuracy, precision, sensitivity, F1-score, and specificity are analysed using confusion matrices. The proposed system outperforms different deep learning networks, demonstrating superior accuracy in kidney cancer detection. The improvement is attributed to effective data engineering and hyperparameter optimization of the deep learning networks. This research contributes to the field of medical image analysis by providing a robust decision support tool for early and rapid diagnosis of kidney cancer. The high accuracy and efficiency of the proposed system make it a promising aid for healthcare professionals in clinical settings.

Spatio‐Temporal Machine Learning for Regional to Continental Scale Terrestrial Hydrology

AbstractIntegrated hydrologic models can simulate coupled surface and subsurface processes but are computationally expensive to run at high resolutions over large domains. Here we develop a novel deep learning model to emulate subsurface flows simulated by the integrated ParFlow‐CLM model across the contiguous US. We compare convolutional neural networks like ResNet and UNet run autoregressively against our novel architecture called the Forced SpatioTemporal RNN (FSTR). The FSTR model incorporates separate encoding of initial conditions, static parameters, and meteorological forcings, which are fused in a recurrent loop to produce spatiotemporal predictions of groundwater. We evaluate the model architectures on their ability to reproduce 4D pressure heads, water table depths, and surface soil moisture over the contiguous US at 1 km resolution and daily time steps over the course of a full water year. The FSTR model shows superior performance to the baseline models, producing stable simulations that capture both seasonal and event‐scale dynamics across a wide array of hydroclimatic regimes. The emulators provide over 1,000× speedup compared to the original physical model, which will enable new capabilities like uncertainty quantification and data assimilation for integrated hydrologic modeling that were not previously possible. Our results demonstrate the promise of using specialized deep learning architectures like FSTR for emulating complex process‐based models without sacrificing fidelity.

SurvNet: A low-complexity convolutional neural network for survival time classification of patients with glioblastoma

Background and objectiveMalignant primary brain tumors cause the greatest number of years of life lost than any other cancer. Grade 4 glioma is particularly devastating: The median survival without any treatment is less than six months and with standard-of-care treatment is only 14.6 months. Accurate identification of the overall survival time of patients with brain tumors is of profound importance in many clinical applications. Automated image analytics with magnetic resonance imaging (MRI) can provide insights into the prognosis of patients with brain tumors. MethodsIn this paper, We propose SurvNet, a low-complexity deep learning architecture based on the convolutional neural network to classify the overall survival time of patients with brain tumors into long-time and short-time survival cohorts. Through the incorporation of diverse MRI modalities as inputs, we facilitate deep feature extraction at various anatomical sites, thereby augmenting the precision of predictive modeling. We compare SurvNet with the Inception V3, VGG 16 and ensemble CNN models on pre-operative magnetic resonance image datasets. We also analyzed the effect of segmented brain tumors and training data on the system performance. ResultsSeveral measures, such as accuracy, precision, and recall, are calculated to examine the perfor-mance of SurvNet on three-fold cross-validation. SurvNet with T1 MRI modality achieved a 62.7 % accuracy, compared with 52.9 % accuracy of the Inception V3 model, 58.5 % accuracy of the VGG 16 model, and 54.9 % of the ensemble CNN model. By increasing the MRI input modalities, SurvNet becomes more accurate and achieves 76.5 % accuracy with four MRI modalities. Combining the segmented data, SurvNet achieved the highest accuracy of 82.4 %. ConclusionsThe research results show that SurvNet achieves higher metrics such as accuracy and f1-score than the comparisons. Our research also proves that by using multiparametric MRI modalities, SurvNet is able to learn more image features and performs a better classification accuracy. We can conclude that SurvNet with the complete scenario, i.e., segmented data and four MRI modalities, achieved the best accuracy, showing the validity of segmentation information during the survival time prediction process.

Deep learning for personalized health monitoring and prediction: A review

AbstractPersonalized health monitoring and prediction are indispensable in advancing healthcare delivery, particularly amidst the escalating prevalence of chronic illnesses and the aging population. Deep learning (DL) stands out as a promising avenue for crafting personalized health monitoring systems adept at forecasting health outcomes with precision and efficiency. As personal health data becomes increasingly accessible, DL‐based methodologies offer a compelling strategy for enhancing healthcare provision through accurate and timely prognostications of health conditions. This article offers a comprehensive examination of recent advancements in employing DL for personalized health monitoring and prediction. It summarizes a diverse range of DL architectures and their practical implementations across various realms, such as wearable technologies, electronic health records (EHRs), and data accumulated from social media platforms. Moreover, it elucidates the obstacles encountered and outlines future directions in leveraging DL for personalized health monitoring, thereby furnishing invaluable insights into the immense potential of DL in this domain.

Unsteady flow-field forecasting leveraging a hybrid deep-learning architecture

With the rapid development of data-driven technology, deep learning has been widely used to study unsteady flow phenomena, owing to its high-dimensional, nonlinear, and excellent big-data processing capabilities. Rapidly and accurately predicting unsteady flow fields has been a major challenge in fluid mechanics. Therefore, we designed a new U-shaped hybrid deep neural network (UDNN) framework using a multilayer convolution. Through the multilayer learning interaction of high-dimensional flow-field data, the temporal and spatial characteristics of the flow-field flow are captured, and the flow-field characteristics are predicted in an end-to-end form. The UDNN comprises a convolutional neural layer, deconvolutional layer, convolutional long-term and short-term layers, and attention-mechanism layer. First, based on computational fluid dynamics, we generated unsteady flow-field datasets of the flow around fixed and rotating cylinders at different Reynolds numbers, which were used as training samples for the network framework. Second, we designed a U-shaped convolutional layer, added horizontal time-series feature processing and attention-mechanism units, and fused the deep feature information predicted by the model with shallow semantic information to predict the flow-field features. In addition, we compared the UDNN, proper orthogonal decomposition – long short-term memory, and traditional convolutional autoencoder – long short-term memory models in terms of the flow-field prediction error, model training time, and inference speed. The final results showed that the proposed UDNN framework achieved high accuracy and strong robustness in predicting unsteady flow fields.

A VMD-deep learning approach for individual load monitoring and forecasting for residential buildings energy management

This paper presents a comprehensive solution for Multi-Individual load disaggregation and Multi-individual load Forecasting in residential energy systems. The approach combines Variable Mode Decomposition (VMD) and deep learning algorithms, incorporating feature selection using Random Forest Regressor and parameter optimization through the Salp Swarm Algorithm (SSA). Correlation analysis is used to identify the most relevant Intrinsic Mode Functions (IMFs) for each appliance, while self-correlation is used to determine the past window for forecasting appliances’ usage a day ahead. The robustness of VMD against noise contributes to the improved accuracy of appliance designation. The load disaggregation strategy yields significant results, indicating the high performance of the disaggregation tasks of these models. Specifically, the proposed disaggregation models exhibit an average RMSE and MAPE of 4.10 W and 4.14 %, respectively. Furthermore, we conduct a comparison among three deep learning architectures (GRU, LSTM, 1DCNN) based Multi-Individual load forecasting models, revealing that the GRU-based MILF model outperforms the other two models, it shows an average performance of RMSE and MAPE of 9.54 W and 8.10 %. The research provides valuable insights and a practical solution for accurate load disaggregation and forecasting, contributing to enhanced energy efficiency, cost savings, and sustainable power consumption in residential settings, and give detailed energetic bill with the aim to analyze the occupant consumption behavior, and gives depth insights about power consumption and cost of each appliance and their operation time.

Forecasting Bitcoin returns: Econometric time series analysis vs. machine learning

AbstractWe study the statistical properties of the Bitcoin return series and provide a thorough forecasting exercise. Also, we calibrate state‐of‐the‐art machine learning techniques and compare the results with econometric time series models. The empirical assessment provides evidence that the application of machine learning techniques outperforms econometric benchmarks in terms of forecasting precision for both in‐ and out‐of‐sample forecasts. We find that both deep learning architectures as well as complex layers, such as LSTM, do not increase the precision of daily forecasts. Specifically, a simple recurrent neural network describes a sensible choice for forecasting daily return series.

Summarization and Translation Using NLP

Abstract: Text summarization and translation are two critical tasks in natural language processing with significant applications in various domains such as news aggregation, document summarization, machine translation, and information retrieval. In recent years, there has been remarkable progress in the development of techniques and models for both tasks, leveraging advancements in deep learning and neural network architectures. This paper presents a comprehensive review and comparative analysis of state-of-the-art methods in text summarization and translation. First, we provide an overview of the different approaches to text summarization, including extractive, abstractive, and hybrid methods, highlighting their strengths and weaknesses. We discuss various evaluation metrics and datasets commonly used for benchmarking summarization systems, shedding light on the challenges and opportunities in this field.

Brain Tumor Detection Using Deep Learning Technique

Abstract: One of the most aggressive disorders affecting both children and adults is brain tumors. Brain tumors develop very quickly, and if not treated at the proper time, they decrease the patient's chances of survival. It is crucial to find brain tumors at an early stage. To increase patients’ life expectancy, proper treatment planning and precise diagnostics are most important. Magnetic Resonance Imaging (MRI) is the most effective method for finding brain tumors. Therefore, to find the types of tumors, an automated brain tumor detection system is needed [1]. This work uses Deep Learning (DL) architectures like Convolutional Neural Network (CNN) and EfficientNetB0 for Transfer Learning to detect the brain tumor. This model is used to predict the types of brain tumors

CNN-LSTM for Heartbeat Sound Classification

Cardiovascular disorders are among the primary causes of death. Regularly monitoring the heart is of paramount importance in preventing fatalities arising from heart diseases. Heart disease monitoring encompasses various approaches, including the analysis of heartbeat sounds. The auditory patterns of a heartbeat can serve as indicators of heart health. This study aims to build a new model for categorizing heartbeat sounds based on associated ailments. The Phonocardiogram (PCG) method digitizes and records heartbeat sounds. By converting heartbeat sounds into digital data, researchers are empowered to develop a deep learning model capable of discerning heart defects based on distinct cardiac rhythms. This study proposes the utilization of Mel-frequency cepstral coefficients for feature extraction, leveraging their application in voice data analysis. These extracted features are subsequently employed in a multi-step classification process. The classification process merges a convolutional neural network (CNN) with a long short-term memory network (LSTM), forming a comprehensive deep learning architecture. This architecture is further enhanced through optimization utilizing the Adagrad optimizer. To examine the effectiveness of the proposed method, its classification performance is evaluated using the "Heartbeat Sounds" dataset sourced from Kaggle. Experimental results underscore the effectiveness of the proposed method by comparing it with simple CNN, CNN with vanilla LSTM, and traditional machine learning methods (MLP, SVM, Random Forest, and k-NN).

Stress-Level Detection Using RepVGG Neural Network

Stress used to impact our mental and physical well-being,productivity,and overall quality of life. Detecting stress accurately is vital for timely intervention and effective management. In this study, we introduce a new method for detecting stress levels using the RepVGG deep learning architecture. RepVGG stands out for its efficient performance and straightforward structure, making it ideal for analyzing physiological signals and other stress indicators. We using standard metrics to calculate the things like accuracy, precision, recall and etc. Ourfindings reveal that the RepVGG-based method excels in detecting stress levels, surpassing many traditional methods and other deep learning models. Moreover, the model shows strong generalization capabilities across various datasets and conditions. This research underscores the potential of advanced deep learning models like RepVGG in stress detection, opening doors for real-time, scalable, and precise stress monitoring systems. Looking ahead, weaim to integrate this model into wearable devices andmobile apps, enabling continuous stress monitoring and offering personalized stress management advice. Some approach utilizes a rich dataset consists of signals such (HRV) and (GSR), along with other relevant biomarkers. To ensure the model's robustness, we preprocess and augment this data. We then train the RepVGG architecture on this dataset, harnessing its Neural layers for feature extraction and its unique re-parameterizable design for efficient use.

Deepfake Video Detection Using Res-Next CNN and LSTM

Abstract: Deepfake videos are becoming more and more common in the digital age, which has led to major worries about how they can undermine the trustworthiness of visual media. With deep learning algorithms' rising processing power, producing lifelike human-synthesized films or deepfakes has never been easier. Disinformation and political unrest can be caused by these videos. A novel deep learning-based method has been created to distinguish between actual and AI-generated fraudulent videos to address this problem. The suggested technique uses Attention-based networks (Res-Next CNN), a kind of deep learning architecture that can selectively focus on significant features in a video, to fine-tune the transformer module to search for new sets of feature space to detect false images.

Cloud Cover Segmentation and Motion Prediction from Satellite Imagery

Abstract: This research project is dedicated to achieving precise identification, segmentation, and motion prediction of cloud formations in satellite imagery. Leveraging the powerful U-Net, a renowned deep learning architecture for image segmentation, is crucial in addressing the intricate challenge of cloud detection and segmentation within remote sensing imagery. The automation of cloud identification processes within the project not only enhances weather forecasting capabilities but also contributes significantly to advancements in climate monitoring and environmental analysis. The incorporation of Long ShortTerm Memory (LSTM) networks facilitates cloud motion prediction, providing insights into the dynamic behavior of cloud formations over time. The robustness of the U-Net model, coupled with the enhanced capability of capturing intricate patterns and predicting cloud motion, establishes it as a valuable and comprehensive tool. This contributes to improving the accuracy and efficiency of cloud segmentation in satellite imagery, fostering progress in critical domains such as environmental research and meteorological applications

Klasifikasikan Jenis Cacat Kulit Menggunakan SMOTE-GoogLeNet

Deep learning has been proven to be able to provide significant contributions to several fields, including industry. It has also been proven that it has resulted in an outstanding performance for classification, detection, and even segmentation processes. In the leather industry, it also successfully gave valuable results, especially for the leather defect inspection process. However, despite its outstanding performance, it remained a drawback because it produced insignificant results if employed in a small or imbalanced dataset. This research work focuses on the analysis of the implementation of the data balancing method for improving the performance of the deep learning method for classifying the types of leather defects. This research work was done by employing three processes. In the first step, we utilized the data balancing method to balance the data proportion. In the next step, we employed GoogLeNet as a deep learning architecture for training and testing processes. Our experiment was conducted in two scenarios. The first scenario was done by using the original dataset. Whereas the second scenario was accomplished by utilizing the data balancing method before training and testing. According to the experiment results, implementing the data balancing method successfully increased the performance of the deep learning method by more than 15%. It can be inferred that the proportion or the number of data strongly affected the performance of deep learning models.

Residual Echo State Networks: Residual recurrent neural networks with stable dynamics and fast learning

Residual connections have been established as a staple for modern deep learning architectures. Most of their applications are cast towards feedforward computing. In this paper, we study the architectural bias of residual connections in the context of recurrent neural networks (RNNs), specifically in the temporal dimension. We frame our discussion from the perspective of Reservoir Computing and dynamical system theory, focusing on important aspects of neural computation like memory capacity, long-term information processing, stability, and nonlinear computation capability. Experiments corroborate the striking advantage brought by temporal residual connections for a plethora of different time series processing tasks, comprehending memory-based, forecasting, and classification problems.

Detection of forest disturbance across California using deep-learning on PlanetScope imagery

California forests have recently experienced record breaking wildfires and tree mortality from droughts, However, there is inadequate monitoring, and limited data to inform policies and management strategies across the state. Although forest surveys and satellite observations of forest cover changes exist at medium to coarse resolutions (30–500 m) annually, they remain less effective in mapping small disturbances of forest patches (<5 m) occurring multiple times a year. We introduce a novel method of tracking California forest cover using a supervised U-Net deep learning architecture and PlanetScope’s Visual dataset which provides 3-band RGB (Red, Green, and Blue) mosaicked imagery. We created labels of forest and non-forest to train the U-Net model to map tree cover based on a semi-unsupervised classification method. We then detected changes of tree cover and disturbance with the U-Net model, achieving an overall accuracy of 98.97% over training data set, and 95.5% over an independent validation dataset, obtaining a precision of 82%, and a recall of 74%. With the predicted tree cover mask, we created wall to wall monthly tree cover maps over California at 4.77 m resolution for 2020, 2021, and 2022. These maps were then aggregated in a post-processing step to develop annual maps of disturbance, while accounting for the time of disturbance and other confounding factors such as topography, phenological and snow cover variability. We compared our high-resolution disturbance maps with wildfire GIS survey data from CALFIRE, and satellite-based forest cover changes and achieved an F-1 score of 54% and 88% respectively. The results suggest that high-resolution maps capture variability of forest disturbance and fire that wildfire surveys and medium resolution satellite products cannot. From 2020 to 2021, California maintained 30,923.5 sq km of forest while 5,994.9 sq km were disturbed. The highest observed forest loss rate was located at the Sierra Nevada mountains at 21.4% of the forested area being disturbed between 2020 and 2021. Our findings highlight the strong potential of deep learning and high-resolution RGB optical imagery for mapping complex forest ecosystems and their changes across California, as well as the application of these techniques on a national to global scale.

Unravelling Stock Market Patterns: Analysis and Predictive Modelling Using Time Series and Deep Learning

Abstract: Within the dynamic landscape of the financial markets, where fortunes can shift with the wind, the precision of future prediction holds immense value. This research embarks on a quest to enhance the accuracy of stock market forecasts, venturing into the domain of advanced time series models. We shall closely examine three prominent methodologies: the venerable ARIMA, the multifaceted VARMA, and the potent deep learning architecture known as LSTM. Through a rigorous investigation of their strengths and limitations, we shall subject these models to both univariate and multivariate data extracted from the Indian stock market. This comparative analysis aims to glean invaluable insights into their predictive capabilities, ultimately paving the path for the development of even more sophisticated forecasting tools. By wielding these instruments with confidence, investors can navigate the intricate dynamics of the market with a newfound certainty and optimize their decision-making for greater returns. Our endeavor transcends mere numerical analysis; it seeks to illuminate the complex dance of the market, unravel its hidden patterns, and ultimately empower investors to gain the upper hand in this challenging yet potentially rewarding financial sphere.