Deep Convolutional Architecture Research Articles

An intelligent predictive and optimized wastewater treatment plant.

The intelligent predictive and optimized wastewater treatment plant method represents a ground-breaking shift in how we manage wastewater. By capitalizing on data-driven predictive modeling, automation, and optimization strategies, it introduces a comprehensive framework designed to enhance the efficiency and sustainability of wastewater treatment operations. This methodology encompasses various essential phases, including data gathering and training, the integration of innovative computational models such as Chimp-based GoogLeNet (CbG), data processing, and performance prediction, all while fine-tuning operational parameters. The designed model is a hybrid of the Chimp optimization algorithm and GoogLeNet. The GoogLeNet is a type of deep convolutional architecture, and the Chimp optimization is one of the bio-inspired optimization models based on chimpanzee behavior. It optimizes the operational parameters, such as pH, dosage rate, effluent quality, and energy consumption, of the wastewater treatment plant, by fixing the optimal settings in the GoogLeNet. The designed model includes the process such as pre-processing and feature analysis for the effective prediction of the operation parameters and its optimization. Notably, this innovative approach provides several key advantages, including cost reduction in operations, improved environmental outcomes, and more effective resource management. Through continuous adaptation and refinement, this methodology not only optimizes wastewater treatment plant performance but also effectively tackles evolving environmental challenges while conserving resources. It represents a significant step forward in the quest for efficient and sustainable wastewater treatment practices. The RMSE, MAE, MAPE, and R2 scores for the suggested technique are 1.103, 0.233, 0.012, and 0.002. Also, the model has shown that power usage decreased to about 1.4%, while greenhouse gas emissions have significantly decreased to 0.12% than the existing techniques.

Multi-scale image recognition strategy based on convolutional neural network

The accurate recognition and interpretation of multi-scale visual information is a critical focus within contemporary computer vision research. To this end, this study explores and innovatively constructs a multi-scale image recognition strategy based on a Convolutional Neural Network (CNN) with a multi-level and multi-resolution perception domain. This strategy is embedded with an advanced multi-level convolutional operation mechanism, which enables the model to intelligently explore and learn the multi-scale feature representation space of images from tiny texture to grand structure, from shallow simple features to deep semantic abstraction. The core technology path of this paper is to design a deep separable convolutional architecture and combine pyramid pool technology to form a unique network module. This modular design not only ensures the computational efficiency of the model but also improves the ability of extracting and integrating multi-scale image features. Following intensive experimentation on an array of extensively recognized and substantial image datasets, the multi-scale image recognition approach introduced in our study has demonstrated marked enhancements in both recognition capability and stability, manifesting clear superiority compared to conventional, single-scale image recognition methodologies. This research not only enriches the theoretical framework of image recognition, but also provides a new and efficient solution for dealing with complex multi-scale image recognition challenges in practical applications, and further promotes the development of image understanding and recognition technology.

Handwritten Vedic Sanskrit Text Recognition Using Deep Learning and Convolutional Neural Networks

Recognizing Vedic Sanskrit text is essential for accessing classical Indo-Aryan language, predominantly utilized in the Vedas. Currently, there is limited awareness about the Vedas, making this field a highly demanding and challenging area in pattern recognition. To accelerate progress in optical character recognition (OCR), deep learning methods are indispensable. This article presents a novel approach to Vedic Sanskrit text recognition, incorporating deep convolutional architectures with their respective interpretations. We introduce three modified 4-fold CNN architectures and the AlexNet model. Our system comprises a handwritten dataset containing 140 distinct Vedic Sanskrit words, with approximately 500 images per word, totaling around 70,000 images. The dataset is partitioned for training and testing in an 80:20 ratio. Training is conducted using 20% of the samples, and the resulting model is applied to the deep convolutional network with varied sets of neurons in their hidden layers. Our proposed method demonstrates robust support for accurate Vedic Sanskrit word classification. The recognition rate achieved in our research is 97.42%, with an average recognition time of 0.3640 milliseconds, surpassing existing CNN-based approaches.

Feature Extraction in Time-Lapse Seismic Using Deep Learning for Data Assimilation

Summary Assimilation of time-lapse (4D) seismic data with ensemble-based methods is challenging because of the massive number of data points. This situation requires excessive computational time and memory usage during the model updating step. We addressed this problem using a deep convolutional autoencoder to extract the relevant features of 4D images and generate a reduced representation of the data. The architecture of the autoencoder is based on the VGG-19 network, a deep convolutional architecture with 19 layers well-known for its effectiveness in image classification and object recognition. Some advantages of VGG-19 are the possibility of using some pretrained convolutional layers to create a feature extractor and taking advantage of the transfer learning technique to address other related problem domains. Using a pretrained model bypasses the need for large training data sets and avoids the high computational demand to train a deep network. For further improvements in the reconstruction of the seismic images, we apply a fine-tuning of the weights of the latent convolutional layer. We propose to use a fully convolutional architecture, which allows the application of distance-based localization during data assimilation with the ensemble smoother with multiple data assimilation (ES-MDA). The performance of the proposed method is investigated in a synthetic benchmark problem with realistic settings. We evaluate the methodology with three variants of the autoencoder, each one with a different level of data reduction. The experiments indicate that it is possible to use latent representations with major data reductions without impairing the quality of the data assimilation. Additionally, we compare central processing unit (CPU) and graphics processing unit (GPU) implementations of the ES-MDA update step and show in another synthetic problem that the reduction in the number of data points obtained with the application of the deep autoencoder may provide a substantial improvement in the overall computation cost of the data assimilation for large reservoir models.

Deep Recurrent Graph Convolutional Architecture for Sensor Fault Detection, Isolation, and Accommodation in Digital Twins

Deep Recurrent Graph Convolutional Architecture for Sensor Fault Detection, Isolation, and Accommodation in Digital Twins

Deep convolutional architectures for extrapolative forecasts in time-dependent flow problems

Physical systems whose dynamics are governed by partial differential equations (PDEs) find numerous applications in science and engineering. The process of obtaining the solution from such PDEs may be computationally expensive for large-scale and parameterized problems. In this work, deep learning techniques developed especially for time-series forecasts, such as LSTM and TCN, or for spatial-feature extraction such as CNN, are employed to model the system dynamics for advection-dominated problems. This paper proposes a Convolutional Autoencoder(CAE) model for compression and a CNN future-step predictor for forecasting. These models take as input a sequence of high-fidelity vector solutions for consecutive time steps obtained from the PDEs and forecast the solutions for the subsequent time steps using auto-regression; thereby reducing the computation time and power needed to obtain such high-fidelity solutions. Non-intrusive reduced-order modeling techniques such as deep auto-encoder networks are utilized to compress the high-fidelity snapshots before feeding them as input to the forecasting models in order to reduce the complexity and the required computations in the online and offline stages. The models are tested on numerical benchmarks (1D Burgers’ equation and Stoker’s dam-break problem) to assess the long-term prediction accuracy, even outside the training domain (i.e. extrapolation). The most accurate model is then used to model a hypothetical dam break in a river with complex 2D bathymetry. The proposed CNN future-step predictor revealed much more accurate forecasting than LSTM and TCN in the considered spatiotemporal problems.

Initial Study of Batik Generation using Variational Autoencoder

One of the most promising architectures for generative models is the variational autoencoder (VAE). To reconstruct Batik patterns for this work, we used a deep convolutional VAE architecture. Reconstruction outcomes from various batik motifs are mapped and contrasted using some criteria. As another crucial component of the convolutional network, batch normalization's impact on the model's performance was also examined. The dataset is used to study some learned latent space features. Through these findings, we laid the framework for next research on Batik generation utilizing VAE.

Fully automated 3D body composition analysis and its association with overall survival in head and neck squamous cell carcinoma patients.

We developed a method for a fully automated deep-learning segmentation of tissues to investigate if 3D body composition measurements are significant for survival of Head and Neck Squamous Cell Carcinoma (HNSCC) patients. 3D segmentation of tissues including spine, spine muscles, abdominal muscles, subcutaneous adipose tissue (SAT), visceral adipose tissue (VAT), and internal organs within volumetric region limited by L1 and L5 levels was accomplished using deep convolutional segmentation architecture - U-net implemented in a nnUnet framework. It was trained on separate dataset of 560 single-channel CT slices and used for 3D segmentation of pre-radiotherapy (Pre-RT) and post-radiotherapy (Post-RT) whole body PET/CT or abdominal CT scans of 215 HNSCC patients. Percentages of tissues were used for overall survival analysis using Cox proportional hazard (PH) model. Our deep learning model successfully segmented all mentioned tissues with Dice's coefficient exceeding 0.95. The 3D measurements including difference between Pre-RT and post-RT abdomen and spine muscles percentage, difference between Pre-RT and post-RT VAT percentage and sum of Pre-RT abdomen and spine muscles percentage together with BMI and Cancer Site were selected and significant at the level of 5% for the overall survival. Aside from Cancer Site, the lowest hazard ratio (HR) value (HR, 0.7527; 95% CI, 0.6487-0.8735; p = 0.000183) was observed for the difference between Pre-RT and post-RT abdomen and spine muscles percentage. Fully automated 3D quantitative measurements of body composition are significant for overall survival in Head and Neck Squamous Cell Carcinoma patients.

Utilizing deep convolutional neural architecture with attention mechanism for objective diagnosis of schizophrenia using wearable IoMT devices

Utilizing deep convolutional neural architecture with attention mechanism for objective diagnosis of schizophrenia using wearable IoMT devices

An Efficient and Effective Framework for Intestinal Parasite Egg Detection Using YOLOv5.

Intestinal parasitic infections pose a grave threat to human health, particularly in tropical and subtropical regions. The traditional manual microscopy system of intestinal parasite detection remains the gold standard procedure for diagnosing parasite cysts or eggs. This approach is costly, time-consuming (30 min per sample), highly tedious, and requires a specialist. However, computer vision, based on deep learning, has made great strides in recent years. Despite the significant advances in deep convolutional neural network-based architectures, little research has been conducted to explore these techniques' potential in parasitology, specifically for intestinal parasites. This research presents a novel proposal for state-of-the-art transfer learning architecture for the detection and classification of intestinal parasite eggs from images. The ultimate goal is to ensure prompt treatment for patients while also alleviating the burden on experts. Our approach comprised two main stages: image pre-processing and augmentation in the first stage, and YOLOv5 algorithms for detection and classification in the second stage, followed by performance comparison based on different parameters. Remarkably, our algorithms achieved a mean average precision of approximately 97% and a detection time of only 8.5 ms per sample for a dataset of 5393 intestinal parasite images. This innovative approach holds tremendous potential to form a solid theoretical basis for real-time detection and classification in routine clinical examinations, addressing the increasing demand and accelerating the diagnostic process. Our research contributes to the development of cutting-edge technologies for the efficient and accurate detection of intestinal parasite eggs, advancing the field of medical imaging and diagnosis.

Data-driven modeling of noise time series with convolutional generative adversarial networks ∗ ∗U.S. government work not protected by U.S. copyright.

Random noise arising from physical processes is an inherent characteristic of measurements and a limiting factor for most signal processing and data analysis tasks. Given the recent interest in generative adversarial networks (GANs) for data-driven modeling, it is important to determine to what extent GANs can faithfully reproduce noise in target data sets. In this paper, we present an empirical investigation that aims to shed light on this issue for time series. Namely, we assess two general-purpose GANs for time series that are based on the popular deep convolutional GAN architecture, a direct time-series model and an image-based model that uses a short-time Fourier transform data representation. The GAN models are trained and quantitatively evaluated using distributions of simulated noise time series with known ground-truth parameters. Target time series distributions include a broad range of noise types commonly encountered in physical measurements, electronics, and communication systems: band-limited thermal noise, power law noise, shot noise, and impulsive noise. We find that GANs are capable of learning many noise types, although they predictably struggle when the GAN architecture is not well suited to some aspects of the noise, e.g. impulsive time-series with extreme outliers. Our findings provide insights into the capabilities and potential limitations of current approaches to time-series GANs and highlight areas for further research. In addition, our battery of tests provides a useful benchmark to aid the development of deep generative models for time series.

A new real-time image semantic segmentation framework based on a lightweight deep convolutional encoder-decoder architecture for robotic environment sensing

Background: Image semantic segmentation can be understood as the allocation of a predefined category label to each pixel in the image to achieve the region segmentation of the image. Different categories in the image are identified with different colors. While achieving pixel classification, the position information of pixel points of different categories in the image is retained. Purpose: Due to the influence of background and complex environment, the traditional semantic segmentation methods have low accuracy. To alleviate the above problems, this paper proposes a new real-time image semantic segmentation framework based on a lightweight deep convolutional encoder-decoder architecture for robotic environment sensing. Methodology: This new framework is divided into three stages: encoding stage, decoding stage and dimension reduction stage. In the coding stage, a cross-layer feature map fusion (CLFMF) method is proposed to improve the effect of feature extraction. In the decoding stage, a new lightweight decoder (LD) structure is designed to reduce the number of convolutional layers to speed up model training and prediction. In the dimension reduction stage, the convolution dimension reduction method (CDR) is presented to connect the encoder and decoder layer by layer to enhance the decoder effect. Results: Compared with other state-of-the-art image semantic segmentation methods, we conduct comparison experiments on datasets Cityscapes, SUN RGB-D, CamVid, KITTI. The Category iIoU combined with the proposed method is more than 70%, and the Category IoU is as high as 89.7%. Conclusion: The results reflect that the new method can achieve the better semantic segmentation effect.

A surrogate modeling method for distributed land surface hydrological models based on deep learning

Surrogate modelling methods have been used to estimate spatially distributed parameters of land surface hydrological models due to their computational efficiency. However, traditional surrogate techniques have trouble in modelling high-dimensional input–output maps. In addition, their applications to a distributed model over large spatial and temporal domains are very unlikely to produce consistent spatial pattern and dynamic relationship of model input and output, leading to unbalanced model performance across different spatiotemporal domains. To address these issues, we employ a deep autoregressive neural network to construct an accurate and reliable surrogate system of distributed dynamic model outputs. The network uses a deep convolutional encoder-decoder architecture to take full advantage of the deep convolutional layers in high-dimensional image processing and spatial feature extraction. Moreover, the autoregressive strategy makes the network good at handling dynamic relationship between the time-varying model input and output. We apply this deep learning (DL) network to building a surrogate model of the Variable Infiltration Capacity (VIC) model simulated soil moisture over the Huaihe river basin of China. The results show that the adopted deep autoregressive neural network can provide an accurate surrogate model of the high-dimensional dynamic relationship between 15 input fields and 1 output field, each covering 408 grids, over multiple years. This surrogate model significantly outperforms the popular long short-term memory (LSTM) model, achieving an average mean squared error (MSE) of 0.26 and an average R2 of 0.76 using the test datasets. The surrogate's MSE and R2 performance represents an average improvement of 43% and 33%, respectively, compared to the LSTM model. In addition, it shows better spatial performance than the LSTM model, demonstrating its superior performance in approximating spatial patterns of VIC simulated soil moisture. The accuracy of this surrogate method and its ability to handle high-dimensional data is promising for advancing parameter uncertainty quantification of distributed land surface hydrological models, which usually have high dimensionality due to the large number of variables involved, including distributed parameters, input forcing variables, and output variables.

Towards Safe Visual Navigation of a Wheelchair Using Landmark Detection

This article presents a method for extracting high-level semantic information through successful landmark detection using 2D RGB images. In particular, the focus is placed on the presence of particular labels (open path, humans, staircase, doorways, obstacles) in the encountered scene, which can be a fundamental source of information enhancing scene understanding and paving the path towards the safe navigation of the mobile unit. Experiments are conducted using a manual wheelchair to gather image instances from four indoor academic environments consisting of multiple labels. Afterwards, the fine-tuning of a pretrained vision transformer (ViT) is conducted, and the performance is evaluated through an ablation study versus well-established state-of-the-art deep architectures for image classification such as ResNet. Results show that the fine-tuned ViT outperforms all other deep convolutional architectures while achieving satisfactory levels of generalization.

A Deep Convolutional Neural Network Framework for Enhancing Brain Tumor Diagnosis on MRI Scans

Brain tumors are a critical public health concern, often resulting in limited life expectancy for patients. Accurate diagnosis of brain tumors is crucial to develop effective treatment strategies and improve patients' quality of life. Computer-aided diagnosis (CAD) systems that accurately classify tumor images have been challenging to develop. Deep convolutional neural network (DCNN) models have shown significant potential for tumor detection, and outperform traditional deep neural network models. In this study, a novel framework based on two pre-trained deep convolutional architectures (VGG16 and EfficientNetB0) is proposed for classifying different types of brain tumors, including meningioma, glioma, and pituitary tumors. Features are extracted from MR images using each architecture and merged before feeding them into machine learning algorithms for tumor classification. The proposed approach achieves a training accuracy of 98% and a test accuracy of 99% on the brain-tumor-classification-mri dataset available on Kaggle and btc_navoneel. The model shows promise to improve the accuracy and generalizability of medical image classification for better clinical decision support, ultimately leading to improved patient outcomes.

Innovative Out-of-Stock Prediction System Based on Data History Knowledge Deep Learning Processing

Research and development efforts in the field of commercial applications have invested strategic interest in the design of intelligent systems that correctly handle out-of-stock events. An out-of-stock event refers to a scenario in which such customers do not have the availability of the products they want to buy. This scenario generates important economic damage to the producer and to the commercial store. Addressing the out-of-stock problem is currently of great interest in the commercial field as it would allow limiting the economic damages deriving from these events. Furthermore, in the era of online commerce (e-commerce), it would significantly limit out-of-stock events which show a considerable economic impact in the field. For these reasons, the authors proposed a solution based on deep learning for predicting the residual stock amount of a commercial product based on the intelligent analysis of specific visual–commercial data as well as seasonality. By means of a combined deep pipeline embedding convolutional architecture boosted with a self-attention mechanism and a downstream temporal convolutional network, the authors will be able to predict the remaining stock of a particular commodity. By integrating and interpreting climate/seasonal information, customers’ behavior data, and full history data on the dynamics of commercial sales, it will be possible to estimate the residual stock of a certain product and, therefore, define purchase orders efficiently. An accurate prediction of remaining stocks allows an efficient trade order policy which results in a significant reduction in out-of-stock events. The experimental results confirmed the effectiveness of the proposed approach with an accuracy (in the prediction of the remaining stock of such products) greater than 90%.

Performance Analysis of Deep Convolutional Network Architectures for Classification of Over-Volume Vehicles

The number of vehicle accidents has increased in recent years due to overloaded goods carriers. Off-road driving, mountain roads, and sharp edges on a road are the main causes of an imbalance in overloaded trucks. In rural areas, where smaller roads cannot accommodate high volume vehicles, such vehicles cause many problems for cars, bikes, bicycles, and other small vehicles, as well as an increase in traffic congestion in those areas. This has become a major problem in the daily lives of drivers in rural areas as well as major urban areas. Solutions are needed to detect over-volume goods carriers and alert drivers to slow down or control the volume in their trucks. This work mainly focuses on a solution that uses deep CNN models. In this work, different deep convolutional architectures are evaluated for their ability to classify goods based on their volume. The model implemented is based on a dataset-specific transfer learning process with CNN layers generated in ImageNet in which only dense layers are learned. The primary objective of this work is to identify a classification method that exhibits proven results with respect to the accuracy parameters. In this work, different deep architectures were tested, and among the efficient networks, Net-B3 was found to perform with 95% accuracy on average. The different architectures were evaluated based on their accuracy, confusion matrix, ROC curve, and AUC score with a real-time dataset.

An Unsupervised Writer Identification Based on Generating Clusterable燛mbeddings

The writer identification system identifies individuals based on their handwriting is a frequent topic in biometric authentication and verification systems. Due to its importance, numerous studies have been conducted in various languages. Researchers have established several learning methods for writer identification including supervised and unsupervised learning. However, supervised methods require a large amount of annotation data, which is impossible in most scenarios. On the other hand, unsupervised writer identification methods may be limited and dependent on feature extraction that cannot provide the proper objectives to the architecture and be misinterpreted. This paper introduces an unsupervised writer identification system that analyzes the data and recognizes the writer based on the inter-feature relations of the data to resolve the uncertainty of the features. A pairwise architecture-based Autoembedder was applied to generate clusterable embeddings for handwritten text images. Furthermore, the trained baseline architecture generates the embedding of the data image, and the K-means algorithm is used to distinguish the embedding of individual writers. The proposed model utilized the IAM dataset for the experiment as it is inconsistent with contributions from the authors but is easily accessible for writer identification tasks. In addition, traditional evaluation metrics are used in the proposed model. Finally, the proposed model is compared with a few unsupervised models, and it outperformed the state-of-the-art deep convolutional architectures in recognizing writers based on unlabeled data.

Trustworthiness of Laser-Induced Breakdown Spectroscopy Predictions via Simulation-based Synthetic Data Augmentation and Multitask Learning

Laser-induced breakdown spectroscopy is a versatile technique that can be used to quickly measure the concentration of elements in ambient air. We tackle the issues of performance and trustworthiness of the statistical model used for predictions. We propose a method for improving the performance and trustworthiness of statistical models for LIBS. Our method uses deep convolutional multitask learning architectures to predict the concentration of the analyte and additional information as auxiliary outputs. We also introduce a simulation-based data augmentation process to synthesize more training samples. The secondary predictions from the model are used to characterize, quantify and validate its trustworthiness, taking advantage of the mutual dependencies of the weights of the neural networks. As a consequence, these output can be used to successfully detect anomalies, such as changes in the experimental conditions, and out-of-distribution samples. Results on different types of materials show that the proposed method improves the robustness and trueness of the predictions.

Performance Analysis of Machine Learning and Deep Learning Models for Classification of Alzheimer’s Disease from Brain MRI

Alzheimer's disease (AD) is an irreversible and degenerative brain condition that gradually damages memory and thinking abilities. Despite being incurable, AD causes significant pain and financial hardship to patients and their families. However, medications are most effective when administered early in the course of the disease and early diagnosis is crucial in the treatment of AD to restrict its progression. There are several approaches proposed for computer-assisted AD diagnosis that involve structural and functional imaging modalities, such as sMRI, fMRI, DTI, and PET. Machine learning and deep learning techniques have facilitated the development of novel models for diagnostic accuracy in AD. This research compares the performance of several machine learning and deep convolutional architectures to detect AD from MCI. It is essential to find the effective baseline model for classifying AD, hence all the pre-trained models are evaluated with benchmark dataset. Experimental observations indicate that the DenseNet-169 performed best out of different state-of-the-art architectures, with an average accuracy of 82.2%.