Encoder-decoder Neural Network Research Articles

A Deep Neural Network for Road Extraction with the Capability to Remove Foreign Objects with Similar Spectra

Abstract. Existing road extraction methods based on deep learning often struggle with distinguishing ground objects that share similar spectral information, such as roads and buildings. Consequently, this study proposes a dual encoder-decoder deep neural network to address road extraction in complex backgrounds. In the feature extraction stage, the first encoder-decoder designed for extracting road features. The second encoder-decoder utilized for extracting building features. During the feature fusion stage, road features and building features are integrated using a subtraction method. The resultant road features, constrained by building features, enhance the preservation of accurate road feature information. Within the feature fusion stage, road feature maps and building feature maps designated for fusion are input into the convolutional block attention module. This step aims to amplify the features of different channels and extract key information from diverse spatial positions. Subsequently, feature fusion is executed using the element-by-element subtraction method. The outcome is road features constrained by building features, thus preserving more precise road feature information. Experimental results demonstrate that the model successfully learns both road and building features concurrently. It effectively distinguishes between easily confused roads and buildings with similar spectral information, ultimately enhancing the accuracy of road extraction.

TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection

Abstract Robust road surface estimation is required for autonomous ground vehicles to navigate safely. Despite it becoming one of the main targets for autonomous mobility researchers in recent years, it is still an open problem in which cameras and LiDAR sensors have demonstrated to be adequate to predict the position, size and shape of the road a vehicle is driving on in different environments. In this work, a novel Convolutional Neural Network model is proposed for the accurate estimation of the roadway surface. Furthermore, an ablation study has been conducted to investigate how different encoding strategies affect model performance, testing 6 slightly different neural network architectures. Our model is based on the use of a Twin Encoder–Decoder Neural Network (TEDNet) for independent camera and LiDAR feature extraction and has been trained and evaluated on the Kitti–Road dataset. Bird’s Eye View projections of the camera and LiDAR data are used in this model to perform semantic segmentation on whether each pixel belongs to the road surface. The proposed method performs among other state-of-the-art methods and operates at the same frame rate as the LiDAR and cameras, so it is adequate for its use in real-time applications.

Lunar ground segmentation using a modified U-net neural network

Semantic segmentation plays a significant role in unstructured and planetary scene understanding, offering to a robotic system or a planetary rover valuable knowledge about its surroundings. Several studies investigate rover-based scene recognition planetary-like environments but there is a lack of a semantic segmentation architecture, focused on computing systems with low resources and tested on the lunar surface. In this study, a lightweight encoder-decoder neural network (NN) architecture is proposed for rover-based ground segmentation on the lunar surface. The proposed architecture is composed by a modified MobilenetV2 as encoder and a lightweight U-net decoder while the training and evaluation process were conducted using a publicly available synthetic dataset with lunar landscape images. The proposed model provides robust segmentation results, allowing the lunar scene understanding focused on rocks and boulders. It achieves similar accuracy, compared with original U-net and U-net-based architectures which are 110–140 times larger than the proposed architecture. This study, aims to contribute in lunar landscape segmentation utilizing deep learning techniques, while it proves a great potential in autonomous lunar navigation ensuring a safer and smoother navigation on the moon. To the best of our knowledge, this is the first study which propose a lightweight semantic segmentation architecture for the lunar surface, aiming to reinforce the autonomous rover navigation.

Predicting Time-to-Healing from a Digital Wound Image: A Hybrid Neural Network and Decision Tree Approach Improves Performance

Despite the societal burden of chronic wounds and despite advances in image processing, automated image-based prediction of wound prognosis is not yet in routine clinical practice. While specific tissue types are known to be positive or negative prognostic indicators, image-based wound healing prediction systems that have been demonstrated to date do not (1) use information about the proportions of tissue types within the wound and (2) predict time-to-healing (most predict categorical clinical labels). In this work, we analyzed a unique dataset of time-series images of healing wounds from a controlled study in dogs, as well as human wound images that are annotated for the tissue type composition. In the context of a hybrid-learning approach (neural network segmentation and decision tree regression) for the image-based prediction of time-to-healing, we tested whether explicitly incorporating tissue type-derived features into the model would improve the accuracy for time-to-healing prediction versus not including such features. We tested four deep convolutional encoder–decoder neural network models for wound image segmentation and identified, in the context of both original wound images and an augmented wound image-set, that a SegNet-type network trained on an augmented image set has best segmentation performance. Furthermore, using three different regression algorithms, we evaluated models for predicting wound time-to-healing using features extracted from the four best-performing segmentation models. We found that XGBoost regression using features that are (i) extracted from a SegNet-type network and (ii) reduced using principal components analysis performed the best for time-to-healing prediction. We demonstrated that a neural network model can classify the regions of a wound image as one of four tissue types, and demonstrated that adding features derived from the superpixel classifier improves the performance for healing-time prediction.

Mild Traumatic Brain Injury andCareer Stage Associate with Visible Perivascular Spaces in Special Operations Forces Soldiers.

Mild traumatic brain injury (mTBI) and occupational blast exposure in military Service Members may lead to impaired brain waste clearance which increases neurological disease risk. Perivascular spaces (PVS) are a key part of the glymphatic system which supports brain waste clearance, preferentially during sleep. Visible PVS on clinical magnetic resonance imaging have been previously observed in patients with neurodegenerative diseases and animal neurotrauma models. The purpose of this study was to determine associations between PVS morphological characteristics, military career stage, and mTBI history in Special Operations Forces (SOF)Soldiers. Participants underwent T2-weighed neuroimaging to capture three-dimensional whole brain volumes. Segmentation was performed using a previously validated, multi-scale deep convolutional encoder-decoder neural network. Only PVS clusters within the white matter mask were quantified for analyses. Due to non-normal PVS metric distribution, non-parametric Mann-Whitney U tests were used to determine group differences in PVS outcomes. In total, 223 healthy SOF combat Soldiers (age = 33.1 ± 4.3yrs) were included, 217 reported career stage. Soldiers with mTBI history had greater PVS number (z = 2.51, P = 0.013) and PVS volume (z = 2.42, P = 0.016). In-career SOF combat Soldiers had greater PVS number (z = 2.56, P = 0.01) and PVS volume (z = 2.28, P = 0.02) compared to a baseline cohort. Mild TBI history is associated with increased PVS burden in SOF combat Soldiers that are clinically recovered from mTBI. This may indicate ongoing physiological changes that could lead to impaired waste clearance via the glymphatic system. Future studies should determine if PVS number and volume are meaningful neurobiological outcomes for neurodegenerative disease risk and if clinical interventions such as improving sleep can reduce PVS burden.

Deep Learning Approach to Within-Bank Fault Detection and Diagnostics of Fine Motion Control Rod Drives

Control rod motion is one of the primary means of regulating the rate of fission in a nuclear reactor core to ensure safe and stable operation. Reactor power distribution and thermal power output can be fine-tuned by adjusting the control rod position. For high-precision control of rod movements, Fine Motion Control Rod Drives (FMCRDs) are often used. The operation of FMCRDs provides a unique opportunity to implement condition monitoring related to the intermittency of motion and the use of control rod banks. This research sets out to detect three types of faults in an electrically driven FMCRD. In addition to detecting faults, this work will attempt to determine both the type of fault and the source of each fault, completing the fault detection and diagnostics (FDD) pipeline on a scarcely researched system. The three types of faults to be investigated are short-circuit faults, ball screw wear faults, and ball screw jam faults. This is a potential advancement to the within-bank FDD of this specific drive system intended for deployment in an advanced nuclear reactor plant. Using encoder-decoder structured convolutional neural networks and autoencoders, the three tested faults were confidently detected and isolated as well as reasonably diagnosed by monitoring the FMCRD servomotor torque.

Predicting wind farm wake losses with deep convolutional hierarchical encoder–decoder neural networks

Wind turbine wakes are the most significant factor affecting wind farm performance, decreasing energy production and increasing fatigue loads in downstream turbines. Wind farm turbine layouts are designed to minimize wake interactions using a suite of predictive models, including analytical wake models and computational fluid dynamics simulations (CFD). CFD simulations of wind farms are time-consuming and computationally expensive, which hinder their use in optimization studies that require hundreds of simulations to converge to an optimal turbine layout. In this work, we propose DeepWFLO, a deep convolutional hierarchical encoder–decoder neural network architecture, as an image-to-image surrogate model for predicting the wind velocity field for Wind Farm Layout Optimization (WFLO). We generate a dataset composed of image representations of the turbine layout and undisturbed flow field in the wind farm, as well as images of the corresponding wind velocity field, including wake effects generated with both analytical models and CFD simulations. The proposed DeepWFLO architecture is then trained and optimized through supervised learning with an application-tailored loss function that considers prediction errors in both wind velocity and energy production. Results on a commonly used test case show median velocity errors of 1.0%–8.0% for DeepWFLO networks trained with analytical and CFD data, respectively. We also propose a model-fusion strategy that uses analytical wake models to generate an additional input channel for the network, resulting in median velocity errors below 1.8%. Spearman rank correlations between predictions and data, which evidence the suitability of DeepWFLO for optimization purposes, range between 92.3% and 99.9%.

Motion-Based Wave Inference With Neural Networks: Transfer Learning From Numerical Simulation to Experimental Data

Abstract The directional wave spectrum, which describes the distribution of wave energy along frequencies and directions, can be estimated from the measured motions of a vessel subjected to a particular sea condition by resorting to the wave-buoy analogy. Several methods have been proposed to address the inverse estimation problem; recently, machine learning techniques have been assessed as further alternatives. However, it may be difficult to gather large datasets of in-service motion responses and the associated sea states to train effective data-driven models. In this work, an encoder–decoder neural network is trained with the synthetic responses of a station-keeping platform supply vessel (PSV) to estimate the directional wave spectrum. This estimation model is directly applied to perform wave inference from motion data of wave basin tests with a small-scale model of the same vessel. Furthermore, fine-tuning is also used to incorporate experimental data into the neural network model. Results show a satisfactory match between estimated and measured values, both with respect to the energy distribution and the integral spectrum parameters, indicating that the proposed approach can be employed to obtain data-driven wave inference models when there is little or no availability of measured motion records and the corresponding sea conditions.

Regional ship collision risk prediction: An approach based on encoder-decoder LSTM neural network model

Ship collision risk prediction is vital for maritime traffic surveillance, which determines if there is a sustainable and efficient development of the shipping industry. In order to solve the problem of ship collision risk prediction, this study proposes a regional ship collision risk prediction model based on the Encoder-decoder LSTM neural network. Firstly, the regional ship collision risk is quantified by aggregation density-based method, and the spatial-temporal collision risk matrix is synthesized by grid processing method. Then, a key aspect of the model involves employing the CNN encoder to capture spatial risk features and compress the collision risk structure scale. The LSTM neural network as the middle layer of the model is adopted to predict the risk with spatial-temporal characteristics. Finally, the CNN decoder processes the prediction results by deconvolution, which restore the original dimension of data. In our study, we conducted numerous experiments using AIS data from Zhoushan Port of Ningbo. The results show that the proposed model has high accuracy in predicting the spatial-temporal collision risk, which the accuracy rate reached 97.9%. The proposed approach of regional ship collision prediction is efficient and accurate, which can provide decision support for intelligent maritime traffic surveillance.

Seismic vulnerability signal analysis of low tower cable-stayed bridges method based on convolutional attention network

Abstract Due to the particularity and complexity of sedimentary environments, the wave impedance differences between different reflection interfaces in underground media may vary greatly. Therefore, an encoder–decoder neural network is proposed to enhance erroneous seismic weak reflection signals. The convolutional neural network (CNN) has the problem of difficulty in parallel computing, resulting in slow network training and computational efficiency. Considering that attention has an innate global self-attention mechanism, can compensate for long-term dependency deficiencies, and has the ability to perform parallel computing, which greatly compensates for the shortcomings of CNNs and recurrent neural networks, a seismic impedance inversion method based on convolutional attention networks is proposed. To improve the ability to extract noise, residual structure and convolutional attention module (CBAM) were introduced. The residual structure utilizes residual jump to weaken network degradation and reduce the difficulty of feature mapping. The CBAM uses a mixed attention weight of channel and space, which can enhance features with high correlation and suppress features with low correlation. In the decoder, in order to improve the dimension recovery ability of feature fusion, bilinear interpolation is selected for upsampling. The application results of the model and actual data indicate that this method can effectively enhance the weak reflection signals caused by the formation itself and improve the reservoir identification ability of seismic data.

A neural network encoder-decoder for time series prediction: Application on 137Cs particulate concentrations in nuclearized rivers

Monitoring the impact of human activities on the environment is a major challenge as many pollutants can be found in the different ecosystems. This is the case of the caesium-137 that has been present in the environment for many decades as a result of atmospheric tests, accidents such as Chernobyl and release from nuclear industries. With the recent advance in data-driven models, this study evaluate the relevance of a deep learning tool for reconstructing caesium-137 chronics particulate concentration in rivers. An encoder-decoder neural network, “Hierarchical Attention-Based Recurrent Highway Networks”(HRHN), is proposed notably for its ability to extract the most relevant temporal and spatial information from the databases. Three monitoring stations were studied, one on the Rhône River and two on the Loire River, all of them downstream nuclear industries in these catchments affected by the global fallout and the accident of Chernobyl. The objective is to predict the future concentration from a set of variables providing past information on water discharge, washout flux and industrial radioactive releases. Once optimised, the model generates first results in agreement with the real concentration curves by correctly following the main trends, with a NSE of 0.89, 0.53 and 0.35 respectively for the Rhone station and the two stations on the Loire. The main reason of inaccuracies is due to the quantity of data available. The originality of this model is its capacity to make predictions on different catchment areas. In fact the training was conducted on the Rhône station as the range of the concentration was higher (from 265.4 to 2700.0 Bq/kg) and the testing on the two Loire station. Another encoder-decoder model DA-RNN (Dual-Stage Attention-Based Recurrent Neural Network) was also evaluate in order to compare the performance of an alternative architecture, without convolution layer. The conclusion is that HRHN remains more powerful in the predictions on the 3 systems. With these first interesting results for HRHN, further investigations should be taken into account for other pollutants than caesium-137 to better understand the robustness of the model.

An indeterminacy fusion of encoder-decoder network based on neutrosophic set for white blood cells segmentation

Medical image analysis, particularly the segmentation of White Blood Cells (WBCs), holds critical importance in scrutinizing the quantity and morphology of these cells in smear images, a pivotal step in disease detection. Existing methodologies often fall short in differentiating between healthy and diseased WBCs, emphasizing color components over internal organizational nuances and external morphologies. This paper presents a pioneering contribution through the introduction of an encoder-decoder deep neural network with an indeterminacy fusion-based model explicitly designed for WBC extraction from blood images. In contrast to prior studies, our approach addresses the inadequacies by delving into the intricacies of WBC structure, characterizing object indeterminacy within the neutrosophic set domain. The encoder-decoder network is fortified by integrating WBC indeterminacy as a fusion component, enabling the segmentation of WBCs into distinct nucleus and cytoplasm regions. The efficacy of our methodology is evaluated across three datasets with varying resolutions. Quantitative metrics such as segmentation accuracy and intersection over union (IoU) are employed to assess the proposed network's performance, which consistently outperforms three original encoder-decoder networks. Notably, our model achieves commendable precision rates, with the highest mean segmentation accuracy reaching 0.95301 across the three datasets. These results underscore the enhanced performance of our segmentation network, attributing its success to the influential role of indeterminacy fusion in augmenting conventional segmentation methods. In summary, this work not only introduces a novel solution for WBC segmentation but establishes its superiority over existing methods. The interpretive insights gained from the results highlight the potential of indeterminacy fusion to drive advancements in the realm of medical image segmentation, emphasizing originality, significance, and superior performance metrics.

A computational intelligence approach for solar photovoltaic power generation forecasting

This article describes an approach applying computational intelligence methods for the problem of forecasting solar photovoltaic power generation at country level. Precise forecast of power generation plays a vital role in designing a dependable photovoltaic power generation system. The computed predictions enable the implementation of efficient planning, management, and distribution strategies for the generated power, ultimately enhancing the performance and efficiency of the system. The study analyzes and compares artificial neural network approaches for a specific case study using real solar photovoltaic power generation data from Uruguay in the period 2018 to 2022. Several artificial neural network architectures are evaluated for forecasting. The main results indicate that the approach applying a combination of Encoder-Decoder and Long Short Term Memory artificial neural networks is the most effective method for the addressed forecasting problem. The approach yielded promising results, with an average mean error value of 0.09, improving over the other artificial neural network architectures. Even better results were obtained for sunny days. The generated forecasts hold significant value for its application in planning and scheduling processes, aiming to enhance the overall quality of service of the electricity grid.

A Stochastic Recurrent Encoder Decoder Network for Multistep Probabilistic Wind Power Predictions.

In this article, a stochastic recurrent encoder decoder neural network (SREDNN), which considers latent random variables in its recurrent structures, is developed for the first time for the generative multistep probabilistic wind power predictions (MPWPPs). The SREDNN enables the stochastic recurrent model under the encoder-decoder framework to engage exogenous covariates to produce better MPWPP. The SREDNN consists of five components, the prior network, the inference network, the generative network, the encoder recurrent network, and the decoder recurrent network. The SREDNN is equipped with two critical advantages compared with conventional RNN-based methods. First, the integration over the latent random variable builds an infinite Gaussian mixture model (IGMM) as the observation model, which drastically increases the expressiveness of the wind power distribution. Secondly, hidden states of the SREDNN are updated in a stochastic way, which builds an infinite mixture of the IGMM for describing the ultimate wind power distribution and enables the SREDNN to model complex patterns across wind speed and wind power sequences. Computational experiments are conducted on a dataset of a commercial wind farm having 25 wind turbines (WTs) and two publicly assessable WT datasets to verify the advantages and effectiveness of the SREDNN for MPWPP. Experimental results show that the SREDNN achieves a lower negative form of the continuously ranked probability score (CRPS ∗) as well as a superior sharpness and comparable reliability of prediction intervals by comparing against considered benchmarking models. Results also show the clear benefit gained from considering latent random variables in SREDNN.

GEEF: A neural network model for automatic essay feedback generation by integrating writing skills assessment

Assessing students' writing ability has always been an important component of research on automatic essay evaluation. The existing methods usually focus on outputting numerical scores, and there is little research on providing feedback on writing quality based on text generation techniques. We believe that the generation of quality feedback in essay writing is more valuable as a reference. In this study, we address the problem of essay feedback generation by proposing an encoder-decoder neural network model called GEEF (Generate Essay Feedback) and suppose that feedbacks are written based on the source essay text and the grading of important writing skills. Besides from the text of input essay, our model also takes in additional features including fluency, coherence, richness, and literary talent. We construct an essay feedback corpus along with human-tagged resources to facilitate the study. Experimental results demonstrate that the proposed approach achieves promising performance compared with other baseline methods on automatic and human evaluation metrics. The generated feedback sentences on different aspects of essay writing can help students understand their strengths and weaknesses in writing skills. In addition, the potential application of this study is that it can assist raters in writing more complex feedback on this basis, as it is often difficult to remember many commonly used feedback patterns when writing comments.

Convolutional LSTM-Attention Based Encoder–Decoder Neural Network for Prediction of Chaotic Vibrations of Multi-Dimensional Dynamic Systems

The present research proposes a convolutional long-short term memory (ConvLSTM) with an attention mechanism (AM) model, termed as ConvLSTM-AM, to conduct prediction of chaotic vibrations of multi-dimensional dynamic systems. The proposed data-driven model is based on an encoder–decoder architecture where the lengths of inputs and outputs are variable. Different from other conventional benchmarks which consider the temporal correlation solely to deal with the chaotic sequences, this research work takes the spatial information into account. In this sense, the ConvLSTM is adopted as an encoder to acquire useful chaotic spatiotemporal patterns and retain long-term successive dependencies. LSTM and AM in this research are taken as the main structures of the procedure in decoding, in which the LSTM is used as the further temporal processor and AM is stacked on the top to exploit more salient information of the historical data. Among that, a residual connection between the outputs of LSTM and the information of attention is considered in AM to prevent gradient vanishment. Two datasets of chaotic vibrations of multi-dimensional systems are employed to adequately illustrate the effectiveness and feasibility of the proposed model. Besides, five conventional benchmarks are built to demonstrate the advantages of the proposed model in terms of both training and generalization performance. As found in the research, the training time is reduced with lower testing loss in comparing with the other five counterparts, as the spatial information introduced expedites the training convergence. The present research provides a useful guidance for predicting and analysing chaotic vibrations of multi-dimensional dynamic systems.

Multi-agent deep reinforcement learning based real-time planning approach for responsive customized bus routes

Customized bus can meet many passengers’ personalized travel demand in a public transportation system by providing an innovative shared travel service. Customized bus offers multiple bus routes that jointly form a route network to serve its passengers. It must frequently adjust the station sequences of each bus route in response to trip cancellations and new trip bookings during its operation. Different from relevant research works in the literature, this paper proposes a multi-agent deep reinforcement learning based real-time planning approach for tackling the multiple customized bus routes planning problem. We model the problem as a multi-agent Markov decision process for the first time in literature where a separate agent is assigned to each route to plan its station sequence. We then develop a new multi-agent system. Each agent in the system is powered by an encoder–decoder neural network that consolidates the station sequence decision policy for each bus route. We employ a policy gradient-based reinforcement learning algorithm to train the network parameters of the multi-agent system so as to maximize the number of passengers served while ensuring the customized bus service quality and minimizing the operating cost of all customized bus routes. On three (six) problem instances in offline (online) scenarios, the trained multi-agent system can significantly outperform several existing algorithms in terms of the total cost, adaptiveness and computation time.

Underwater Acoustic Signal Noise Reduction Based on a Fully Convolutional Encoder-Decoder Neural Network

Underwater Acoustic Signal Noise Reduction Based on a Fully Convolutional Encoder-Decoder Neural Network

Metal Surface Defect Detection Based on a Transformer with Multi-Scale Mask Feature Fusion

In the production process of metal industrial products, the deficiencies and limitations of existing technologies and working conditions can have adverse effects on the quality of the final products, making surface defect detection particularly crucial. However, collecting a sufficient number of samples of defective products can be challenging. Therefore, treating surface defect detection as a semi-supervised problem is appropriate. In this paper, we propose a method based on a Transformer with pruned and merged multi-scale masked feature fusion. This method learns the semantic context from normal samples. We incorporate the Vision Transformer (ViT) into a generative adversarial network to jointly learn the generation in the high-dimensional image space and the inference in the latent space. We use an encoder–decoder neural network with long skip connections to capture information between shallow and deep layers. During training and testing, we design block masks of different scales to obtain rich semantic context information. Additionally, we introduce token merging (ToMe) into the ViT to improve the training speed of the model without affecting the training results. In this paper, we focus on the problems of rust, scratches, and other defects on the metal surface. We conduct various experiments on five metal industrial product datasets and the MVTec AD dataset to demonstrate the superiority of our method.

An expandable machine learning-optimization framework to sequential decision-making

We present an integrated prediction-optimization (PredOpt) framework to efficiently solve sequential decision-making problems by predicting the values of binary decision variables in an optimal solution. We address the key issues of sequential dependence, infeasibility, and generalization in machine learning (ML) to make predictions for optimal solutions to combinatorial problems. The sequential nature of the combinatorial optimization problems considered is captured with recurrent neural networks and a sliding-attention window. We integrate an attention-based encoder–decoder neural network architecture with an infeasibility-elimination and generalization framework to learn high-quality feasible solutions to time-dependent optimization problems. In this framework, the required level of predictions is optimized to eliminate the infeasibility of the ML predictions. These predictions are then fixed in mixed-integer programming (MIP) problems to solve them quickly with the aid of a commercial solver. We demonstrate our approach to tackling the two well-known dynamic NP-Hard optimization problems: multi-item capacitated lot-sizing (MCLSP) and multi-dimensional knapsack (MSMK). Our results show that models trained on shorter and smaller-dimensional instances can be successfully used to predict longer and larger-dimensional problems. The solution time can be reduced by three orders of magnitude with an average optimality gap below 0.1%. We compare PredOpt with various specially designed heuristics and show that our framework outperforms them. PredOpt can be advantageous for solving dynamic MIP problems that need to be solved instantly and repetitively.