Wide Receptive Fields Research Articles

Temporal convolutional network with soft threshold and contractile self-attention mechanism for remaining useful life prediction of rolling bearings

Abstract RUL prediction is an effective approach to prevent system failures and cut maintenance expenditures. Due to the wide receptive field and the avoidance of future information leakage, temporal convolutional neural network (TCN) is widely applied for RUL estimation of bearings. However, the predictive performance of TCN is limited by the loss of degradation features and the breakdown of continuity of timing information. To overcome the above defects, hybrid temporal convolutional neural network with soft threshold and contractile self-attention mechanism (HTCN-SC) is presented. Firstly, the adaptive threshold is determined by the contraction self-attention mechanism with higher interpretability, which captures the contribution of different features to the estimation of RUL. Then, the soft threshold is employed to activate the degraded features. On the one hand, the degeneracy features endowed by the dilated causal convolution with obvious negative values are fully preserved. On the other hand, the noise components that are given low weights are completely suppressed compared to the original TCN. Finally, parallel branch composed of one-dimensional convolutional networks are used to supplement the continuity of time series. Degradation signals from different working conditions and bearings are employed to verify the performance of the HTCN-SC. The results indicate that HTCN-SC with accurate RUL estimation and generalization ability is an effective tool for rolling bearing health monitoring.

An advanced hybrid deep learning model for accurate energy load prediction in smart building

In smart cities, sustainable development depends on energy load prediction since it directs utilities in effectively planning, distributing and generating energy. This work presents a novel hybrid deep learning model including components of the Improved-convolutional neural network (CNN), bidirectional long short-term memory (Bi-LSTM), Graph neural network (GNN), Transformer and Fusion Layer architectures for precise energy load forecasting. Better feature extraction results from the Improved-CNN's dilated convolution and residual block accommodation of wide receptive fields reduced the vanishing gradient problem. By capturing temporal links in both directions, Bi-LSTM networks help to better grasp complicated energy use patterns. Graph neural networks improve predictive capacities across linked systems by characterizing the spatial relationships between energy-consuming units in smart cities. Emphasizing critical trends to guarantee reliable forecasts, transformer models use attention methods to manage long-term dependencies in energy consumption data. Combining CNN, Bi-LSTM, Transformer and GNN component predictions in a Fusion Layer synthesizes numerous data representations to increase accuracy. With Root Mean Square Error of 5.7532 Wh, Mean Absolute Percentage Error of 3.5001%, Mean Absolute Error of 6.7532 Wh and R2 of 0.9701, the hybrid model fared better than other models on the ‘Electric Power Consumption’ Kaggle dataset. This work develops a realistic model that helps informed decision-making and enhances energy efficiency techniques, promoting energy load forecasting in smart cities.

Surface fatigue crack segmentation in steel box girder bridges using an improved Unet convolutional neural network

Despite the success of crack segmentation, the traditional segmentation strategy and networks are not necessarily suitable for fatigue cracks with unique visual features. Towards the indistinct crack features, a network named Fatigue Crack Unet (FC-Unet) was proposed, where the novel split attention mechanism was incorporated to boost diverse feature extraction. Scaled by various ratios, raw images were cropped and constituted three datasets to represent different segmentation strategies. FC-Unet was trained and tested on each dataset, where the class imbalance problem was alleviated by the adjustment of loss functions. Results showed that the segmentation performance is mainly limited to noise interference. Preserving abundant global contexts to distinguish noises, the global inference is the superior strategy to the previous local inference. Leveraging the wide receptive field in global inference, the down-sampling rate can be decreased to save resolution losses. Combining the focus-based and region-based loss functions, the accuracy of imbalanced data was further improved. Compared with nine classical networks, the proposed FC-Unet enjoyed a powerful backbone and achieved a superior metric of 83.1% mean intersection over union (MIoU).

MRI-Based Brain Tumor Classification Using a Dilated Parallel Deep Convolutional Neural Network

Brain tumors are frequently classified with high accuracy using convolutional neural networks (CNNs) to better comprehend the spatial connections among pixels in complex pictures. Due to their tiny receptive fields, the majority of deep convolutional neural network (DCNN)-based techniques overfit and are unable to extract global context information from more significant regions. While dilated convolution retains data resolution at the output layer and increases the receptive field without adding computation, stacking several dilated convolutions has the drawback of producing a grid effect. This research suggests a dilated parallel deep convolutional neural network (PDCNN) architecture that preserves a wide receptive field in order to handle gridding artifacts and extract both coarse and fine features from the images. This article applies multiple preprocessing strategies to the input MRI images used to train the model. By contrasting various dilation rates, the global path uses a low dilation rate (2,1,1), while the local path uses a high dilation rate (4,2,1) for decremental even numbers to tackle gridding artifacts and to extract both coarse and fine features from the two parallel paths. Using three different types of MRI datasets, the suggested dilated PDCNN with the average ensemble method performs best. The accuracy achieved for the multiclass Kaggle dataset-III, Figshare dataset-II, and binary tumor identification dataset-I is 98.35%, 98.13%, and 98.67%, respectively. In comparison to state-of-the-art techniques, the suggested structure improves results by extracting both fine and coarse features, making it efficient.

ESatSR: Enhancing Super-Resolution for Satellite Remote Sensing Images with State Space Model and Spatial Context

Super-resolution (SR) for satellite remote sensing images has been recognized as crucial and has found widespread applications across various scenarios. Previous SR methods were usually built upon Convolutional Neural Networks and Transformers, which suffer from either limited receptive fields or a lack of prior assumptions. To address these issues, we propose ESatSR, a novel SR method based on state space models. We utilize the 2D Selective Scan to obtain an enhanced capability in modeling long-range dependencies, which contributes to a wide receptive field. A Spatial Context Interaction Module (SCIM) and an Enhanced Image Reconstruction Module (EIRM) are introduced to combine image-related prior knowledge into our model, therefore guiding the process of feature extraction and reconstruction. Tailored for remote sensing images, the interaction of multi-scale spatial context and image features is leveraged to enhance the network’s capability in capturing features of small targets. Comprehensive experiments show that ESatSR demonstrates state-of-the-art performance on both OLI2MSI and RSSCN7 datasets, with the highest PSNRs of 42.11 dB and 31.42 dB, respectively. Extensive ablation studies illustrate the effectiveness of our module design.

Edge‐guided representation learning for underwater object detection

AbstractUnderwater object detection (UOD) is crucial for marine economic development, environmental protection, and the planet's sustainable development. The main challenges of this task arise from low‐contrast, small objects, and mimicry of aquatic organisms. The key to addressing these challenges is to focus the model on obtaining more discriminative information. The authors observe that the edges of underwater objects are highly unique and can be distinguished from low‐contrast or mimicry environments based on their edges. Motivated by this observation, an Edge‐guided Representation Learning Network, termed ERL‐Net is proposed, that aims to achieve discriminative representation learning and aggregation under the guidance of edge cues. Firstly, an edge‐guided attention module is introduced to model the explicit boundary information, which generates more discriminative features. Secondly, a hierarchical feature aggregation module is proposed to aggregate the multi‐scale discriminative features by regrouping them into three levels, effectively aggregating global and local information for locating and recognising underwater objects. Finally, a wide and asymmetric receptive field block is proposed to enable features to have a wider receptive field, allowing the model to focus on smaller object information. Comprehensive experiments on three challenging underwater datasets show that our method achieves superior performance on the UOD task.

Efficient FPGA Binary Neural Network Architecture for Image Super-Resolution

Super-resolution systems refer to computer-based systems designed to enhance the quality of images or video by producing high-resolution renditions from low-resolution counterparts using computational algorithms and technologies. Various methods and techniques have been used in development of super-resolution systems. The development of Convolution Neural Networks (CNNs) and the Deep Learning (DL) methods have outperformed traditional methods. However, as models become increasingly deeper with wider receptive fields, the number of parameters significantly increases. While this often results in better performance, it renders these models impractical for real-life scenarios such as smartphones or other mobile systems. Currently, most proposed methods with higher perceptual quality demand a substantial amount of time to process a single image, even on powerful hardware like NVIDIA GPUs. Such computationally expensive models are not cost-effective for real-world application scenarios. Optimization is needed to reduce the computational costs and memory requirements to enhance their suitability for less powerful hardware configurations. In this work, we propose an efficient binary neural network architecture, ResBinESPCN, designed for image super-resolution. In our design, we improved the energy efficiency of the architecture through algorithmic and hardware-level optimizations. These optimizations not only enhance computational efficiency and reduce memory consumption but also achieve effective image super-resolution in resource-constrained environments. Our experimental validation highlights the effectiveness of this network structure and includes ablation studies on models with varying data bit widths. Hardware analysis substantiates the efficiency and real-time capabilities of this model. Additionally, deploying the model on FPGA using FINN demonstrates its low hardware resource usage and low power consumption.

SwinHR: Hemodynamic-powered hierarchical vision transformer for breast tumor segmentation

Accurate and automated segmentation of breast tumors in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) plays a critical role in computer-aided diagnosis and treatment of breast cancer. However, this task is challenging, due to random variation in tumor sizes, shapes, appearances, and blurred boundaries of tumors caused by inherent heterogeneity of breast cancer. Moreover, the presence of ill-posed artifacts in DCE-MRI further complicate the process of tumor region annotation. To address the challenges above, we propose a scheme (named SwinHR) integrating prior DCE-MRI knowledge and temporal–spatial information of breast tumors. The prior DCE-MRI knowledge refers to hemodynamic information extracted from multiple DCE-MRI phases, which can provide pharmacokinetics information to describe metabolic changes of the tumor cells over the scanning time. The Swin Transformer with hierarchical re-parameterization large kernel architecture (H-RLK) can capture long-range dependencies within DCE-MRI while maintaining computational efficiency by a shifted window-based self-attention mechanism. The use of H-RLK can extract high-level features with a wider receptive field, which can make the model capture contextual information at different levels of abstraction. Extensive experiments are conducted in large-scale datasets to validate the effectiveness of our proposed SwinHR scheme, demonstrating its superiority over recent state-of-the-art segmentation methods. Also, a subgroup analysis split by MRI scanners, field strength, and tumor size is conducted to verify its generalization. The source code is released on (https://github.com/GDPHMediaLab/SwinHR).

Severe Precipitation Recognition Using Attention-UNet of Multichannel Doppler Radar

Quantitative precipitation estimation (QPE) plays an important role in meteorology and hydrology. Currently, multichannel Doppler radar image is used for QPE based on some traditional methods like the Z − R relationship, which struggles to capture the complicated non-linear spatial relationship. Encouraged by the great success of using Deep Learning (DL) segmentation networks in medical science and remoting sensing, a UNet-based network named Reweighted Regression Encoder–Decoder Net (RRED-Net) is proposed for QPE in this paper, which can learn more complex non-linear information from the training data. Firstly, wavelet transform (WT) is introduced to alleviate the noise in radar images. Secondly, a wider receptive field is obtained by taking advantage of attention mechanisms. Moreover, a new Regression Focal Loss is proposed to handle the imbalance problem caused by the extreme long-tailed distribution in precipitation. Finally, an efficient feature selection strategy is designed to avoid exhaustion experiments. Extensive experiments on 465 real processes data demonstrate that the superiority of our proposed RRED-Net not only in the threat score (TS) in the severe precipitation (from 17.6% to 39.6%, ≥20 mm/h) but also the root mean square error (RMSE) comparing to the traditional Z-R relationship-based method (from 2.93 mm/h to 2.58 mm/h, ≥20 mm/h), baseline models and other DL segmentation models.

A Dual-Branch Deep Learning Architecture for Multisensor and Multitemporal Remote Sensing Semantic Segmentation

Multi-sensor data analysis allows exploiting heterogeneous data regularly acquired by the many available Remote Sensing (RS) systems. Machine- and deep-learning methods use the information of heterogeneous sources to improve the results obtained by using single-source data. However, the State-of-the-Art (SoA) methods analyze either the multi-scale information of multi-sensor multi-resolution images or the time component of image time series. We propose a supervised Deep-Learning (DL) classification method that jointly performs a multi-scale and multi-temporal analysis of RS multi-temporal images acquired by different sensors. The proposed method processes Very-High-Resolution (VHR) images using a Residual Network (ResNet) with a wide receptive field that handles geometrical details and multi-temporal High-Resolution (HR) image using a 3D Convolutional Neural Network (3D-CNN) that analyzes both the spatial and temporal information. The multi-scale and multi-temporal features are processed together in a decoder to retrieve a land-cover map. We tested the proposed method on two multi-sensor and multi-temporal datasets. One is composed of VHR orthophotos and Sentinel-2 multi-temporal images for pasture classification, and another is composed of VHR orthophotos and Sentinel-1 multi-temporal images. Results proved the effectiveness of the proposed classification method.

WRA-Net: Wide Receptive Field Attention Network for Motion Deblurring in Crop and Weed Image.

Automatically segmenting crops and weeds in the image input from cameras accurately is essential in various agricultural technology fields, such as herbicide spraying by farming robots based on crop and weed segmentation information. However, crop and weed images taken with a camera have motion blur due to various causes (e.g., vibration or shaking of a camera on farming robots, shaking of crops and weeds), which reduces the accuracy of crop and weed segmentation. Therefore, robust crop and weed segmentation for motion-blurred images is essential. However, previous crop and weed segmentation studies were performed without considering motion-blurred images. To solve this problem, this study proposed a new motion-blur image restoration method based on a wide receptive field attention network (WRA-Net), based on which we investigated improving crop and weed segmentation accuracy in motion-blurred images. WRA-Net comprises a main block called a lite wide receptive field attention residual block, which comprises modified depthwise separable convolutional blocks, an attention gate, and a learnable skip connection. We conducted experiments using the proposed method with 3 open databases: BoniRob, crop/weed field image, and rice seedling and weed datasets. According to the results, the crop and weed segmentation accuracy based on mean intersection over union was 0.7444, 0.7741, and 0.7149, respectively, demonstrating that this method outperformed the state-of-the-art methods.

Automatic lower limb bone segmentation in radiographs with different orientations and fields of view based on a contextual network.

Bone identification and segmentation in X-ray images are crucial in orthopedics for the automation of clinical procedures, but it often involves some manual operations. In this work, using a modified SegNet neural network, we automatically identify and segment lower limb bone structures on radiographs presenting various fields of view and different patient orientations. A wide contextual neural network architecture is proposed to perform a high-quality pixel-wise semantic segmentation on X-ray images presenting structures with a similar appearance and strong superposition. The proposed architecture is based on the premise that every output pixel on the label map has a wide receptive field. This allows the network to capture both global and local contextual information. The overlapping between structures is handled with additional labels. The proposed approach was evaluated on a test dataset composed of 70 radiographs with entire and partial bones. We obtained an average detection rate of 98.00% and an average Dice coefficient of 95.25 ± 9.02% across all classes. For the challenging subset of images with high superposition, we obtained an average detection rate of 96.36% and an average Dice coefficient of 93.81 ± 10.03% across all classes. The results show the effectiveness of the proposed approach in segmenting and identifying lower limb bone structures and overlapping structures in radiographs with strong bone superposition and highly variable configurations, as well as in radiographs containing only small pieces of bone structures.

Temporal convolution network based novel parallel disaggregation method for non-intrusive monitoring of the appliances’ consumptions in residential buildings

Energy or load disaggregation, as one essential part of non-intrusive load monitoring (NILM), is an efficient way to separate the consumption information of target appliances from the whole consumption data, and can accordingly help to regulate people’s energy consumption behaviors. However, the consumptions of the target appliances are usually affected by the variance of the opening time, working condition and user interference, so it is a difficult task to realize precise disaggregation. To further improve the energy disaggregation accuracy, this paper proposes a new parallel disaggregation strategy with two subnets for the energy consumption disaggregation of the target appliances in the residential buildings. In the proposed strategy, the parallel disaggregation network contains a long-term disaggregation network and a short-term disaggregation network, which can automatically and respectively learn the long-term trend features and short-term dynamic characteristics of the electrical appliances. This parallel structure can make full use of the advantages of different methods in feature extraction, so as to model the appliance features more comprehensively. To better extract the long-term and short-term features, in the long-term disaggregation subnet, we propose the double branch bi-directional temporal convolution network (DBB-TCN) which has a wider receptive field than the traditional temporal convolution networks (TCN), while in the short-term disaggregation subnet, we adopt the convolution auto-encoder to learn the short-term characteristics of the target appliances. Finally, detailed experiments and comparisons are made with two real-world datasets. Experimental results verified that the proposed parallel disaggregation method performs better than the existing methods under various evaluation criteria.

Fuzzy STUDENT’S T-Distribution Model Based on Richer Spatial Combination

Fuzzy c-means (FCM) algorithms with spatial information have been widely applied in the field of image segmentation. However, most of them suffer from two challenges. One is that the introduction of fixed or adaptive single neighboring information with narrow receptive field limits contextual constraints leading to clutter segmentations. The other is that the incorporation of superpixels with wide receptive field enlarges spatial coherency leading to block effects. To address these challenges, we propose fuzzy STUDENT’S t-distribution model based on richer spatial combination (FRSC) for image segmentation. In this article, we make two significant contributions. The first is that both the narrow and wide receptive fields are integrated into the objective function of FRSC, which is convenient to mine image features and distinguish local difference. The second is that the rich spatial combination under STUDENT’S t-distribution ensures that spatial information is introduced into the updated parameters of FRSC, which is helpful in finding a balance between the noise-immunity and detail-preservation. Experimental results on synthetic and publicly available images further demonstrate that the proposed FRSC addresses successfully the limitations of FCM algorithms with spatial information, and provides better segmentation results than state-of-the-art clustering algorithms.

Investigation of nonlocal data-driven methods for subgrid-scale stress modeling in large eddy simulation

A nonlocal subgrid-scale stress (SGS) model is developed based on the convolution neural network (CNN), which is a powerful supervised data-driven method and also an ideal approach to naturally consider spatial information due to its wide receptive field. The CNN-based models used in this study take primitive flow variables as input only, and then, the flow features are automatically extracted without any a priori guidance. The nonlocal models trained by direct numerical simulation (DNS) data of a turbulent channel flow at Reτ = 178 are accessed in both the a priori and a posteriori tests, providing reasonable flow statistics (such as mean velocity and velocity fluctuations) close to the DNS results even when extrapolating to a higher Reynolds number Reτ = 600. It is identified that the nonlocal models outperform local data-driven models, such as the artificial neural network, and some typical SGS models (e.g., the dynamic Smagorinsky model) in large eddy simulation (LES). The model is also robust with stable numerical simulation since the solutions can be well obtained when examining the grid resolution from one-half to double of the spatial resolution used in training. We also investigate the influence of receptive fields and propose using the two-point correlation analysis as a quantitative method to guide the design of nonlocal physical models. The present study provides effective data-driven nonlocal methods for SGS modeling in LES of complex anisotropic turbulent flows.

Towards an image-based brightness model for self-luminous stimuli.

Brightness is one of the most important perceptual correlates of color appearance models (CAMs) when self-luminous stimuli are targeted. However, the vast majority of existing CAMs adopt the presence of a uniform background surrounding the stimulus, which severely limits their practical application in lighting. In this paper, a study on the brightness perception of a neutral circular stimulus surrounded by a non-uniform background consisting of a neutral ring-shaped luminous area and a dark surround is presented. The ring-shaped luminous area is presented with 3 thicknesses (0.33 cm, 0.67 cm and 1.00 cm), at 4 angular distances to the edge of the central stimulus (1.2°, 6.4°, 11.3° and 16.1°) and at 3 luminance levels (90 cd/m2, 335 cd/m2, 1200 cd/m2). In line with the literature, the results of the visual matching experiments show that the perceived brightness decreases in presence of a ring and the effect is maximal at the highest luminance of the ring, for the largest thickness and at the closest distance. Based on the observed results, an image-based model inspired by the physiology of the retina is proposed. The model includes the calculation of cone-fundamental weighted spectral radiance, scattering in the eye, cone compression and receptive field post-receptor organization. The wide receptive field assures an adaptive shift determined by both the adaptation to the stimulus and to the background. It is shown that the model performs well in predicting the matching experiments, including the impact of the thickness, the distance and the intensity of the ring, showing its potential to become the basic framework of a Lighting Appearance Model.

C3D-UNET: A Comprehensive 3D Unet for Covid-19 Segmentation with Intact Encoding and Local Attention.

For COVID-19 prevention and treatment, it is essential to screen the pneumonia lesions in the lung region and analyze them in a qualitative and quantitative manner. Three-dimensional (3D) computed tomography (CT) volumes can provide sufficient information; however, extra boundaries of the lesions are also needed. The major challenge of automatic 3D segmentation of COVID-19 from CT volumes lies in the inadequacy of datasets and the wide variations of pneumonia lesions in their appearance, shape, and location. In this paper, we introduce a novel network called Comprehensive 3D UNet (C3D-UNet). Compared to 3D-UNet, an intact encoding (IE) strategy designed as residual dilated convolutional blocks with increased dilation rates is proposed to extract features from wider receptive fields. Moreover, a local attention (LA) mechanism is applied in skip connections for more robust and effective information fusion. We conduct five-fold cross-validation on a private dataset and independent offline evaluation on a public dataset. Experimental results demonstrate that our method outperforms other compared methods.

Wide receptive field networks for single image super-resolution

Wide receptive field networks for single image super-resolution

Deep learning framework based on Spectral and Spatial properties for Land-Cover Classification using Landsat Hyperspectral Images

Hyperspectral Imaging is used to monitor the earth on basis of spectral continuous data ranges starting from visible to short wave infrared region of the electromagnetic spectrum. It enables the detailed identification and classification of minerals and land cover on basis of with improved spectral and spatial resolutions provides the opportunity to obtain accurate land-cover classification. Several challenges have been generated due to Hughes phenomenon (curse of dimensionality) and Quantification of land cover in urban area. In order to alleviate those problems, novel framework named as Deep learning framework on spectral and spatial properties on Landsat image has been proposed which composed of several techniques. Initially hyperspectral (HS) data exploitation model on identification of pure spectral signatures (endmembers) and their corresponding fractional abundances in each pixel of the HS data cube has been proposed. Feature reduction strategy based on principle component analysis has been employed to generate reduced dimensionality of the features on retaining the most useful information. The reduced features have been taken for the spectral analysis and spatial analysis using Multiobjective Discrete Spectral and Spatial optimized representation model through encompassing the sparse and low-rank structure on the spectral signature of pixels. Identification and mapping of the land cover classification categorized as agriculture area and bare land has been identified using spectral indices (end members). The spectral indices calculation provides the type of land cover on the pixel purity index and it classifies based on the spectral and spatial value using N finder algorithm. N finder Algorithm is a change vector analysis. Further Ensemble based method has been proposed in addition to generate diverse classification results and the discrete high correlation classifier method which can enhance the accuracy and diversity of a single classifier simultaneously. Finally an efficient agriculture land cover spectral evolution mapping has been proposed using Multivariate principle component analysis. It is considered as change detection method explores efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of Context towards mapping Agriculture Land cover spectral evolution. It computes the spectral correlation between two images on spectral similarity. It predicts the accurately on temporal changes of the earth surfaces. Experimental analysis has been carried out using Landsat-8 dataset to evaluating the performance of the proposed representative framework using available spectral indices against the state of art approaches. Proposed framework achieves accuracy of 99% on reflectance value against the different wavelength which superior with other existing classification approaches.

Single shot multi-oriented text detection based on local and non-local features

In order to improve the robustness of text detector on scene text of various scales, a single shot text detector that combines local and non-local features is proposed in this paper. A dilated inception module for local feature extraction and a text self-attention module for non-local feature extraction are presented, and these two kinds of modules are integrated into single shot detector (SSD) of generic object detection so as to perform multi-oriented text detection in natural scene. The proposed modules make a contribution to richer and wider receptive field and enhance feature representation. Furthermore, the performance of our text detector is improved. In addition, compared with previous text detectors based on SSD which classify positive and negative samples depending on default boxes, we exploit pixels as reference for more accurate matching with ground truth which avoids complex anchor design. Furthermore, to evaluate the effectiveness of the proposed method, we carry out several comparative experiments on public standard benchmarks and analyze the experimental results in detail. The experimental results illustrate that the proposed text detector can compete with the state-of-the-art methods.