Coarse Prediction Research Articles

A three-stage model for camouflaged object detection

Camouflaged objects are typically assimilated into their backgrounds and exhibit fuzzy boundaries. The complex environmental conditions and the high intrinsic similarity between camouflaged targets and their surroundings pose significant challenges in accurately locating and segmenting these objects in their entirety. While existing methods have demonstrated remarkable performance in various real-world scenarios, they still face limitations when confronted with difficult cases, such as small targets, thin structures, and indistinct boundaries. Drawing inspiration from human visual perception when observing images containing camouflaged objects, we propose a three-stage model that enables coarse-to-fine segmentation in a single iteration. Specifically, our model employs three decoders to sequentially process subsampled features, cropped features, and high-resolution original features. This proposed approach not only reduces computational overhead but also mitigates interference caused by background noise. Furthermore, considering the significance of multi-scale information, we have designed a multi-scale feature enhancement module that enlarges the receptive field while preserving detailed structural cues. Additionally, a boundary enhancement module has been developed to enhance performance by leveraging boundary information. Subsequently, a mask-guided fusion module is proposed to generate fine-grained results by integrating coarse prediction maps with high-resolution feature maps. Our network shows superior performance without introducing unnecessary complexities. Upon acceptance of the paper, the source code will be made publicly available at https://github.com/clelouch/TSNet.

Electron Microscopy Images as Set of Fragments for Mitochondrial Segmentation

Automatic mitochondrial segmentation enjoys great popularity with the development of deep learning. However, the coarse prediction raised by the presence of regular 3D grids in previous methods regardless of 3D CNN or the vision transformers suggest a possibly sub-optimal feature arrangement. To mitigate this limitation, we attempt to interpret the 3D EM image stacks as a set of interrelated 3D fragments for a better solution. However, it is non-trivial to model the 3D fragments without introducing excessive computational overhead. In this paper, we design a coherent fragment vision transformer (FragViT) combined with affinity learning to manipulate features on 3D fragments yet explore mutual relationships to model fragment-wise context, enjoying locality prior without sacrificing global reception. The proposed FragViT includes a fragment encoder and a hierarchical fragment aggregation module. The fragment encoder is equipped with affinity heads to transform the tokens into fragments with homogeneous semantics, and the multi-layer self-attention is used to explicitly learn inter-fragment relations with long-range dependencies. The hierarchical fragment aggregation module is responsible for hierarchically aggregating fragment-wise prediction back to the final voxel-wise prediction in a progressive manner. Extensive experimental results on the challenging MitoEM, Lucchi, and AC3/AC4 benchmarks demonstrate the effectiveness of the proposed method.

TSRNet: Tongue image segmentation with global and local refinement

Tongue Image Segmentation is an essential task for intelligent Traditional Chinese Medicine (TCM), as the tongue is sensitive to the physiological conditions and pathological changes of patients and can help physicians determine strategies for the syndrome differentiation. However, it is a big challenge to acquire an accurate tongue segmentation mask, due to the varied shape and texture of the tongue. This paper proposes a novel tongue segmentation network based on an encoder–decoder framework with global and local refinement, named TSRNet. In the global refinement module, we design an effective module for fusing features from an autoencoder, which is pre-trained on tongue images with segmentation labels, so that the network can make use of the prior knowledge. Moreover, in the local refinement module, we perform patch sampling according to the coarse prediction boundary and correct errors through a patch segmentation module. Both two modules are plugged into the decoder to obtain better tongue segmentation results by training end-to-end. Experimental results compared with state-of-the-art models on two real-world tongue datasets demonstrate the effectiveness of the proposed TSRNet.

A Dual Self-Calibrating Framework for Noninvasive Fetal ECG R-Peak Detection

Fetal heart rate (fHR) is critical for assessing fetal health and diagnosing disorders such as fetal distress, congenital heart disease, and intrauterine growth retardation. With the rapid development of the Internet of Medical Things (IoT), fetal R-peak detection plays an important role in diagnosing heart defects during pregnancy. However, due to the nonlinear mixing of multiple sources in the non-invasive signals and the low signal-to-noise ratio (SNR), it is difficult to obtain accurate R-peak detection result. This article presents a dual Self-calibrating system based on a Spectral Attention Kernel Independent Component Analysis (SA-KICA) module and a Selfcalibrating fetal R-peak detection (SC-FRD) module. SA-KICA is an ICA-based calibration module constructed by SA mechanism, which was sought from Short-Time Fourier Transform (STFT) and was shipped back to original signal with convolution to achieve perfect maternal electrocardiogram (MECG) separation in high-dimensional linear separable space. Then a periodic and morphological based Channel Selector is designed to select the optimal MECG. After MECG removal, to further improve the performance of fetal R-peak detection, the SC-FRD module is introduced to utilize the interior peak information and selfcalibrating strategy, which includes variance-based fetal R-peak seed selection, time-varying coarse prediction and adaptive probability mask calibration. The proposed framework is a primary attempt to concurrently introduce the nonlinear feature, spectral information and self-calibrating strategy in the field of fetal ECG processing. The framework achieved excellent performance in fetal R-peak detection accuracy on a simulated dataset and two public datasets with varying divergence and richness of resources. The experimental results show that our framework is superior to existing methods and can be used as a potential fetal monitoring method in the application of Internet of Medical Things. The code is released in https://github.com/bfyjr/NI-FECG-Extraction.

A Combination of FPU-Net and Feature Clustering Methods for Accurate Segmentation of Femoral Neck in Radiographic Diagnosis.

In this study, we develop an innovative method that assists computer-aided diagnosis in the determination process of the exact location of the femoral neck junction in plain radiographs. Our algorithm consists of two phases, i.e., coarse prediction and fine matching, which are implemented by supervised deep learning method and unsupervised clustering, respectively. In coarse prediction, standard masks are first produced by a specialist and trained in our proposed feature propagation network (FPU-Net) with supervised learning on the femoral neck dataset. In fine matching, the standard masks are first classified into different categories using our proposed three parameters with unsupervised learning. The predicted mask from FPU-Net is matched with each category of standard masks by calculating the values of intersection of union (IOU), and finally the predicted mask is substituted by the standard mask with the largest IOU value. A total of 4320 femoral neck parts in anterior-posterior (AP) pelvis radiographs collected from China Medical University Hospital database were used to test our method. Simulation results show that, on the one hand, compared with other segmentation methods, the method proposed in this paper has a larger IOU value and better suppression of noise outside the region of interest; on the other hand, the introduction of unsupervised learning for fine matching can help in the accurate localization segmentation of femoral neck images. Accurate femoral neck segmentation can assist surgeons to diagnose and reduce the misdiagnosis rate and burden.

CG-3DSRGAN: A classification guided 3D generative adversarial network for image quality recovery from low-dose PET images.

Positron emission tomography (PET) is the most sensitive molecular imaging modality routinely applied in our modern healthcare. High radioactivity caused by the injected tracer dose is a major concern in PET imaging and limits its clinical applications. However, reducing the dose leads to inadequate image quality for diagnostic practice. Motivated by the need to produce high quality images with minimum 'low-dose', convolutional neural networks (CNNs) based methods have been developed for high quality PET synthesis from its low-dose counterparts. Previous CNNs-based studies usually directly map low-dose PET into features space without consideration of different dose reduction level. In this study, a novel approach named CG-3DSRGAN (Classification-Guided Generative Adversarial Network with Super Resolution Refinement) is presented. Specifically, a multi-tasking coarse generator, guided by a classification head, allows for a more comprehensive understanding of the noise-level features present in the low-dose data, resulting in improved image synthesis. Moreover, to recover spatial details of standard PET, an auxiliary super resolution network - Contextual-Net - is proposed as a second-stage training to narrow the gap between coarse prediction and standard PET. We compared our method to the state-of-the-art methods on whole-body PET with different dose reduction factors (DRF). Experiments demonstrate our method can outperform others on all DRF.Clinical Relevance- Low-Dose PET, PET recovery, GAN, task driven image synthesis, super resolution.

FRBNet: Feedback refinement boundary network for semantic segmentation in breast ultrasound images

Ultrasound breast tumour segmentation is a key step in computer-aided diagnosis and provides an important basis for clinical diagnosis and analysis. Accurate segmentation of breast tumours from ultrasound images is a challenging task due to the characteristics of black shadows, blurred boundaries and uneven colour intensity variations between classes. Currently, most breast tumour segmentation methods focus on extracting multi-scale information and fusing contextual information while underestimating the importance of feature information that can assist in identifying object boundaries in segmentation tasks. The loss of boundary feature information can easily lead to discontinuity or inaccuracy of the target boundary when the network generates the final prediction map. To address this problem, we propose a new feedback refinement boundary network (FRBNet) for accurate segmentation of breast tumour regions in ultrasound images, which mainly consists of a channel calibration module (CCM), boundary detection (BD) module, and feedback refinement module (FRM). Specifically, before fusing low-level feature maps with high-level feature maps, CCM first adopts the method of redistributing feature channel responses to enhance the channels carrying key target information and suppress the noisy channels in low-level feature maps. The BD module then improves the quality of the boundaries in the segmentation results by additionally learning the boundaries of breast tumours to provide accurate boundary feature information for subsequent prediction. The FRM employs a feedback mechanism that complementarily fuses the coarse prediction map and the feature map containing the target boundary feature information, thus achieving the best prediction results before generating the final prediction map. Experimental results on a public ultrasound breast dataset show that our network outperforms other medical image segmentation methods.

Progressive Neighborhood Aggregation for Semantic Segmentation Refinement

Multi-scale features from backbone networks have been widely applied to recover object details in segmentation tasks. Generally, the multi-level features are fused in a certain manner for further pixel-level dense prediction. Whereas, the spatial structure information is not fully explored, that is similar nearby pixels can be used to complement each other. In this paper, we investigate a progressive neighborhood aggregation (PNA) framework to refine the semantic segmentation prediction, resulting in an end-to-end solution that can perform the coarse prediction and refinement in a unified network. Specifically, we first present a neighborhood aggregation module, the neighborhood similarity matrices for each pixel are estimated on multi-scale features, which are further used to progressively aggregate the high-level feature for recovering the spatial structure. In addition, to further integrate the high-resolution details into the aggregated feature, we apply a self-aggregation module on the low-level features to emphasize important semantic information for complementing losing spatial details. Extensive experiments on five segmentation datasets, including Pascal VOC 2012, CityScapes, COCO-Stuff 10k, DeepGlobe, and Trans10k, demonstrate that the proposed framework can be cascaded into existing segmentation models providing consistent improvements. In particular, our method achieves new state-of-the-art performances on two challenging datasets, DeepGlobe and Trans10k. The code is available at https://github.com/liutinglt/PNA.

Extraction of vascular wall in carotid ultrasound via a novel boundary-delineation network

Ultrasound imaging plays an essential role in the diagnosis of vascular lesions. Accurate segmentation of the vascular wall is important for preventing, diagnosing, and treating vascular diseases. However, existing methods have inaccurate localization of the vascular wall boundary. Segmentation errors occur in discontinuous vascular wall boundaries and dark boundaries. To overcome these problems, we propose a new boundary-delineation network (BDNet). First, we design the feature extraction module to prevent the feature information loss of low-quality images by multi-scale information fusion and multi-receptive field feature fusion. Secondly, we generate the initial coarse prediction results based on the extracted features. Based on the coarse prediction results, we use the boundary refinement module to obtain the boundary point locations and re-delineate the boundary points to prevent the boundary points from the offset. Finally, we use region mutual information loss and our designed global pixel relationship loss to model the relationship between pixels from global and neighbourhood aspects using the structural features of the vessel wall to help the model extract important structured information. To facilitate clinical applications, we design the model to be lightweight. Experimental results show that our model achieves the best segmentation results and significantly reduces memory consumption compared to existing models.

Learned visual localization with camera pose refinement and verification based on differentiable renderer

This manuscript presents a new CNN-based visual localization method that seeks a camera location of an input RGB image with respect to a pre-collected RGB-D images database. To determine an accurate camera pose, we employ a coarse-to-fine localization manner that firstly finds coarse location candidates via image retrieval, then refines them using local 3D structure represented by each retrieved RGB-D image. We use a CNN feature extractor and a relative pose estimator for coarse prediction that do not sufficiently require a scene-specific training. Furthermore, we propose a new pose refinement-verification module that simultaneously evaluates and refines camera poses using differentiable renderer. Experimental results on public datasets show that our proposed pipeline achieves accurate localization on both trained and unknown scenes.

Salient object detection based on edge‐interior feature fusion

Recently, existing FCNs-based methods have shown their advantages in processing object boundaries. However, these methods still suffer from false object interference, which appears in saliency predictions. To solve this problem, an edge-interior feature fusion (EIFF) framework is proposed, which consists of an internal-boundary decoupled generation structure with receptive field enlargement and attention mechanism enhancement, and a salient feature refinement module. Specifically, the framework first learns edge features and interior features through an internal-boundary decoupling generation network, which is supervised by labels obtained by decoupling ground-truth through an image erosion algorithm. Then, feature refinement module (FRM) is designed to purify the coarse prediction by focusing on the ambiguous regions through a mining strategy to generate the final saliency map. To compensate for shortcomings of the BCE and IU loss, we also introduce a weighted loss to guide our model to focus more on the error-prone parts. Experimental results on five benchmark datasets demonstrate that the proposed method performs favorably against 19 state-of-the-art approaches under four standard metrics.

Enterprise Credit Security Prediction and Evaluation Based on Multimodel Fusion

Based on the industry data and enterprise data from tens of thousands of small and medium-sized enterprises, a deep learning and machine learning model of credit prediction is constructed through the division of data sets, processing, and integration of models. At first, with the help of two characteristic selection methods, several subsets separated from the dataset are analyzed based on convolutional neural network as coarse prediction. Then, combined with the tree model, the precise prediction is further made for the enterprise credit evaluation. Finally, the model fusion is carried out to obtain high-precision results. In the simulation experiment, this paper takes a data set of 14,366 small and medium-sized enterprise credit evaluations as the analysis samples to verify the results. The accuracy of the model is 97%, which is far more than 93% of single model with metadata set.

Recognition of fatigue and severe sliding wear particles using a CNN model with multi-scale feature extractor

PurposeThe purpose of this study is to provide a new convolutional neural network (CNN) model with multi-scale feature extractor to segment and recognize wear particles in complex ferrograph images, especially fatigue and severe sliding wear particles, which are similar in morphology while different in wear mechanism.Design/methodology/approachA CNN model named DWear is proposed to semantically segment fatigue, severe sliding particles and four other types of particles, that is, chain, spherical, cutting and oxide particles, which unifies segmentation and recognition together. DWear is constructed using four modules, namely, encoder, densely connected atrous spatial pyramid pooling, decoder and fully connected conditional random field. Different from the architectures of ordinary semantic segmentation CNN models, a multi-scale feature extractor using cascade connections and a coprime atrous rate group is incorporated into the DWear model to obtain multi-scale receptive fields and better extract features of wear particles. Moreover, fully connected conditional random field module is adopted for post-processing to smooth coarse prediction and obtain finer results.FindingsDWear is trained and verified on the ferrograph image data set, and experimental results show that the final Mean Pixel Accuracy is 95.6% and the Mean Intersection over Union is 92.2%, which means that the recognition and segmentation accuracy is higher than those of previous works.Originality/valueDWear provides a promising approach for wear particle analysis and can be further developed in equipment condition monitoring applications.

Machine learning based data driven inkjet printed electronics: jetting prediction for novel inks

Machine learning (ML) as a predictive methodology can potentially reduce the configuration cost and workload of inkjet printing. Inkjet printing has many advantages for additive manufacturing and printed electronics including low cost, scalability, non-contact printing and on-demand customization. Inkjet generates droplets with a piezoelectric dispenser controlled through frequency, voltage pulse and timing parameters. A major challenge is the design of jettable inks and the rapid optimization of stable jetting conditions whilst preventing common problems (no ejection, perturbation, satellite drop, multiple drops, drop breaking, nozzle clogging). Material consuming trial and error experiments are replaced here with a ML based jetting window. A dataset of machine and material properties is created from literature and experimental data. After exploratory data analysis and feature identification, various (linear and non-linear) regression models are compared in detail. The models are trained on 80% of the data and root mean square error (RMSE) is calculated on 20% test data. Simple polynomial relationships between the input and output features yield coarse prediction. Instead, small ensembles of decision trees (DTs) (boosted DTs and random forests) have improved predictive power for drop velocity and radius with RMSE of 0.39 m s−1 and 2.21 µm respectively. The mean absolute percentage error is 3.87%. The models are validated with experimentally collected data for a novel ink where no data points with this ink were included in the training set. Additionally, several classification algorithms are utilized to categorize ink and printer parameters by jetting regime (‘single drop’, ‘multiple drops’, ‘no ejection’). Categorization and regression models are combined to improve overall model prediction. This article demonstrates that ML can be used to predict ink jetting behavior from 11 different ink and printing parameters. Different algorithms are analyzed and the optimal combination of algorithms is identified. It is shown that experimental and literature data can be combined and an initial dataset is created that other reserachers can build on in the future. ML enables efficient material and printing parameter selection speeding up the development of novel ink materials for printed electronics by eliminating jetting experiments that are money, time and material intensive.

ToStaGAN: An end-to-end two-stage generative adversarial network for brain tumor segmentation

Brain tumor segmentation using MRI data remains challenging for some reasons. Hence, how to accurately segment the brain tumor is kept as a significant topic in the area of medical image segmentation. To follow the idea of “coarse-to-fine” in practice, this paper proposes an end-to-end two-stage generative adversarial neural network (ToStaGAN) to improve the braintumor is kept as a significant topic in the area of medical image segmentation. To follow the idea of “coarse-to-fine” in practice, this paper proposes an end-to-end two-stage generative adversarial neural network (ToStaGAN) to improve the brain tumor segmentation performance by making full use of semantic information in high-level.In the proposed ToStaGAN, the UNET network is adopted as the “coarse” generation network in the first stage, while a U-Shaped contextual autoencoder (ConEnDer), which consists of the proposed fine-grained extraction module (FEM) and dense skip connection, is proposed as the “fine” generation network in the second stage. To be more specific, the FEM is mainly used to effectively extract “fine-grained” features that plays a crucial role in semantic segmentation with only increasing a few parameters. Furthermore, it has the ability to obtain more diverse and abstract features. And by further utilizing the features extracted from the FEM and the coarse prediction map from the first stage, the ConEnDer holds the great potential to generate the optimized segmentation results. We then evaluate the performance of ToStaGAN on the BARATS 2015. Findings from the evaluations suggest that the proposed two-stage generation network achieves a better performance than the one-stage network; thus, also implying the effectiveness of the proposed ConEnder for “fine” segmentation. A comparative summary also shows that the ToStaGAN achieves a better segmentation performance in comparison to other competing approaches.

Amodal Segmentation Based on Visible Region Segmentation and Shape Prior

Almost all existing amodal segmentation methods make the inferences of occluded regions by using features corresponding to the whole image. This is against the human's amodal perception, where human uses the visible part and the shape prior knowledge of the target to infer the occluded region. To mimic the behavior of human and solve the ambiguity in the learning, we propose a framework, it firstly estimates a coarse visible mask and a coarse amodal mask. Then based on the coarse prediction, our model infers the amodal mask by concentrating on the visible region and utilizing the shape prior in the memory. In this way, features corresponding to background and occlusion can be suppressed for amodal mask estimation. Consequently, the amodal mask would not be affected by what the occlusion is given the same visible regions. The leverage of shape prior makes the amodal mask estimation more robust and reasonable. Our proposed model is evaluated on three datasets. Experiments show that our proposed model outperforms existing state-of-the-art methods. The visualization of shape prior indicates that the category-specific feature in the codebook has certain interpretability. The code is available at https://github.com/YutingXiao/Amodal-Segmentation-Based-on-Visible-Region-Segmentation-and-Shape-Prior.

CF-CNN: Coarse-to-Fine Convolutional Neural Network

In this paper, we present a coarse-to-fine convolutional neural network (CF-CNN) for learning multilabel classes. The basis of the proposed CF-CNN is a disjoint grouping method that first creates a class group with hierarchical association, and then assigns a new label to a class belonging to each group so that each class acquires multiple labels. CF-CNN consists of one main network and two subnetworks. Each subnetwork performs coarse prediction using the group labels created by the disjoint grouping method. The main network includes a refine convolution layer and performs fine prediction to fuse the feature maps acquired from the subnetwork. The generated class set in the upper level has the same classification boundary to that in the lower level. Since the classes belonging to the upper level label are classified with a higher priority, parameter optimization becomes easier. In experimental results, the proposed method is applied to various classification tasks to show a higher classification accuracy by up to 3% with a much smaller number of parameters without modification of the baseline model.

Lightweight boundary refinement module based on point supervision for semantic segmentation

An effective semantic segmentation approach should contain fine object boundaries and continuous regions. However, recent mask-based segmentation cannot extract boundary features well on the coarse prediction, which causes obvious problems of blurry edges. Although several segmentation methods embed boundary detection branch to calculate contours directly, this type of architecture will lead to the increased computational complexity and miss the edge detailed information such as inter-class distinction. In order to obtain fine boundaries, we present a lightweight boundary refinement module with point supervision named BRPS to improve the boundary quality for the segmentation result generated by various existing segmentation models. Firstly, a direction field is learned to complete the initial feature rectification, which is defined as pointing away from the nearest object boundary to each pixel, where the weighted Euclidean and Cosine distance function is used as the loss between predicted boundary pixels and Ground-truth labels. Then, point-based supervised learning is performed at uncertain and certain locations including random distribution and key feature points based on a new point convolutional operation to output final crisp object boundaries. Finally, we verify that our BRPS module can effectively reduce the prediction errors for segmentation results generated from various state-of-the-art models such as DeepLabv3 and HRNet on the Pascal VOC2012, NYUD v2 datasets, Cityscapes and BDD100K datasets.

Crowd counting based on attention-guided multi-scale fusion networks

In this paper, we propose an attention-guided multi-scale fusion network (named as AMS-Net) for crowd counting in dense scenarios. The overall model is mainly comprised by the density and the attention networks. The density network is able to provide a coarse prediction of the crowd distribution (density map), while the attention network helps to distinguish crowded regions from backgrounds. The output of the attention network serves as a mask of the coarse density map. The number of persons in the scene is finally estimated by applying integration on the refined density map. In order to deal with persons of varied resolutions, we introduce a multi-scale fusion strategy which is built upon dilated convolution. Experiments are carried out on the standard benchmark datasets, covering varied over-crowded scenarios. Experimental results demonstrate the effectiveness of the proposed approach.

Two-Step Wind Power Prediction Approach With Improved Complementary Ensemble Empirical Mode Decomposition and Reinforcement Learning

The strong stochastic nature of wind power generation makes it extremely challenging to accurately predict and support the planning and operation of modern power systems with significant penetration of renewable energy. This article proposes a two-step wind power prediction method, which consists of two phases: long time-scale coarse prediction and short time-scale fine correction. In the long time-scale phase, a complementary ensemble empirical mode decomposition-based sigma point Kalman filter approach is proposed to coarsely predict wind power merely with historical data. In the short time-scale phase, a deep deterministic policy gradient approach learns from real-time weather information to correct the coarse prediction result, which results in an improved prediction accuracy. A real-life case study confirms that the proposed method can properly predict wind power generation and have a better prediction accuracy than existing techniques, thus offering a viable and promising alternative for predicting wind power generation.