Generalized Gradient Vector Flow Research Articles

A new deep CNN for 3D text localization in the wild through shadow removal

Text localization in the wild is challenging due to the presence of 2D and 3D texts, the presence of shadows, arbitrary orientated text with non-linear arrangements, varying lighting conditions as well as complex background. This paper proposes the first approach for 3D text localization in natural scene images through shadow removal and a new deep CNN model. In a first step, exploiting the observation that 3D text generates shadow information in natural scenes, the proposed model detects and removes the shadow pixels of 3D text based on the Generalized Gradient Vector Flow concept and a new clustering approach. The performance of the classification of 2D and 3D texts in the scene images is strengthened by using key features, including pixel strength, sharpness and edge potential, which are extracted to eliminate false text and shadow pixels. For text localization after removing shadow information, EfficientNet is used as an encoder (backbone) and UNet as a decoder in a novel way employing differential binarization. Experimental validation and comparative analysis with state-of-the-art approaches on both a new purpose-built dataset as well as on the benchmark datasets of ICDAR MLT 2019, ICDAR ArT 2019, CTW1500, DAST1500, Total-Text, and MSRATD500 for each of the different steps of the method, show that the proposed approach outperforms the existing methods.

A New Deep Wavefront Based Model for Text Localization in 3D Video

With the evolution of electronic devices, such as 3D cameras, addressing the challenges of text localization in 3D video (e.g., for indexing) is increasingly drawing the attention of the multimedia and video processing community. Existing methods focus on 2D video and their performance in the presence of the challenges in 3D video, such as shadow areas associated with text and irregularly sized and shaped text, degrades. This paper proposes the first approach that successfully addresses the challenges of 3D video in addition to those of 2D. It employs a number of innovations, among which, the first is the Generalized Gradient Vector Flow (GGVF) for dominant points detection. The second is the Wavefront concept for text candidate point detection from those dominant points. In addition, an Adaptive B-Spline Polygon Curve Network (ABS-Net) is proposed for accurate text localization in 3D videos by constructing tight fitting bounding polygons using text candidate points. Extensive experiments on custom (3D video) and standard datasets (2D video and scene text) show that the proposed method is practical and useful, and overall outperforms existing state-of-the-art methods.

Oil palm tree counting in drone images

When the images are captured by drones, the effect of oblique angles, distance variations and open environment are the main challenges for successful palm tree detection. This paper presents a method towards palm tree counting in Drone images using a novel idea of detecting dominant points by exploring Generalized Gradient Vector Flow, which defines symmetry based on gradient direction of the pixels. For each dominant point, we use angle information for classifying diagonal dominant points. It is intuition that the direction of the branches of tree converges at center of tree irrespective of the type of tree and plants. This observation motivated us to expand the direction of diagonal dominant points until it finds intersection point with another diagonal dominant point and this results in candidate points. For each candidate point, the proposed method constructs the ring by considering the distance between the intersection point and nearest neighbor candidate point as radius. This outputs region of interest and it includes center of each tree in the image. To ease the effect of complex background, we explore YOLOv5 architecture to remove false region of interests. This step results in counting oil palm trees in the mages irrespective of tree type of palm family. Experimental results on our dataset of the images captured by drones and standard dataset of coconut images captured by unmanned aerial vehicle of different trees show that the proposed method is effective and performs better than SOTA methods.

Object-Oriented Building Contour Optimization Methodology for Image Classification Results via Generalized Gradient Vector Flow Snake Model

Building boundary optimization is an essential post-process step for building extraction (by image classification). However, current boundary optimization methods through smoothing or line fitting principles are unable to optimize complex buildings. In response to this limitation, this paper proposes an object-oriented building contour optimization method via an improved generalized gradient vector flow (GGVF) snake model and based on the initial building contour results obtained by a classification method. First, to reduce interference from the adjacent non-building object, each building object is clipped via their extended minimum bounding rectangles (MBR). Second, an adaptive threshold Canny edge detection is applied to each building image to detect the edges, and the progressive probabilistic Hough transform (PPHT) is applied to the edge result to extract the line segments. For those cases with missing or wrong line segments in some edges, a hierarchical line segments reconstruction method is designed to obtain complete contour constraint segments. Third, accurate contour constraint segments for the GGVF snake model are designed to quickly find the target contour. With the help of the initial contour and constraint edge map for GGVF, a GGVF force field computation is executed, and the related optimization principle can be applied to complex buildings. Experimental results validate the robustness and effectiveness of the proposed method, whose contour optimization has higher accuracy and comprehensive value compared with that of the reference methods. This method can be used for effective post-processing to strengthen the accuracy of building extraction results.

Enhanced vector flow of significant directions for five-axis machining of STL surfaces

Matching a five-axis toolpath and a vector field (VF) of preferred directions (VFPD) is increasingly popular in the five-axis machining industry. However, surfaces represented by industrial formats often produce irregular non-homogeneous VFPD. The current methods are often unable to match such complicated VFs. We propose a new technique based on an enhanced vector flow (EVF), similar to the gradient flow by Xu and Prince (1998. “Generalized Gradient Vector Flow External Forces for Active Contours.” Signal Processing 71 (2): 131–139) for image processing. The EVF keeps the high-rank vectors unchanged while extending them to unimportant regions, using a diffusion process based on a system of parabolic equations. The resulting enhanced vector field of statistically significant directions (EVFSD) is close to the original VFPD but is characterised by better continuity and regularity. Our second contribution is the clustering of the EVFSD using a prescribed library of templates by means of complex moments. The library includes three basic patterns: ‘parallel’-zigzag, ‘circular’-contour, and ‘star’-radial patterns. The toolpaths are generated by an extension of transfinite interpolation (TFI). Virtual and real machining shows the advantages of EVFSD with reference to the standard iso-parametric paths and several state-of-the-art VF-based methods. The experiments have been performed on a five-axis machining centre, Haas VF-2TR. A video, illustrating the proposed procedure is at https://drive.google.com/open?id=1qCrLZSSNKOpURs0JmbGpqV2oeJEUhBmT.

An efficient modification of generalized gradient vector flow using directional contrast for salient object detection and intelligent scene analysis

In the field of computer vision, scene analysis is a very important area of study. To analyze the scene present in an image, in this paper, the attempt is to enhance salient object information with background information. For this, the Generalized Gradient Vector Flow model is modified by adding contrast information. The contrast information of an image is obtained by computing the Minimum Directional Contrast of the image. Using Minimum Directional Contrast as a determinant of the salient object arises from the fact that the salient objects have higher Minimum Directional Contrast than the non-salient objects. The Minimum Directional Contrast information is added to the data term of Generalized Gradient Vector Flow so that for producing contours, not only edge information is utilized, but saliency information is also used. The result gives us the salient object and added relevant background information. The algorithm is tested on three public datasets. The evaluation is done based on precision, recall, accuracy, and F1-score after comparing with six state-of-the-art methods.

Evaluation of Automatically Quantified Foveal Avascular Zone Metrics for Diagnosis of Diabetic Retinopathy Using Optical Coherence Tomography Angiography.

PurposeTo describe an automated algorithm to quantify the foveal avascular zone (FAZ), using optical coherence tomography angiography (OCTA), and to compare its performance for diagnosis of diabetic retinopathy (DR) and association with best-corrected visual acuity (BCVA) to that of extrafoveal avascular area (EAA).MethodsWe obtained 3 × 3-mm macular OCTA scans in diabetic patients with various levels of DR and healthy controls. An algorithm based on a generalized gradient vector flow (GGVF) snake model detected the FAZ, and metrics assessing FAZ size and irregularity were calculated. We compared the automated FAZ segmentation to manual delineation and tested the within-visit repeatability of FAZ metrics. The correlations of two conventional FAZ metrics, two novel FAZ metrics, and EAA with DR severity and BCVA, as determined by Early Treatment Diabetic Retinopathy Study (ETDRS) charts, were assessed.ResultsSixty-six eyes from 66 diabetic patients and 19 control eyes from 19 healthy participants were included. The agreement between manual and automated FAZ delineation had a Jaccard index > 0.82, and the repeatability of automated FAZ detection was excellent in eyes at all levels of DR severity. FAZ metrics that incorporated both FAZ size and shape irregularity had the strongest correlation with clinical DR grade and BCVA. Of all the tested OCTA metrics, EAA had the greatest sensitivity in differentiating diabetic eyes without clinical evidence of retinopathy, mild to moderate nonproliferative DR (NPDR), and severe NPDR to proliferative DR from healthy controls.ConclusionsThe GGVF snake algorithm tested in this study can accurately and reliably detect the FAZ, using OCTA data at all DR severity grades, and may be used to obtain clinically useful information from OCTA data regarding macular ischemia in patients with diabetes. While FAZ metrics can provide clinically useful information regarding macular ischemia, and possibly visual acuity potential, EAA measurements may be a better biomarker for DR.

Improved Snake Model Using Subsection Functions and Helmholtz Decomposition

The snake model in segmentation of real-world images has some problems because the object in the image has complex textures and weak edges and the snake model has a drawback between noise smoothing and weak edge protection. To solve this problem, we adopt the snake model based on generalized gradient vector flow (GGVF) which has higher segmentation accuracy. We also adopt a new type of coefficient setting in the form of subsection function to improve the ability of protecting weak edges while smoothing noises. Moreover, we optimize the method by using Helmholtz decomposition. Experimental results and comparisons against other methods indicate that our proposed snake model demonstrates higher ability than that of other snake models in terms of real-world image segmentation.

A Gradient Vector Flow Snake Model using Novel Coefficients Setting for Infrared Image Segmentation

In this paper, a novel gradient vector flow snake model is proposed for infrared image segmentation. Our model has several advantages in terms of infrared image segmentation. Infrared images are characterized by high noise and low contrast. We propose a novel generalized gradient vector flow snake model combining the advantages of several snake models. And we also use a new type of coefficient setting to improve the capacity of protecting weak edges in infrared images during segmentation process. Experimental results and comparisons against other methods show that our proposed model performs much better in terms of infrared image segmentation than other snake models.

Segmentasi Citra Sel Tunggal Smear Serviks Menggunakan Radiating Component Normalized Generalized GVFS

Component Normalized Generalized Gradient Vector Flow Snake (CNGGVFS) method is the development of Gradient Vector Flow Snake (GVFS) method as an external force algorithm for active contour (snake) that can be used to get the contour of nucleus and cytoplasm of cervical smear image. However, CNGGVFS using a conventional calculation of edge map such as Sobel can not detect the nucleus area correctly in single cell cervical smear image segmentation. In this study, an external force algorithm in snake that uses Radiating Edge Map (REM) calculation to search the edge map in CNGGVFS, called as Radiating Component Normalized Generalized Gradient Vector Flow Snake (RCNGGVFS), is proposed. RCNGGVFS is used to get the contour of nucleus and cytoplasm of single cervical smear image. There are three main stages in this study, which are: pre-processing, initial segmentation, and contour segmentation. Experiments are conducted on Herlev data-set. The proposed method is compared with other methods in previous research in single cell cervical smear image segmentation. The experiment results show that the proposed method can detect the nucleus area correctly better than Radiating GVFS & Fuzzy C-Means (FCM) and Radiating GVFS & K-means. The average value of accuracy and Zijdenbos similarity index (ZSI) for nucleus segmentation is 95.34% and 88.06%. Then, the average value of accuracy and ZSI for cytoplasm segmentation is 83.48% and 87.16%. The evaluations show the proposed method can be used as a segmentation process of cervical smear image on automatic identification of cervical cancer.

A Novel Gradient Vector Flow Snake Model Based on Convex Function for Infrared Image Segmentation.

Infrared image segmentation is a challenging topic because infrared images are characterized by high noise, low contrast, and weak edges. Active contour models, especially gradient vector flow, have several advantages in terms of infrared image segmentation. However, the GVF (Gradient Vector Flow) model also has some drawbacks including a dilemma between noise smoothing and weak edge protection, which decrease the effect of infrared image segmentation significantly. In order to solve this problem, we propose a novel generalized gradient vector flow snakes model combining GGVF (Generic Gradient Vector Flow) and NBGVF (Normally Biased Gradient Vector Flow) models. We also adopt a new type of coefficients setting in the form of convex function to improve the ability of protecting weak edges while smoothing noises. Experimental results and comparisons against other methods indicate that our proposed snakes model owns better ability in terms of infrared image segmentation than other snakes models.

A novel snake model using new multi-step decision model for complex image segmentation

Active contours, or snakes, have a wide range of applications in object segmentation, which use an energy minimizing spline to extract objects’ borders. Classical snakes have several drawbacks, such as the initial contour sensitivity and convergence ability to local minima. Many approaches based on active contours are put forward to addressing these problems. However, these approaches have limitation that they all depend too much on the amplitude of edge gradient and abandon directional information. This can lead to poor convergence toward the object boundary in the presence of strong background edges and cluttered noises. To deal with these issues, we first propose a novel external force, called adaptive edge preserving generalized gradient vector flow based on component-based normalization (CN-AEGGVF), which can adaptively adjust the process of diffusion according to the local characteristics of an image and preserve weak edges by adding the gradient information of an image. The experimental results show that the new model provides much better results than other approaches in terms of noise robustness, weak edge preserving, and convergence. Secondly, an improved multi-step decision model based on CN-AEGGVF is presented, which added new effective weighting function to attenuate the magnitudes of unwanted edges and adopted narrow band method to reduce time complexity. The novel method is analyzed visually and qualitatively on nature image dataset. Experimental results and comparisons against other methods show that the proposed method has better segmentation accuracy than other comparative approaches.

A novel generalized gradient vector flow snake model using minimal surface and component-normalized method for medical image segmentation

Abstract Active contours, or snakes, have a wide range of applications in medical image segmentation. Gradient vector flow (GVF) field, generalized GVF field and other external force fields have been proposed to address the problems of traditional snake models, such as low accuracy of segmentation and poor convergence ability in indentations. In order to further solve the two problems, we put forward a novel generalized gradient vector flow snake model using minimal surface and component-normalized method. We adopt minimal surface function instead of Laplace operator to settle the problem of low segmentation accuracy. We also use component-based normalization method instead of conventional vector-based normalization method to improve the ability of snake curve to converge into long and thin indentations. Experimental results and comparisons against other methods indicate that the proposed snake model own the ability to protect weak borders and solve the incorrect segmentation problem effectively. Meantime, our method performs much better than generalized GVF snake model in terms of long and thin indentation.

Tumor boundary detection in ultrasound imagery using multi-scale generalized gradient vector flow.

As a key technology in high-intensity focused ultrasound (HIFU) ablation systems, a precise ultrasound image segmentation method for tumor boundary detection is helpful for ablation of tumors and avoiding tumor recurrence. This study explores a new deformable snake model called multi-scale generalized gradient vector flow (MS-GGVF) to segment ultrasound images in HIFU ablation. The main idea of the technique is dealing with two issues including spurious boundary attenuation and setting the standard deviation of the Gaussian filter. We assign the standard deviation as scales to build the MS-GGVF model and create a signed distance map to use its gradient direction information and magnitude information to refine the multi-scale edge map by attenuating spurious boundaries and highlighting the real boundary. In addition, a fast generalized gradient vector flow computation algorithm based on an augmented Lagrangian method is introduced to calculate the external force vector field to improve the computation efficiency of our model. The experimental segmentations were similar to the ground truths delineated by two medical physicians with high area overlap measure and low mean contour distance. The experimental results demonstrate that the proposed algorithm is robust, reliable, and precise for tumor boundary detection in HIFU ablation systems.

Multi-feature gradient vector flow snakes for adaptive segmentation of the ultrasound images of breast cancer

Segmentation of ultrasound (US) images of breast cancer is one of the most challenging problems of the modern medical image processing. A number of popular codes for US segmentation are based on a generalized gradient vector flow (GGVF) method proposed by Xu and Prince. The GGVF equations include a smoothing term (diffusion) applied to regions of small gradients of the edge map and a stopping term to fix and extend large gradients appearing at the boundary of the object.The paper proposes two new directions. The first component is diffusion as a polynomial function of the intensity of the edge map. The second component is the orientation score of the vector field. The new features are integrated into the GGVF equations in the smoothing and the stopping term.The algorithms, having been tested by a set of ground truth images, show that the proposed techniques lead to a better convergence and better segmentation accuracy with the reference to conventional GGVF snakes. The adaptive multi-feature snake does not require any hand-tuning. However, it is as efficient as the standard GGVF with the parameters selected by the “brutal force approach”. Finally, proposed approach has been tested against recent modifications of GGVF, i.e. the Poisson gradient vector flow, the mixed noise vector flow and the convolution vector flow. The numerical tests employing 195 synthetic and 48 real ultrasound images show a tangible improvement in the accuracy of segmentation.

A multi-step directional generalized gradient vector flow snake for target tumor segmentation in US-guided high-intensity focused ultrasound ablation

Getting precise locations of target tumors can help to ensure ablation of cancerous tissues and avoid unwanted destruction of healthy tissues in high-intensity focused ultrasound (HIFU) treatment system. Because of speckle noise and spurious boundaries in ultrasound images, traditional image segmentation methods are not suitable for achieving the precise locations of target tumors in HIFU ablation. In this paper, a multi-step directional generalized gradient vector flow snake model is introduced for target tumor segmentation. In the first step, the traditional generalized gradient vector flow (GGVF) snake is used to obtain an approximate contour of the tumor. According to the approximate contour, a new distance map is generated. Subsequently, a new directional edge map is created by calculating a scalar product of the gradients of the distance map and the initial image. In this process, the gradient directional information and the magnitude information of the distance map are used to attenuate unwanted edges and highlight the real edges in the new directional edge map. Finally, a refined GGVF field is derived from a diffusion operation of the gradient vectors of the directional edge map. The GGVF field is used to refine the tumor's contour, by directing the approximate contour to edges with the desired gradient directionality. Based on the newly developed snake model, the influences of the spurious boundaries and the speckle noise are significantly reduced in the ultrasound image segmentation. Experimental results indicate that this technique is greatly useful for target tumor segmentation in HIFU treatment system

Generalized Gradient Vector Flow for Snakes: New Observations, Analysis, and Improvement

Snakes, or active contours, have been widely used in image processing applications. An external force for snakes called gradient vector flow (GVF) attempts to address traditional snake problems of initialization sensitivity and poor convergence to concavities, while generalized GVF (GGVF) aims to improve GVF snake convergence to long and thin indentations (LTIs). In this paper, we find and show that both GVF and GGVF snakes essentially yield the same performance in capturing LTIs of odd widths, and generally neither can converge to even-width LTIs. Based on a thorough investigation of the GVF and GGVF fields within the LTI during their iterative processes, we identify the crux of the convergence problem, and accordingly propose a novel external force termed as component-normalized GGVF (CN-GGVF) to eliminate the problem. CN-GGVF is obtained by normalizing each component of initial GGVF vectors with respect to its own magnitude. Experimental results and comparisons against GGVF snakes show that the proposed CN-GGVF snakes can capture LTIs regardless of odd or even widths with a remarkably faster convergence speed, while preserving other desirable properties of GGVF snakes with lower computational complexity in vector normalization.

Parcellation of the Thalamus Using Diffusion Tensor Images and a Multi-object Geometric Deformable Model.

The thalamus is a sub-cortical gray matter structure that relays signals between the cerebral cortex and midbrain. It can be parcellated into the thalamic nuclei which project to different cortical regions. The ability to automatically parcellate the thalamic nuclei could lead to enhanced diagnosis or prognosis in patients with some brain disease. Previous works have used diffusion tensor images (DTI) to parcellate the thalamus, using either tensor similarity or cortical connectivity as information driving the parcellation. In this paper, we propose a method that uses the diffusion tensors in a different way than previous works to guide a multiple object geometric deformable model (MGDM) for parcellation. The primary eigenvector (PEV) is used to indicate the homogeneity of fiber orientations. To remove the ambiguity due to the fact that the PEV is an orientation, we map the PEV into a 5D space known as the Knutsson space. An edge map is then generated from the 5D vector to show divisions between regions of aligned PEV's. The generalized gradient vector flow (GGVF) calculated from the edge map drives the evolution of the boundary of each nucleus. Region based force, balloon force, and curvature force are also employed to refine the boundaries. Experiments have been carried out on five real subjects. Quantitative measures show that the automated parcellation agrees with the manual delineation of an expert under a published protocol.

Fast gradient vector flow computation based on augmented Lagrangian method

Gradient vector flow (GVF) and generalized GVF (GGVF) have been widely applied in many image processing applications. The high cost of GVF/GGVF computation, however, has restricted their potential applications on images with large size. Motivated by progress in fast image restoration algorithms, we reformulate the GVF/GGVF computation problem using the convex optimization model with equality constraint, and solve it using the inexact augmented Lagrangian method (IALM). With fast Fourier transform (FFT), we provide two novel simple and efficient algorithms for GVF/GGVF computation, respectively. To further improve the computational efficiency, the multiresolution approach is adopted to perform the GVF/GGVF computation in a coarse-to-fine manner. Experimental results show that the proposed methods can improve the computational speed of the original GVF/GGVF by one or two order of magnitude, and are more efficient than the state-of-the-art methods for GVF/GGVF computation.

Continuous force field analysis for generalized gradient vector flow field

We propose a modification of the generalized gradient vector flow field techniques based on a continuous force field analysis. At every iteration the generalized gradient vector flow method obtains a new, improved vector field. However, the numerical procedure always employs the original image to calculate the gradients used in the source term. The basic idea developed in this paper is to use the resulting vector field to obtain an improved edge map and use it to calculate a new gradient based source term. The improved edge map is evaluated by new continuous force field analysis techniques inspired by a preceding discrete version. The approach leads to a better convergence and better segmentation accuracy as compared to several conventional gradient vector flow type methods.