Candidate Segments Research Articles

A Novel Deep Learning Framework for Pulmonary Embolism Detection for Covid-19 Management

Pulmonary Embolism is a blood clot in the lung which restricts the blood flow and reduces blood oxygen level resulting in mortality if it is untreated. Further, pulmonary embolism is evidenced prominently in the segmental and sub-segmental regions of the computed tomography angiography images in COVID-19 patients. Pulmonary embolism detection from these images is a significant research problem in the challenging COVID-19 pandemic in the venture of early disease detection, treatment, and prognosis. Inspired by several investigations based on deep learning in this context, a two-stage framework has been proposed for pulmonary embolism detection which is realized as a segmentation model. It is implemented as a cascade of convolutional superpixel neural network and a regularized UNet network for the segmentation of embolism candidates as well as embolisms, respectively. The proposed model has been tested with two public datasets and it has achieved a testing accuracy of 99%. The proposed model demonstrates high sensitivities of 88.43%, 88.36%, and 89.93% at 0, 2, and 5 mm localization errors, respectively for two false positives and they are superior to the state-of-the-art models, signifying potential applications in the treatment protocols of diverse pulmonary diseases and COVID-19.

Temporal Video Segmentation Using Optical Flow Estimation

Shot boundary detection is the process of segmenting a video into basic units known as shots by discovering transition frames between shots. Researches have been conducted to accurately detect the shot boundaries. However, the acceleration of the shot detection process with higher accuracy needs improvement. A new method was introduced in this paper to find out the boundaries of abrupt shots in the video with high accuracy and lower computational cost. The proposed method consists of two stages. First, projection features were used to distinguish non boundary transitions and candidate transitions that may contain abrupt boundary. Only candidate transitions were conserved for next stage. Thus, the speed of shot detection was improved by reducing the detection scope. In the second stage, the candidate segments were refined using motion feature derived from the optical flow to remove non boundary frames. The results manifest that the proposed method achieved excellent detection accuracy (0.98 according to F-Score) and effectively speeded up detection process. In addition, the comparative analysis results confirmed the superior performance of the proposed method versus other methods.

Quantitative analysis of metastatic breast cancer in mice using deep learning on cryo-image data

Cryo-imaging sections and images a whole mouse and provides ~ 120-GBytes of microscopic 3D color anatomy and fluorescence images, making fully manual analysis of metastases an onerous task. A convolutional neural network (CNN)-based metastases segmentation algorithm included three steps: candidate segmentation, candidate classification, and semi-automatic correction of the classification result. The candidate segmentation generated > 5000 candidates in each of the breast cancer-bearing mice. Random forest classifier with multi-scale CNN features and hand-crafted intensity and morphology features achieved 0.8645 ± 0.0858, 0.9738 ± 0.0074, and 0.9709 ± 0.0182 sensitivity, specificity, and area under the curve (AUC) of the receiver operating characteristic (ROC), with fourfold cross validation. Classification results guided manual correction by an expert with our in-house MATLAB software. Finally, 225, 148, 165, and 344 metastases were identified in the four cancer mice. With CNN-based segmentation, the human intervention time was reduced from > 12 to ~ 2 h. We demonstrated that 4T1 breast cancer metastases spread to the lung, liver, bone, and brain. Assessing the size and distribution of metastases proves the usefulness and robustness of cryo-imaging and our software for evaluating new cancer imaging and therapeutics technologies. Application of the method with only minor modification to a pancreatic metastatic cancer model demonstrated generalizability to other tumor models.

Intersection-Based Routing with Fuzzy Multi-Factor Decision for VANETs

Due to the limitations of the urban environment, the data transferred between vehicles can only change direction at the intersections. Therefore, the routing decision at an intersection will largely affect the overall routing decision. In this article, we propose an Intersection-Based Routing with Fuzzy Multi-Factor Decision (IRFMF), which utilizes several factors to decide the next road segment. In the scheme, each intersection introduces three factors including the direction, the number of lanes, and the traffic. After the fuzzification and defuzzification of these factors, the candidate segment with the highest evaluation will be selected. The simulation shows a significant improvement of VANETs performance on packet delivery ratio and end-to-end delay.

One-Pot Synthesis of Supertough, Sustainable Polyester Thermoplastic Elastomers Using Block-Like, Gradient Copolymer as Soft Midblock

It remains challenging to synthesize supertough thermoplastic elastomers (TPEs) since the stretchability and tensile strength are mutually exclusive. Here, we report a one-pot strategy for the prep...

A class-independent flexible algorithm to generate region proposals

Generating a sufficient number of regions with high accuracy is an important objective in the region proposal generation techniques. This paper presents a new, robust, and effective approach, which is based on the bottom-up segmentation, to produce a pool of well-quality regions. After image segmentation, the segmented candidates are expanded into the surrounding regions. The suggested algorithm produces some enlarged regions, which better cover objects and stuff. The proposed process can be applied in three different modes, namely fixed_mode, all_mode, and efficient_mode. The fixed_mode extends each region into parts of all the adjacent regions using an extension controller, which considers adjacent sequential pixels for each point on the region boundary. In all_mode, the current region is merged with all the adjacent regions to generate a larger region. The efficient_mode is then implemented using the accumulation of the results from both the fixed_mode and all_mode. Besides, the algorithm can be repeated in the fixed_mode and all_mode by considering a variety of values for the extension controller factor. No features are required to be extracted in the proposed algorithm, except for the image segmentation stage. In this study, four challenging datasets known as MSRC, VOC2007, VOC2012, and COCO 2017 are used to compare the proposed algorithm with other segmentation and region proposal algorithms. As a significant advantage compared to well-known region proposal algorithms, our approach achieves a greater Recall with the desirable number of regions. Furthermore, the algorithm shows a good improvementin extraction of small, medium, and large objects.

Context-aware network with foreground recalibration for grounding natural language in video

Grounding natural language in video aims at retrieving a matching moment in a long, untrimmed video described by a referring natural language query. It is a challenging issue due to the dominating influence from noise background in untrimmed video and the complex temporal relationships introduced by the query. Existing methods treat different candidate segments separately in a matching and aligning manner and thus neglect that different target segments require different levels of context information. In this paper, we present the semantic modulation residual module, a novel single-shot feed-forward residual network that explicitly integrates various temporal scale features and introduces less noise to the final moments representation with the guide of query semantic information. To establish more fine-grained interactions between different moments, a global interaction module is embedded in the network. Moreover, the data imbalance issue caused by the sparse annotated moments weakens the effect of binary cross-entropy criterion. Therefore, we design a foreground recalibration mechanism to enhance the intra-class consistency and highlight the positive moments. We evaluate our method on three benchmark datasets i.e., TACoS, Charades-STA and ActivityNet Captions, achieving state-of-the-art performance without any post-processing. In particular, we reach 32.17%, 45.11% and 43.76% under the metric Rank@1, IoU@0.5 on TACoS, Charades-STA and ActivityNet Captions, respectively. Furthermore, ablation studies were performed to show the effectiveness of individual components in our proposed method. We hope that the proposed method can serve as a strong and simple alternative for fine-grained video retrieval.

Fast Shot Boundary Detection Based on Separable Moments and Support Vector Machine

The large number of visual applications in multimedia sharing websites and social networks contribute to the increasing amounts of multimedia data in cyberspace. Video data is a rich source of information and considered the most demanding in terms of storage space. With the huge development of digital video production, video management becomes a challenging task. Video content analysis (VCA) aims to provide big data solutions by automating the video management. To this end, shot boundary detection (SBD) is considered an essential step in VCA. It aims to partition the video sequence into shots by detecting shot transitions. High computational cost in transition detection is considered a bottleneck for real-time applications. Thus, in this paper, a balance between detection accuracy and speed for SBD is addressed by presenting a new method for fast video processing. The proposed SBD framework is based on the concept of candidate segment selection with frame active area and separable moments. First, for each frame, the active area is selected such that only the informative content is considered. This leads to a reduction in the computational cost and disturbance factors. Second, for each active area, the moments are computed using orthogonal polynomials. Then, an adaptive threshold and inequality criteria are used to eliminate most of the non-transition frames and preserve candidate segments. For further elimination, two rounds of bisection comparisons are applied. As a result, the computational cost is reduced in the subsequent stages. Finally, machine learning statistics based on the support vector machine is implemented to detect the cut transitions. The enhancement of the proposed fast video processing method over existing methods in terms of computational complexity and accuracy is verified. The average improvements in terms of frame percentage and transition accuracy percentage are 1.63% and 2.05%, respectively. Moreover, for the proposed SBD algorithm, a comparative study is performed with state-of-the-art algorithms. The comparison results confirm the superiority of the proposed algorithm in computation time with improvement of over 38%.

Combining Appearance and Gradient Information for Image Symmetry Detection.

This work addresses the challenging problem of reflection symmetry detection in unconstrained environments. Starting from the understanding on how the visual cortex manages planar symmetry detection, it is proposed to treat the problem in two stages: i) the design of a stable metric that extracts subsets of consistently oriented candidate segments, whenever the underlying 2D signal appearance exhibits definite near symmetric correspondences; ii) the ranking of such segments on the basis of the surrounding gradient orientation specularity, in order to reflect real symmetric object boundaries. Since these operations are related to the way the human brain performs planar symmetry detection, a better correspondence can be established between the outcomes of the proposed algorithm and a human-constructed ground truth. When compared to the testing sets used in recent symmetry detection competitions, a remarkable performance gain can be observed. In additional, further validation has been achieved by conducting perceptual validation experiments with users on a newly built dataset.

Relief Extraction From a Rough Stele Surface Using SVM-Based Relief Segment Selection

Archaeological steles having a rough surface due to long periods of weathering make recognizing the inscription difficult. In this paper, we propose a machine learning-based method to extract reliefs for the inscription from a rough stele surface. Relief candidate segments are initially obtained by using a curvature-based method, which include not only actual reliefs but also noises such as dents and scratches. Then, relief segments are selected using a support vector machine classifier that is trained with various features extracted from relief candidate segments. While conventional methods using a single geometric feature easily fail to detect reliefs from the rough surface, the proposed method utilizes 79-dimensional features consisting of appearance-based, cross section-based, and local extrema-based characteristics of each candidate segment to determine whether the segment is relief or not. Using the proposed method, the inscription of the stele Musul-ojakbi made during the Silla Dynasty AD578 were completely recognized. The experimental results demonstrate that the proposed method accurately extracts reliefs and achieves the highest performance on the rough stele data. The performance of the proposed method is about 8.95% and 10.4% higher than the best of the conventional methods in terms of the F1-score and the SIRI, respectively.

Detection of Microaneurysms Using Grey Wolf Optimization for Early Diagnosis of Diabetic Retinopathy

The diabetic retinopathy is the leading cause of blindness worldwide, so early detection of diabetic retinopathy is necessary to reduce eye-related diseases. The accurate identification of microaneurysms is crucial for the detection of diabetic retinopathy, because it appears as the first sign of the disease. In this study, a new model is proposed to detect microaneurysms from the retinal images for early diagnosis of diabetic retinopathy. At first, the fundus images are collected from e-ophtha microaneurysms and DiaretDB1 datasets. Next, image pre-processing is accomplished using image normalization, low light image enhancement, gradient weighting and shade correction. The pre-processing methods significantly brighten the contrast of the fundus images for better visual quality and extract the hidden details of the dark conditions. In addition, Hessian based filter and Otsu threshold are used to extract the foreground objects like microaneurysms from the enhanced fundus images. At last, Grey Wolf Optimization (GWO) is used to predict the correctness of segmented microaneurysms candidates. The experimental results have revealed that the proposed model enhanced the microaneurysms detection up to 0.06-0.30 f-score value compared to the other existing models local convergence index features and local features with k-nearest neighbor. In addition, the proposed model has achieved 85.72% and 86.16% of accuracy respectively on e-ophtha microaneurysms and DiaretDB1 datasets.

Story-driven Video Editing

This paper proposes a novel multimedia task: story-driven video editing. Given a story paragraph, this task aims to retrieve related video segments from a gallery of collected video segments and compose them into a video sequence by the storyline order. Our proposed baseline solution consists of three modules: a retrieval module, which returns lists of candidate segments for all query sentences in the story paragraph using an object-aware sentence-segment matching method; a sequence candidate proposal module, which aggregates the retrieved segment sets into a sequence proposal by the submodular optimization method; a sorting module, which arranges the candidates according to the storyline of the paragraph using the Sinkhorn network. We build a benchmark for this task including a reorganized version of the ActivityNet Captions dataset, a well-defined quantitative metric called Evaluation of Segment-to-Sequence Matching (ESSM) for measuring the difference between the generated video segment sequence and the ground truth. Quantitative results of the proposed baseline solution are reported. We hope this new task and benchmark will bring broad research attention and push forward a lot of novel online short video editing applications.

ILP formulations for monitoring-cycle design based on segment routing

The minimum cycle cover problem in segment routing is to find a set of monitoring cycles such that (a) they originate from the same vantage point, (b) cover every directed link in the network, (c) each cycle can be encoded by a segment list smaller than a given maximum size, and (d) the total length of all cycles is minimized. To solve the problem using Integer Linear Programming (ILP), the key challenge is how to capture all the subtle features of segment routing using linear constraints. In this paper, five original ILP formulations, numbered from ILP1 to ILP5, are designed. In ILP1, the classic flow-based approach is adopted for constructing cycles. In ILP2, a more efficient voltage-based approach is adopted. We then extend ILP2 to ILP3 to jointly minimize the total segment list size. Since the time required to detect a network failure is determined by the longest cycle, we also extend ILP2 to ILP4 to jointly minimize the length of the longest cycle. As finding optimal solutions for large-sized topologies is time-consuming, we propose to limit the set of candidate segments for constructing cycles. Although the resulting ILP5 cannot guarantee optimal solutions, we show that it always outperforms the best heuristic algorithm. Finally, the performance gain of using a dedicated monitoring topology for probe packets, as adopted by all existing heuristic algorithms, is studied.

Heliport Detection Using Artificial Neural Networks

Automatic image exploitation is a critical technology for quick content analysis of high-resolution remote sensing images. The presence of a heliport on an image usually implies an important facility, such as military facilities. Therefore, detection of heliports can reveal critical information about the content of an image. In this article, two learning-based algorithms are presented that make use of artificial neural networks to detect H-shaped, light-colored heliports. The first algorithm is based on shape analysis of the heliport candidate segments using classical artificial neural networks. The second algorithm uses deep-learning techniques. While deep learning can solve difficult problems successfully, classical-learning approaches can be tuned easily to obtain fast and reasonable results. Therefore, although the main objective of this article is heliport detection, it also compares a deep-learning based approach with a classical learning-based approach and discusses advantages and disadvantages of both techniques.

Identification of Retinal Ganglion Cells from β-III Stained Fluorescent Microscopic Images.

Optic nerve crush in mouse model is widely used for investigating the course following retinal ganglion cell (RGCs) injury. Manual cell counting from β-III tubulin stained microscopic images has been routinely performed to monitor RGCs after an optic nerve crush injury, but is time-consuming and prone to observer variability. This paper describes an automatic technique for RGC identification. We developed and validated (i) a sensitive cell candidate segmentation scheme and (ii) a classifier that removed false positives while retaining true positives. Two major contributions were made in cell candidate segmentation. First, a homomorphic filter was designed to adjust for the inhomogeneous illumination caused by uneven penetration of β-III tubulin antibody. Second, the optimal segmentation parameters for cell detection are highly image-specific. To address this issue, we introduced an offline-online parameter tuning approach. Offline tuning optimized model parameters based on training images and online tuning further optimized the parameters at the testing stage without needing access to the ground truth. In the cell identification stage, 31 geometric, statistical and textural features were extracted from each segmented cell candidate, which was subsequently classified as true or false positives by support vector machine. The homomorphic filter and the online parameter tuning approach together increased cell recall by 28%. The entire pipeline attained a recall, precision and coefficient of determination (r2) of 85.3%, 97.1% and 0.994. The availability of the proposed pipeline will allow efficient, accurate and reproducible RGC quantification required for assessing the death/survival of RGCs in disease models.

Shadow detection in SAR images based on greyscale distribution, a saliency model, and geometrical matching

ABSTRACT Shadow information has been widely used in synthetic aperture radar interpretation, but corresponding shadow detection technology has not been given much attention in past studies. In this paper, we propose a hierarchical architecture based on greyscale distribution, a saliency model, and geometrical matching for shadow detection in SAR images. We find that the greyscale distribution of one Gaussian filtered image might contain a ‘distortion’ in its uphill part, which is the effect of shadow existence. Based on this distortion, a global threshold can be determined and then be used to segment candidate shadows. If there is no obvious distortion, a shadow saliency model is proposed as a substitute to extract such candidate areas. Usually, these candidate areas may contain some non-shadow components. According to the geometric relationships between shadow and object, we design a matching strategy to eliminate non-shadow parts from candidate regions. The remained areas are final shadow detection results. Experiments on two real datasets, Moving and Stationary Target Acquisition Recognition and MiniSAR, show that our method performs much better than two other published methods. The results demonstrate the effectiveness and feasibility of our proposed algorithm in practical SAR shadow detection tasks.

A Fuzzy Segmentation Method to Learn Classification of Mitosis

Mitotic counts are widely used as a metric for cellular proliferation for prognosis and to determine the aggressiveness of individual cancers. This study presents a less labor-intensive method to count mitotic cells in breast cell sections. The proposed algorithm involves two phases: candidate segmentation and detection. During candidate segmentation, images are filtered through a blue ratio threshold to remove unnecessary background information and to increase the color difference between targets and non-targets for an entire digitized image. A fuzzy candidate segmentation method is used to adaptively determine threshold values in order to dichotomize gray-level images and distinguish the images of mitotic candidates from the background. The thresholding scheme integrates the spatial characteristics’ distribution in a histogram to determine an intensity threshold for the processed image, in order to filter insignificant information. During the detection phase, a two-class classification uses an attention mechanism that is realized by a set of fully connected neural networks, instead of convolutional layers, which decreases the computational cost. The validation test using ICPR2012 competition datasets shows that the proposed model outperforms current state-of-art techniques, in terms of the metrics, Accuracy, F1-score, and Precision and Recall.

Unsupervised Action Proposals Using Support Vector Classifiers for Online Video Processing.

In this work, we introduce an intelligent video sensor for the problem of Action Proposals (AP). AP consists of localizing temporal segments in untrimmed videos that are likely to contain actions. Solving this problem can accelerate several video action understanding tasks, such as detection, retrieval, or indexing. All previous AP approaches are supervised and offline, i.e., they need both the temporal annotations of the datasets during training and access to the whole video to effectively cast the proposals. We propose here a new approach which, unlike the rest of the state-of-the-art models, is unsupervised. This implies that we do not allow it to see any labeled data during learning nor to work with any pre-trained feature on the used dataset. Moreover, our approach also operates in an online manner, which can be beneficial for many real-world applications where the video has to be processed as soon as it arrives at the sensor, e.g., robotics or video monitoring. The core of our method is based on a Support Vector Classifier (SVC) module which produces candidate segments for AP by distinguishing between sets of contiguous video frames. We further propose a mechanism to refine and filter those candidate segments. This filter optimizes a learning-to-rank formulation over the dynamics of the segments. An extensive experimental evaluation is conducted on Thumos’14 and ActivityNet datasets, and, to the best of our knowledge, this work supposes the first unsupervised approach on these main AP benchmarks. Finally, we also provide a thorough comparison to the current state-of-the-art supervised AP approaches. We achieve 41% and 59% of the performance of the best-supervised model on ActivityNet and Thumos’14, respectively, confirming our unsupervised solution as a correct option to tackle the AP problem. The code to reproduce all our results will be publicly released upon acceptance of the paper.

BraTS Toolkit: Translating BraTS Brain Tumor Segmentation Algorithms Into Clinical and Scientific Practice.

Despite great advances in brain tumor segmentation and clear clinical need, translation of state-of-the-art computational methods into clinical routine and scientific practice remains a major challenge. Several factors impede successful implementations, including data standardization and preprocessing. However, these steps are pivotal for the deployment of state-of-the-art image segmentation algorithms. To overcome these issues, we present BraTS Toolkit. BraTS Toolkit is a holistic approach to brain tumor segmentation and consists of three components: First, the BraTS Preprocessor facilitates data standardization and preprocessing for researchers and clinicians alike. It covers the entire image analysis workflow prior to tumor segmentation, from image conversion and registration to brain extraction. Second, BraTS Segmentor enables orchestration of BraTS brain tumor segmentation algorithms for generation of fully-automated segmentations. Finally, Brats Fusionator can combine the resulting candidate segmentations into consensus segmentations using fusion methods such as majority voting and iterative SIMPLE fusion. The capabilities of our tools are illustrated with a practical example to enable easy translation to clinical and scientific practice.

ETIP: a lengthy nested NER problem for Chinese insurance policy analysis

Contract analysis can significantly ease the work for humans using AI techniques. This paper shows a lengthy nested NER problem of element tagging on insurance policy (ETIP). Compared to NER, ETIP deals with not only different types of entities which vary from a short phrase to a long sentence, but also phrase or clause entities that could be nested. We present a novel hybrid framework of deep learning and heuristic filtering method to recognize the lengthy nested elements. First, a convolutional neural network is constructed to obtain good initial candidates of sliding windows with high softmax probability. Then, the concatenation operator on adjacent candidate segments is introduced to create phrase, clause, or sentence candidates. We design an effective voting strategy to resolve the classification conflict of the concatenated candidates and present a theoretical proof of F1-score optimization. In experiments, we have collected a large Chinese insurance contract dataset to test the performance of the proposed method. An extensive set of experiments is performed to investigate how sliding window candidates can work effectively in our filtering and voting strategy. The optimal parameters are determined by statistical analysis of the experimental data. The results show the promising performance of our method in the ETIP problem.