Segmentation Matching Research Articles

Robust line segment mismatch removal using point-pair representation and Gaussian-uniform mixture formulation

Line segment matching (LSM) plays an important role in image matching, while it is always a challenging problem due to line fracture and high geometric complexity. In this paper, we propose a line segment mismatch removal to address the sensitivity to segment length and the fracture problem. The mismatch removal removes the massive false matches obtained by the descriptors like LBD. Specifically, we suggest calculating the endpoint error in LSM rather than the direction error in considering the error band and segment length. We propose a point-pair representation, and transform the LSM problem to be an intuitive point-pair alignment problem with a mixture model. The point-pair representation is more robust on the segment length than the linear equation representation used in RANSAC, which significantly preserves short line segments. Then, based on the point-pair representation, we propose a novel Directed Endpoint Drift method to solve the inherent fracture problem of line segments by allowing the endpoints to move along the line to complement the fracture. Compared with the recent learning-based method, the proposed method nearly doubles the number of correct matches on the remote sensing dataset. Especially, the recall is improved by 10% to 30% for both the short segments and the slightly fractured segments, compared with the popular RANSAC and MAGSAC++ methods. The code is available at https://github.com/shenliang16/DEpD.

SoccerNet: A Gated Recurrent Unit-based model to predict soccer match winners.

Winning football matches is the major goal of all football clubs in the world. Football being the most popular game in the world, many studies have been conducted to analyze and predict match winners based on players' physical and technical performance. In this study, we analyzed the matches from the professional football league of Qatar Stars League (QSL) covering the matches held in the last ten seasons. We incorporated the highest number of professional matches from the last ten seasons covering from 2011 up to 2022 and proposed SoccerNet, a Gated Recurrent Unit (GRU)-based deep learning-based model to predict match winners with over 80% accuracy. We considered match- and player-related information captured by STATS platform in a time slot of 15 minutes. Then we analyzed players' performance at different positions on the field at different stages of the match. Our results indicated that in QSL, the defenders' role in matches is more dominant than midfielders and forwarders. Moreover, our analysis suggests that the last 15-30 minutes of match segments of the matches from QSL have a more significant impact on the match result than other match segments. To the best of our knowledge, the proposed model is the first DL-based model in predicting match winners from any professional football leagues in the Middle East and North Africa (MENA) region. We believe the results will support the coaching staff and team management for QSL in designing game strategies and improve the overall quality of performance of the players.

Automatic Calculation of Graphic Area Change Rate for Icing Overhead Power Line Insulators Based on Bounding Box Automatic Matching and GrabCut Contour Automatic Segmentation

Image monitoring of icing insulators is valuable for ice-disaster risk management of overhead power lines. This paper proposes an automatic calculation method of graphic area change rate for icing insulators to address laborious manual viewing, computational complexity, poor generalization, weak interpretability, and label data demand of existing approaches. The method automatically identifies insulators in monitoring images by the trained YOLOv5 model, matches the ice-free and test images taken by the same terminal at the same angle utilizing intersection over union of identified bounding boxes, and segments them using an improved GrabCut algorithm. In the segmentation maps, insulator string contours before and after icing are automatically detected to compute the graphic area change rate. The graphic area change rate of insulators in 815 images monitored by 22 terminals from December 2017 to February 2018 is automatically calculated, which is separately data-fused with the tension monitoring value and equivalent ice thickness collected closest to the image acquisition time to obtain the 22 terminals' time series correlations. The correlations approximately follow normal distributions with mathematical expectations of 0.70 and 0.68 and reach 0.95 and 0.94 for high-quality image samples, respectively, verifying the effectiveness. The method enables real-time evaluation of insulators' icing degrees.

Factors Affecting Heart Rate Variability

The heart rate is individual for each person and it is influenced by various factors that lead to its increase or decrease. Good cardiac function is a prerequisite for a healthy life, and heart rate variability (HRV) analysis is a powerful tool for assessing the autonomic nervous system (ANS), in which the sympathetic and parasympathetic systems interact to regulate cardiac function of the vascular system. A high HRV is associated with a good state of health, while a low HRV is associated not only with pathological conditions in the activity of the cardiovascular system, but also with a number of other factors, such as: overweight, type 2 diabetes, stress and others. Tracking HRV over time and matching segments of data related to specific activities or life events can provide unique information about a person's physical and psychological health. On the basis of the obtained results, it can be concluded that the indices of HRV can be used as non-specific indicators of the impact of factors of a different nature on the human body.

Three-dimensional femur morphology analysis for the optimal location of subtrochanteric osteotomy with an implanted Wagner cone stem in total hip arthroplasty for Crowe type IV developmental dysplasia of the hip

BackgroundThis study aimed to accurately evaluate the matching of proximal and distal femoral segments and fitting of the femur–femoral stem in patients with Crowe type IV developmental dysplasia of the hip (DDH) who have undergone subtrochanteric osteotomy at different locations with an implanted Wagner cone stem to improve the rate of the bone union at the osteotomy site.MethodsThree-dimensional femur morphology of 40 patients with Crowe type IV DDH was evaluated at each cross-section to determine the femoral cortical bone area. This study focused on five osteotomy lengths (2.5, 3, 3.5, 4, and 4.5 cm). The overlapped area between the proximal and distal cortical bone segments was defined as the contact area (S, mm2), and the contact area to distal cortical bone area ratio was defined as the coincidence rate (R). Three indicators were used to evaluate the matching and fitting of the osteotomy sites with the implanted Wagner cone stems: (1) higher S and R between the proximal and distal segments; (2) the effective fixation length of the femoral stem at the distal segments being at least 1.5 cm; and (3) osteotomy did not involve the isthmus.ResultsIn all groups, S significantly decreased in the two proximal levels above the 0.5 cm level below the lesser trochanter (LT) compared with those below this level. In comparison, at osteotomy lengths from 2.5 to 4 cm, R significantly decreased in the three proximal levels. The optimal osteotomy levels ranged from 1.5 and 2.5 cm below the LT for an appropriately sized stem.ConclusionsSubtrochanteric osteotomy at the optimal level not only ensures fitting of the femur–femoral stem but also meets the requirements of a higher S and R to ensure adequate reduction and stabilization at the osteotomy site, which may contribute to the bone union. Although the optimal osteotomy level varies with the size of the femoral stem and the length of the subtrochanteric osteotomy, the optimal osteotomy levels for an appropriately sized Wagner cone femoral stem implantation range from 1.5 to 2.5 cm below the LT.

Object measurement in real underwater environments using improved stereo matching with semantic segmentation

Accurately measuring distances in underwater environments is critical for applications such as underwater exploration and maintenance of underwater structures, but is limited by underwater environmental factors. To address this challenge, semantic segmentation and visual stereo matching are promising methods to achieve this goal. In this paper, we proposed a combination of Retinex and Curvature filter for image enhancement, and a modified SGBM (Semi-Global Block Matching) method using dilated SAD(sum of absolute differences) window and the dynamic disparity range obtained using semantic segmentation results to accelerate stereo matching, while also providing insight into the content of the sensed scene. By using the method we proposed, the measurement error is significantly reduced, even in extreme environments with low visibility and complex backgrounds. It showed strong adaptability when dealing with irregular shape objects, and sped up the calculation speed of the algorithm by reducing the calculation area. In the comparison experiment, we found that the error of SGBM with our image enhancement was reduced to 22.8%. By using our stereo matching algorithm, the error was further reduced to 6.2%, demonstrating its superiority for underwater measurement tasks. Our approach offered an efficient solution for autonomous distance measurement in challenging underwater environments.

Visible Joint Classification and Temporal Segment Matching based 3D Pose Refinement for Volleyball Receive Analysis

Receive action pose analysis is very meaningful in volleyball games for training and strategy. Because receive action is lack high-quality labeled data sets and with problems like occlusion, body overlap, and abnormal pose. Conventional work fails to obtain high accurate pose results. This paper proposes visible joint refinement and receive action template matching for volleyball receive action analysis. Firstly, the visible joint is using pixel color feature and the potential constraint features between space and joints to classify the visible joint and refine the error visible joint. Secondly, the template is to realize the pose segment refinement at 3D level by matching. It is based on a multi-view system for a real volleyball competition scene. The dataset video is from the Game of 2014 Japan Inter High School of Men Volleyball. The experiment result achieves 95.33 %, 96.92 %, and 98.43 % success rate at the 30 mm, 50 mm, and 70 mm error ranges.

Automatic Counting Method of Fry Based on Computer Vision

Nowadays, fish farming plays an essential role in aquaculture. In the process of fry sales, the traditional manual counting method takes a great deal of time. Because the fish are constantly moving in the water, they are difficult to count and are likely to be unnecessarily damaged. With the rise of computer vision, fry counting based on image processing is also developing. It has the advantages of having high accuracy and fast processing speed. The primary counting process is divided into detection, adhesion segmentation, and fry counting. First, we improved the maximum interclass variance threshold selection method, and we extracted the histogram features under different channels to detect the fry; then, a matching segmentation method based on concave and skeleton crossing points was used to segment the adherent fry to improve the counting accuracy, the concaves were detected by CNN. We tested the method on real fry data sets, and the average accuracy reached 97%. Our approach had better accuracy than any existing current fish counting methods. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Online map-matching assisted by object-based classification of driving scenario

Different types of roads in complex road networks may run side-by-side or across in 2D or 3D spaces, which causes mismatched segments using existing online map-matching algorithms. A driving scenario that represents the driving environment can inform map-matching algorithms. Images from vehicle cameras contain extensive information about driving scenarios, such as surrounding key objects. This research utilized vehicle images and developed an object-based method to classify driving scenarios (Object-Based Driving-Scenario Classification: OBDSC) to calculate the probabilities of the current image in predefined types of driving scenarios. We implemented an online map-matching algorithm with the OBDSC method (OMM-OBDSC) to obtain optimal matching segments. The algorithm was tested on nine trajectories and OpenStreetMap data in Shanghai and compared with five benchmark algorithms in terms of the match rate, recall and accuracy. The OBDSC method is also applied to the benchmark algorithms to verify the effectiveness of map matching. The results show that our algorithm outperforms the benchmark algorithms with both the original interval and downsampled intervals (96.6%, 96.5%, 93.7% on average with 1–20 s intervals for the three metrics, respectively). The average match rate has improved by 8.9% for all benchmark algorithms after the addition of the OBDSC method.

A New Instrument Monitoring Method Based on Few-Shot Learning

As an important part of the industrialization process, fully automated instrument monitoring and identification are experiencing an increasingly wide range of applications in industrial production, autonomous driving, and medical experimentation. However, digital instruments usually have multi-digit features, meaning that the numeric information on the screen is usually a multi-digit number greater than 10. Therefore, the accuracy of recognition with traditional algorithms such as threshold segmentation and template matching is low, and thus instrument monitoring still relies heavily on human labor at present. However, manual monitoring is costly and not suitable for risky experimental environments such as those involving radiation and contamination. The development of deep neural networks has opened up new possibilities for fully automated instrument monitoring; however, neural networks generally require large training datasets, costly data collection, and annotation. To solve the above problems, this paper proposes a new instrument monitoring method based on few-shot learning (FLIMM). FLIMM improves the average accuracy (ACC) of the model to 99% with only 16 original images via effective data augmentation method. Meanwhile, due to the controllability of simulated image generation, FLIMM can automatically generate annotation information for simulated numbers, which greatly reduces the cost of data collection and annotation.

Ubiquitous conservative interaction patterns between post-spliced introns and their mRNAs revealed by genome-wide interspecies comparison.

Introns, as important vectors of biological functions, can influence many stages of mRNA metabolism. However, in recent research, post-spliced introns are rarely considered. In this study, the optimal matched regions between introns and their mRNAs in nine model organism genomes were investigated with improved Smith-Waterman local alignment software. Our results showed that the distributions of mRNA optimal matched frequencies were highly consistent or universal. There are optimal matched frequency peaks in the UTR regions, which are obvious, especially in the 3'-UTR. The matched frequencies are relatively low in the CDS regions of the mRNA. The distributions of the optimal matched frequencies around the functional sites are also remarkably changed. The centers of the GC content distributions for different sequences are different. The matched rate distributions are highly consistent and are located mainly between 60% and 80%. The most probable value of the optimal matched segments is about 20bp for lower eukaryotes and 30bp for higher eukaryotes. These results show that there are abundant functional units in the introns, and these functional units are correlated structurally with all kinds of sequences of mRNA. The interaction between the post-spliced introns and their corresponding mRNAs may play a key role in gene expression.

PLDS-SLAM: Point and Line Features SLAM in Dynamic Environment

Visual simultaneous localization and mapping (SLAM), based on point features, achieves high localization accuracy and map construction. They primarily perform simultaneous localization and mapping based on static features. Despite their efficiency and high precision, they are prone to instability and even failure in complex environments. In a dynamic environment, it is easy to keep track of failures and even failures in work. The dynamic object elimination method, based on semantic segmentation, often recognizes dynamic objects and static objects without distinction. If there are many semantic segmentation objects or the distribution of segmentation objects is uneven in the camera view, this may result in feature offset and deficiency for map matching and motion tracking, which will lead to problems, such as reduced system accuracy, tracking failure, and track loss. To address these issues, we propose a novel point-line SLAM system based on dynamic environments. The method we propose obtains the prior dynamic region features by detecting and segmenting the dynamic region. It realizes the separation of dynamic and static objects by proposing a geometric constraint method for matching line segments, combined with the epipolar constraint method of feature points. Additionally, a dynamic feature tracking method based on Bayesian theory is proposed to eliminate the dynamic noise of points and lines and improve the robustness and accuracy of the SLAM system. We have performed extensive experiments on the KITTI and HPatches datasets to verify these claims. The experimental results show that our proposed method has excellent performance in dynamic and complex scenes.

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds.

Agricultural robotics is an up and coming field which deals with the development of robotic systems able to tackle a multitude of agricultural tasks efficiently. The case of interest, in this work, is mushroom collection in industrial mushroom farms. Developing such a robot, able to select and out-root a mushroom, requires delicate actions that can only be conducted if a well-performing perception module exists. Specifically, one should accurately detect the 3D pose of a mushroom in order to facilitate the smooth operation of the robotic system. In this work, we develop a vision module for 3D pose estimation of mushrooms from multi-view point clouds using multiple RealSense active-stereo cameras. The main challenge is the lack of annotation data, since 3D annotation is practically infeasible on a large scale. To address this, we developed a novel pipeline for mushroom instance segmentation and template matching, where a 3D model of a mushroom is the only data available. We evaluated, quantitatively, our approach over a synthetic dataset of mushroom scenes, and we, further, validated, qualitatively, the effectiveness of our method over a set of real data, collected by different vision settings.

High Precision Mesh-Based Drone Image Stitching Based on Salient Structure Preservation and Regular Boundaries

Addressing problems such as obvious ghost, dislocation, and distortion resulting from the traditional stitching method, a novel drone image-stitching method is proposed using mesh-based local double-feature bundle adjustment and salient structure preservation which aims to obtain more natural panoramas.The proposed method is divided into the following steps. First, reducing parallax error is considered from both global and local aspects. Global bundle adjustment is introduced to minimize global transfer error, and then the local mesh-based feature-alignment model is incorporated into the optimization framework to achieve more accurate alignment. Considering the sensitivity of human eyes to linear structure, the global linear structure that runs through the images obtained by segment fusion is introduced to prevent distortions and align matching line segments better. Rectangular panoramas usually have better visual effects. Therefore, regular boundary constraint combined with mesh-based shape-preserving transform can make the results more natural while preserving mesh geometry. Two new evaluation metrics are also developed to quantify the performance of linear structure preservation and the alignment difference of matching line segments. Extensive experiments show that our proposed method can eliminate parallax and preserve global linear structures better than other state-of-the-art stitching methods and obtain more natural-looking stitching results.

Sign Language Recognition and Response via Virtual Reality

The main aim of this project is to provide a voice to voiceless people with the help of Python and Unity prototype systems. Aforesaid project work focuses on the problem of gesture recognition in real time that sign language is used by the community of deaf people. The problem addressed is based on Digital image processing using color segmentation, Skin detection, Image Segmentation, Image Filtering, and Template Matching techniques. This system recognizes gestures of ISL(Indian Sign Language) including the alphabet and a subset of its words.

Modelify: An approach to incrementally build 3D object models for map completion

The capabilities of discovering new knowledge and updating the previously acquired one are crucial for deploying autonomous robots in unknown and changing environments. Spatial and objectness concepts are at the basis of several robotic functionalities and are part of the intuitive understanding of the physical world for us humans. In this paper, we propose a method, which we call Modelify, to incrementally map the environment at the level of objects in a consistent manner. We follow an approach where no prior knowledge of the environment is required. The only assumption we make is that objects in the environment are separated by concave boundaries. The approach works on an RGB-D camera stream, where object-like segments are extracted and stored in an incremental database. Segment description and matching are performed by exploiting 2D and 3D information, allowing to build a graph of all segments. Finally, a matching score guides a Markov clustering algorithm to merge segments, thus completing object representations. Our approach allows creating single (merged) instances of repeating objects, objects that were observed from different viewpoints, and objects that were observed in previous mapping sessions. Thanks to our matching and merging strategies this also works with only partially overlapping segments. We perform evaluations on indoor and outdoor datasets recorded with different RGB-D sensors and show the benefit of using a clustering method to form merge candidates and keypoints detected in both 2D and 3D. Our new method shows better results than previous approaches while being significantly faster. A newly recorded dataset and the source code are released with this publication.

A novel approach to analyze the association characteristics between post-spliced introns and their corresponding mRNA.

Studies have shown that post-spliced introns promote cell survival when nutrients are scarce, and intron loss/gain can influence many stages of mRNA metabolism. However, few approaches are currently available to study the correlation between intron sequences and their corresponding mature mRNA sequences. Here, based on the results of the improved Smith-Waterman local alignment-based algorithm method (SW method) and binding free energy weighted local alignment algorithm method (BFE method), the optimal matched segments between introns and their corresponding mature mRNAs in Caenorhabditis elegans (C.elegans) and their relative matching frequency (RF) distributions were obtained. The results showed that although the distributions of relative matching frequencies on mRNAs obtained by the BFE method were similar to the SW method, the interaction intensity in 5'and 3'untranslated regions (UTRs) regions was weaker than the SW method. The RF distributions in the exon-exon junction regions were comparable, the effects of long and short introns on mRNA and on the five functional sites with BFE method were similar to the SW method. However, the interaction intensity in 5'and 3'UTR regions with BFE method was weaker than with SW method. Although the matching rate and length distribution shape of the optimal matched fragment were consistent with the SW method, an increase in length was observed. The matching rates and the length of the optimal matched fragments were mainly in the range of 60%-80% and 20-30bp, respectively. Although we found that there were still matching preferences in the 5'and 3'UTR regions of the mRNAs with BFE, the matching intensities were significantly lower than the matching intensities between introns and their corresponding mRNAs with SW method. Overall, our findings suggest that the interaction between introns and mRNAs results from synergism among different types of sequences during the evolutionary process.

Actuation and design innovations in earthworm-inspired soft robots: A review.

Currently, soft robotics technologies are creating the means of robotic abilities and are required for the development of biomimetic robotics. In recent years, earthworm-inspired soft robot has garnered increasing attention as a major branch of bionic robots. The major studies on earthworm-inspired soft robots focuses on the deformation of the earthworm body segment. Consequently, various actuation methods have been proposed to conduct the expansion and contraction of the robot's segments for locomotion simulation. This review article aims to act as a reference guide for researchers interested in the field of earthworm-inspired soft robot, and to present the current state of research, summarize current design innovations, compare the advantages and disadvantages of different actuation methods with the purpose of inspiring future innovative orientations for researchers. Herein, earthworm-inspired soft robots are classified into single- and multi-segment types, and the characteristics of various actuation methods are introduced and compared according to the number of matching segments. Moreover, various promising application instances of the different actuation methods are detailed along with their main features. Finally, motion performances of the robots are compared by two normalized metrics-speed compared by body length and speed compared by body diameter, and future developments in this research direction are presented.

A Prosumer Power Prediction Method Based on Dynamic Segmented Curve Matching and Trend Feature Perception

With the massive installation of distributed renewable energy (DRE) generation, many prosumers with the dual attributes of load and power supply have emerged. Different DRE permeability and the corresponding peak-valley timing characteristics have an impact on the power features of prosumers, so new models and methods are needed to reflect the new features brought about by these factors. This paper proposes a method for predicting the power of prosumers. In this method, dynamic segmented curve matching is applied to reduce the complexity of source–load coupling features and improve the effectiveness of the input features, and trend feature perception based on a temporal convolutional network (TCN) was applied to grasp the power trend of prosumers by predicting the multisegment trend indexes. The LST-Atten prediction model based on a temporal attention mechanism (TAM) and a long short-term memory (LSTM) network was applied to predict “day-ahead” power, which combines the trend indexes and similar curve sets as the input. Simulation results show that the proposed model has higher accuracy than individual models. Furthermore, the proposed model can maintain prediction stability under different renewable energy permeability scenarios.

An adaptive feature region-based line segment matching method for viewpoint-changed images with discontinuous parallax and poor textures

Many methods have been proposed to extract line segment (LS) correspondences for images with viewpoint variations. However, the matching performance on images with viewpoint variations is still limited due to the influence of discontinuous parallax and poor textures on images. Existing line segment matching (LSM) methods rarely consider the coexistence of these two situations. This study attempts to address this problem by proposing an adaptive feature region (FR)-based LSM method based on the observation that different FRs are suitable for different image scenes under image viewpoint change. In the proposed method, LSs are paired, and two types of FRs, symmetric FR centered on the line-pair intersection and asymmetric FR with the intersection as a vertex, are constructed for every line pair. The asymmetric FR of a line pair is a subregion of its symmetric FR. And the complement region of asymmetric FR in symmetric FR is called complement FR. To choose a suitable FR for the matching of each line pair adaptively, the asymmetric FR-based feature descriptor similarity and epipolar line-based constraints are combined to determine candidate line pair matches firstly. Then, the candidate matches with similar complement FR-based feature descriptors are considered initial matches and used to construct a topological constraint to check other candidates. Finally, we estimate an adaptive influence region-based local homography for each matched line pair to constrain the matching of unmatched individual LSs. The experimental results on the test images with viewpoint changes show that our approach outperforms the state-of-the-art methods in terms of Recall, matching precision (MP), and F-Measure (reflecting the overall matching performance of Recall and MP). Especially, it improves the average Recall and average F-Measure of the best method among the comparison methods by 45.20% and 25.83%, respectively.