Corner Points Research Articles

Part Affinity Fields and CoordConv for Detecting Landmarks of Lumbar Vertebrae and Sacrum in X-ray Images.

With the prevalence of degenerative diseases due to the increase in the aging population, we have encountered many spine-related disorders. Since the spine is a crucial part of the body, fast and accurate diagnosis is critically important. Generally, clinicians use X-ray images to diagnose the spine, but X-ray images are commonly occluded by the shadows of some bones, making it hard to identify the whole spine. Therefore, recently, various deep-learning-based spinal X-ray image analysis approaches have been proposed to help diagnose the spine. However, these approaches did not consider the characteristics of frequent occlusion in the X-ray image and the properties of the vertebra shape. Therefore, based on the X-ray image properties and vertebra shape, we present a novel landmark detection network specialized in lumbar X-ray images. The proposed network consists of two stages: The first step detects the centers of the lumbar vertebrae and the upper end plate of the first sacral vertebra (S1), and the second step detects the four corner points of each lumbar vertebra and two corner points of S1 from the image obtained in the first step. We used random spine cutout augmentation in the first step to robustify the network against the commonly obscured X-ray images. Furthermore, in the second step, we used CoordConv to make the network recognize the location distribution of landmarks and part affinity fields to understand the morphological features of the vertebrae, resulting in more accurate landmark detection. The proposed network was evaluated using 304 X-ray images, and it achieved 98.02% accuracy in center detection and 8.34% relative distance error in corner detection. This indicates that our network can detect spinal landmarks reliably enough to support radiologists in analyzing the lumbar X-ray images.

A new approach for measuring the wetting angles of lead-free solder alloys from digital images

This study aims to detect quaternary lead-free solder alloys and measure their dynamic wetting angles with a new approach. The detection of the prepared alloy drop and measurement of its wetting angle are automatically performed by the developed application. A video obtained after experimental studies is filtered over its frames and enhanced a better form in the preprocessing stage. The peak of the half alloy drop is detected by obtaining lane histogram and its global minimum point is found in the segmentation stage. Left and right corner points in contact with the ground are detected by tracking pixel intensity values with the help of the global minimum point. The median point for both left half alloy drop and right half alloy drop is calculated. Thus, circles passing through three points for the left half and right half of the alloy drop are obtained to measure θleft,θrightandθmean wetting angles. In the postprocessing stage, these angles are measured at 0th, 5th, 10th, 15th, 30th, 60th, 90th, 120th, 150th, and 300th seconds after the alloy drop contacted the ground. Besides, measurement of these wetting angles is performed at each second. The obtained results showed that the graph of θmean versus time is an exponential decay graph. This result proves that the alloy drop spreads over the ground as time passes and the value of θmean decreases and is consistent with the observation made during the experiment.

Accurate scoliosis vertebral landmark localization on X-ray images via shape-constrained multi-stage cascaded CNNs

Vertebral landmark localization is a crucial step in various spine-related clinical applications, which requires detecting the corner points of 17 vertebrae. However, the neighboring landmarks often disturb each other because of the homogeneous appearance of vertebrae, making vertebral landmark localization extremely difficult. In this paper, we propose a multi-stage cascaded convolutional neural network (CNN) to split a single task into two sequential steps: center point localization to roughly locate 17 center points of vertebrae, and corner point localization to determine four corner points for each vertebra without any disturbance. The landmarks in each step were located gradually from a set of initialized points by regressing offsets using cascaded CNNs. To resist the mutual attraction of the vertebrae, principal component analysis (PCA) was employed to preserve the shape constraint in offset regression. We evaluated our method on the AASCE dataset, comprising 609 tight spinal anteroposterior X-ray images, and each image contained 17 vertebrae composed of the thoracic and lumbar spine for spinal shape characterization. The experimental results demonstrated the superior performance of vertebral landmark localization over other state-of-the-art methods, with the relative error decreasing from 3.2e−3 to 7.2e−4.

Saturday, June 18, 20227:40 AM - 5:00 PMPO22 Presentation Time: 7:40 AM: A Novel Algorithm for Automatic Interstitial Needle Tracking on PETRA MRI for HDR Brachytherapy

<h3>Purpose</h3> Current practice for interstitial high-dose-rate (HDR) brachytherapy treatment planning involves manual tracking of needle positions on CT images. Recently, PETRA MRI showed the potential to detect empty interstitial gynecological brachytherapy needles owing to low needle signal, intermediate tissue contrast and high spatial resolution. We aimed to develop an algorithm for automatic interstitial needle tracking on PETRA images by employing their negative contrast against surrounding tissue. <h3>Materials and Methods</h3> Fourteen ProGuide Sharp Needles 6Fx294mm were implanted in an endometrial adenocarcinoma patient using a Syed-Neblett template. One of the needles lay in the obturator center, and six inside the obturator slots. The patient underwent standard CT imaging with copper-filled needles. After marker removal, 3D PETRA images of the patient's pelvis (TR\TE 3.32\0.07 ms, voxel size 0.9 mm³ isotropic, 416 slices) were acquired on a 3T Siemens Vida MR scanner. PETRA MR images were analyzed for interstitial needle detection using custom code in MATLAB R2022a. A rectangular part of all axial slices where needles may be contained (slices 149-310) was selected. For each slice part, corner points were extracted from the original part and from its sharpened gradient magnitude image by detecting Harris features. Points found within Euclidean distance (ED) of 3 pixels on both images were matched, and the pixel with minimum signal intensity closest to each such point on the original part was selected. Minimum intensity points lying within 2 pixels ED between subsequent slices were considered as possible catheter coordinates. Such points appearing on at least 10 continuous slices were designated as true catheters, and their positions were compared to those visible on CT. <h3>Results</h3> Figure 1a) displays the selected MR image parts for two example slices. Pixels with detected Harris features and their closest signal minima are marked by red and green crosses, respectively, in b). Pixels accepted as needle positions are shown as magenta crosses in c). Matching CT image parts with bright marker-filled needles are displayed in d). Empty interstitial catheters on PETRA images corresponded to local minima of signal intensity, located near Harris features. The assumption of needle position continuity within an allowed distance along subsequent slices helped avoid spurious signal minima, while following needle position variations caused by different deflection angles. Needles guided through the obturator grooves and the central needle were detected only after they became detached from the obturator (ex. figure left). All 14 needles were tracked along their visible length inside tissue (ex. figure right). Visual comparison to CT images confirmed interstitial catheter detection on MRI. Selecting a range of slices and zooming on a part of each slice for analysis reduced computational time and output data size. <h3>Conclusions</h3> Automated detection of interstitial needles is feasible on PETRA MR images and may facilitate MR-only treatment planning in interstitial HDR brachytherapy. The newly developed algorithm was able to track all needles inside tissue based on their signal intensity characteristics, without requiring manual seed position placement.

Efficient co-registration of UAV and ground LiDAR forest point clouds based on canopy shapes

• Time-efficient co-registration of UAV and ground LiDAR data is proposed for plantation forests. • Coarse alignment of forest point clouds is divided into ground alignment and canopy image matching. • Canopy images are matched by two pairs of canopy corner points and the canopy overlap rate. • The runtime of coarse alignment is less than 1 s for millions of forest points. Registration of unmanned aerial vehicle laser scanning (ULS) and ground light detection and ranging (LiDAR) point clouds in forests is critical to create a detailed representation of a forest structure and an accurate retrieval of forest parameters. However, tree occlusion poses challenges for those registration methods used artificial markers, and some automated registration methods have low time-efficiency due to the process of object (e.g., tree, crown) segmentation. In this study, we propose an automated and time-efficient method to register ULS and ground LiDAR (including terrestrial and backpack laser scanning) forest point clouds. Registration involves coarse alignment and fine registration, where the coarse alignment is divided into vertical and horizontal alignment. The vertical alignment is implemented by rotating grounds to the horizontal plane, and the horizontal alignment is achieved by canopy image matching. During image matching, vegetation points are projected onto the horizontal plane to obtain two binary images, and then, canopy shape feature, which is described by a two-point congruent set and canopy overlap, is used to match the binary images. Finally, we implement coarse alignment of ULS and ground LiDAR datasets by combining the results of ground alignment and image matching and finish fine registration in six plantation forest plots with sizes of 0.03 ha to 0.25 ha. Experimental results show that the ULS and ground LiDAR data in different plots are registered, of which the coarse alignment errors are less than 0.20 m in the horizontal direction, the final registration accuracy is less than 0.15 m, and the average runtime is less than 1 s. Our study demonstrates the effectiveness of the proposed strategy and has able to perform accurate and quick registration of ULS and ground LiDAR data from plantation forests with different attributes.

A Novel Spatio-Temporal Synchronization Method of Roadside Asynchronous MMW Radar-Camera for Sensor Fusion

Roadside sensors, such as camera and millimeter-wave (MMW) radar, provide traffic information beyond the visual range of intelligent vehicles in cooperative vehicle-infrastructure systems. Unlike onboard equipment, roadside sensors are affiliated with different systems and lack synchronization in both space and time. In this paper, we propose a novel spatio-temporal synchronization method of asynchronous roadside MMW radar–camera for sensor fusion, which utilizes features of the scenario to extract lane line corner points to pre-calibrate the camera. Based on the consistent time flow rate of the separate sensors, multiple virtual detection lines are set up to match the time headway of successive vehicles and conduct objective matching to track data. Finally, a synchronization optimization model is formulated and a constrained nonlinear minimization solver is applied to tune the parameters. Measure data from Donghai Bridge in Shanghai is applied to verify the feasibility and effectiveness of the method. The results determine that there are 33 frames (33*40 ms) of temporal deviation between the camera and the radar in this case. After the synchronization, the average spatial deviation is reduced from 2.47 m to 0.42 m in the X-direction and 64.06 m to 2.34 m in the Y-direction, respectively. This study provides an economical and effective way to solve the problem of spatio-temporal synchronization of roadside sensors.

Automatic Extraction of Corner Points from Aerial Images Using Point-Feature Operators and Hough Transform

Low-level feature extraction such as lines and points (i.e. corners), forms a fundamental step in digital photogrammetry and other fields. They supply the inputs for the photogrammetric orientation procedures; and they serve as an intermediate input for other processes such as object recognition. With the accumulation of knowledge, the research community is in a better position to develop new generations of smart algorithms and solutions that possess a new level of maturity and understanding for the underlying challenges of automation. To this end, this paper presents an innovative approach for corner point extraction that combines the outputs from classical point feature operators with Hough Transform to generate a better hypothesis for a corner point that can be used for applications in urban areas. In particular, extracted point features were used to guide line extraction in a local neighbourhood by Hough Transform. Then the corner points that will be obtained from lines intersection in this local neighbourhood will be compared with their nearby ones that were extracted by point feature operators. Based on passing a set of criteria, the intersection points from lines will replace the point feature as aset of potential corner points. Experimental findings show promising results of the proposed approach that raises the confidence level of the extracted corners and eliminating outliers.

Automatic Template Detection for Camera Calibration

Camera calibration is the process of extract the intrinsic and extrinsic parameters of a camera. Those parameters guide the 3-dimensional localization into relation to the 2-dimensional space from the images acquired by the camera. The 3-dimensional correlation can be generated with an object with known measures, being the most common checkerboard for this purpose. From these checker- boards, the usual approach extracts the position of the inner points, equivalent to the corners of the squares, to generate this correlation. A broad range of algorithms tries to find those points on the image. Still, usually, they require previous knowledge about the dimensions of the image, the pattern distribution, or even the pattern type. In some scenario, maybe is difficult, or impossible, to implement such precise solution, targeting these limitations our work proposes a two-step end-to-end convolutional neural network architecture that processes the corner detection on a unique flow. Our proposal is agnostic to checkerboard size, pattern disposal, and positioning. In our work, first, a segmentation CNN extracts only the checkerboard from the input image (CheckerNet); from the extracted checkerboard, we extract the corner points with a corner detection CNN (Point- Net). The PointNet also works as a segmentation CNN, and the generated points are heatmaps related to points on the checkerboard corners. We performed post-processing with a K-Means-based clustering to convert those heatmaps into single positions (x,y) from the image. We compare our proposed method with the other well-known convolutional neural networks used for corner detection MATE and CCDN. For the evaluation, two datasets were used: GoPro e uEye. Our method provides better results in both datasets, reducing missed corners, double detections, false positives, and competitive results on pixel accuracy.

Registration of Building Scan with IFC-Based BIM Using the Corner Points

Progress monitoring is an essential part of large construction projects. As manual progress monitoring is time-consuming, the need for automation emerges, especially as, nowadays, BIM for the design of buildings and laser scanning for capturing the as-built situation have become well adopted. However, to be able to compare the as-built model obtained by laser scanning to the BIM design, both models need to use the same reference system, which often is not the case. Transforming the coordinate system of the as-built model into the BIM model is a specialist process that is pre-requisite in automated construction progress monitoring. The research described in this paper is aimed at the automation of this so-called registration process and is based on the dominant planar geometry of most buildings with evident corner points in their structures. After extracting these corner points from both the as-built and the design model, a RANSAC-based pairwise assessment of the points is performed to identify potential matching points in both models using different discriminative geometric invariants. Next, the transformation for the potential matches is evaluated to find all the matching points. In the end, the most accurate transformation parameter is determined from the individual transformation parameters of all the matching corner points. The proposed method was tested and validated with a range of both simulated and real-life datasets. In all the case studies including the simulated and real-life datasets, the registration was successful and accurate. Furthermore, the method allows for the registration of the as-built models of incomplete buildings, which is essential for effective construction progress monitoring. As the method uses the standard IFC schema for data exchange with the BIM, there is no loss of geometrical information caused by data conversions and it supports the complete automation of the progress-monitoring process.

Focusing on your subject: Deep subject-aware image composition recommendation networks

Photo composition is one of the most important factors in the aesthetics of photographs. As a popular application, composition recommendation for a photo focusing on a specific subject has been ignored by recent deep-learning-based composition recommendation approaches. In this paper, we propose a subject-aware image composition recommendation method, SAC-Net, which takes an RGB image and a binary subject window mask as input, and returns good compositions as crops containing the subject. Our model first determines candidate scores for all possible coarse cropping windows. The crops with high candidate scores are selected and further refined by regressing their corner points to generate the output recommended cropping windows. The final scores of the refined crops are predicted by a final score regression module. Unlike existing methods that need to preset several cropping windows, our network is able to automatically regress cropping windows with arbitrary aspect ratios and sizes. We propose novel stability losses for maximizing smoothness when changing cropping windows along with view changes. Experimental results show that our method outperforms state-of-the-art methods not only on the subject-aware image composition recommendation task, but also for general purpose composition recommendation. We also have designed a multistage labeling scheme so that a large amount of ranked pairs can be produced economically. We use this scheme to propose the first subject-aware composition dataset SACD, which contains 2777 images, and more than 5 million composition ranked pairs. The SACD dataset is publicly available at https://cg.cs.tsinghua.edu.cn/SACD/.

Robust state feedback stabilisation of positive LTI systems with polytopic uncertainty

ABSTRACT This paper deals with the robust state feedback stabilizability problem of linear time-invariant (LTI) systems in the presenceof polytopic uncertainty. The paper contributes two major results involving (i) necessary and sufficient conditions to simultaneouslycheck positivity and robust stability of an uncertain LTI system (ii) the development of a design algorithm to solve the mainstabilizability problem. This problem is not easy to deal with because it is not a convex problem. To cope with this issue, a designalgorithm is proposed that checks the design space of the stabilizer parameters, divides it into smaller subspaces, checks thefeasibility of the corner points of the design subspaces, removes the detected total infeasible design subspaces, and shrinks theremaining parts to solve the design problem, iteratively. The proposed algorithm is applied to the cancer chemotherapy applicationand the results demonstrate its efficacy to control the model considering the positivity condition.

A METHOD FOR THE STAKE-OUT OF THE SEATS IN THE CONFERENCE ROOM

Before the application, the coordinates of the circle shaped seat places that will be applied in indoor space can be obtained by the help of common points determined in numerical project model formed in a local coordinate system. Practically, the coordinates of the points in the digital project model can be also obtained numerically by using the coordinates of the common points. However, converting to numerals manually, the factors such as the support quality of the project, exhaustion of the operator, hand tremble and screen resolution influence the sensitivity of the conversion to numerals. Errors made during the conversion negatively affect the coordinates of the points. The most suitable method not to make errors is the calculation of coordinates of the points in the project model by geodetic method. In this project, the calculation of the coordinates of the seat midpoints placed on the curves in a closed area, benefited from the corner point coordinates of the seats in the indoor space obtained from geodetic measurement in a local coordinate system before the application and benefited from the data of the other points of the indoor space are explained. Furthermore, the ground application of the midpoints of the seats on each curve row is explained. The accuracy of the stake-out of seats in the conference room is obtained between 1 and 2 cm.

Zaman gecikmeli sistemler için Padé yaklaşımı ve kesirli dereceli PI kontrolör için yeni bir tasarım metodu

Kesirli dereceli PI kontrolörler, günden güne getirdiği kolaylıklar sayesinde birçok makalenin araştırma konusu olmuştur. Bu tür bir kontrolör tam sayı dereceli PI ile karşılaştırıldığında çok daha faydalı sonuçlar sunar. Fakat işlemleri daha karmaşık bir hale getirir. Bu çalışmada kararlılık sınır eğrisi metodu kullanılarak geliştirilmiş olan dışbükey kararlılık bölgesinin ağırlık merkezine dayalı basit bir kesirli dereceli PI kontrolör tasarım yöntemi önerilmiştir. Bunun için önce kesirli dereceli PI kontrolörün parametre düzleminde tüm kararlı kontrolörleri sağlayan kararlılık bölgesi elde edilir. Daha sonra bu kararlılık bölgesinin köşe ve tepe noktaları belirlenerek dışbükey kararlılık bölgesi elde edilir. Son olarak elde edilen dışbükey kararlılık bölgesinin ağırlık merkezi bulunarak kontrolör parametreleri hesaplanır. Kullanılan yöntem, kontrolör parametrelerinin karmaşık grafiksel yöntemler kullanılmadan hesaplanması, kapalı döngü sisteminin kararlılığını sağlaması ve hesaplama yükünü oldukça azaltması bakımından önemli avantajlar sağlamaktadır. Ayrıca, bu çalışmada kararlılık bölgesinin elde edilmesinde kullanılan denklemler için literatürdeki çalışmalardan farklı olarak zaman gecikmesi terimi yerine Padé yaklaşımlarının kullanılması tercih edilmiştir. Özellikle ikinci dereceden Padé yaklaşımı kullanılarak elde edilen sonuçların neredeyse orijinal sistemle birebir örtüştüğü görülmektedir. Önerilen yöntemin etkinliğini ve basitliğini göstermek için bazı sayısal örnekler verilmiştir.

Thrust characteristics of a truncated Laval nozzle with a bell-shaped tip

This paper presents the results of a thrust performance study of an unconventionally shaped supersonic nozzle in the form of a truncated Laval nozzle with a bell-shaped tip. This nozzle shape may be used in the development of compact layouts of multistage rockets. The study was carried out using the ANSYS software package in a 3D formulation. The methodological approaches to the numerical calculation of a complex separated gas flow used in this study were verified in a previous study of the flow pattern in similar nozzle. Some results of exact calculations were compared with the results of experimental studies carried out at the Institute of Technical mechanics of the National Academy of Sciences of Ukraine and the State Space Agency of Ukraine for a model of a similar truncated nozzle with a bell-shaped tip blown with a cold air. This study detailed the features of the separated gas flow in a spherical tip connected (at the corner point) to a truncated supersonic Laval nozzle of conical shape. It was found that the pattern of the separated flow in the tip depends on the nozzle flow expansion degree (nozzle inlet pressure). At a relatively low nozzle inlet pressure, a developed separation zone is observed in the nozzle tip (between the jet boundary and the nozzle wall) with a subsonic flow from the external environment, which forms an almost constant static pressure from the tip inlet cross-section to the tip exit. At a nozzle inlet pressure at which the free boundary of the jet flowing from the truncated nozzle adjoins the nozzle wall, the static pressure in the tip varies almost linearly along the tip length from the corner point with the minimum pressure to the tip exit. The dependence of the thrust of a tipped nozzle on the nozzle inlet pressure is nonlinear. As the pressure upstream of the nozzle increases (or the ambient pressure decreases), the effect of the external environment on the tipped-nozzle thrust diminishes. It is shown that under "terrestrial conditions" the thrust of a truncated nozzle with a tip exceeds the thrust of a profiled nozzle with the same geometric expansion degree (due to the atmosphere “entering” the tip). Under "vacuum” conditions, the former is 8% less than the latter.

Real-time detection of crop rows in maize fields based on autonomous extraction of ROI

The current crop rows detection based on machine vision generally has the problems of low detection accuracy and poor real-time performance. Moreover, crop rows detection remains a challenging problem in complex field conditions, such as high weeds pressure, poor illumination conditions, and vegetation foliage shading. We propose a crop rows detection algorithm based on autonomous extraction of ROI (Region of interest). The prior method computes the feature points of the entire image and groups them into the crop rows which they belong to. Instead, we consider the core of crop rows detection problem to be the extraction of the travelling area of agricultural machinery in maize fields. A YOLO (You Only Look Once) neural network is employed to predict the travelling area of the agricultural machinery end-to-end. The prediction boxes are unified into ROI and the crop and soil background are segmented in the ROI by Excess Green operator and Otsu’s method. Then, the feature points of crops are extracted using FAST (Features from Accelerated Segment Test) corner point detection, and finally the detection lines of crop rows are fitted with least squares method. Because image recognition is limited to a valid region after the ROI is extracted, the processing speed of our algorithm is remarkably fast. It takes only about 25 ms to process a single image (640*360 pixels) and the frame rate of video stream exceeds 40FPS. Meanwhile, it can achieve high accuracy and robust extraction of ROI in various maize fields. The average error angle of the detection lines is 1.88°, which can meet the real-time and accuracy requirements of field navigation. The proposed algorithm can provide a new solution to the current machine vision-based navigation technology for agricultural machinery. Code is available at: https://github.com/WoodratTradeCo/crop-rows-detection.

Automatic registration of optical image and airborne LiDAR data based on centers and corner points of building boundaries

ABSTRACT Applications based on the fusion of aerial imagery and Light Detection and Ranging (LiDAR) data have attracted more and more interest in recent years. Registration is the crucial prerequisite of data integration because of the spatial shift between two data modalities. However, there are many challenges for registration due to the different mechanisms of LiDAR and camera, especially in complex urban scenes. To address the problem, this paper proposes an automatic and robust registration method by using the centres and corner points of building boundaries as tie points. Primary registration is achieved by using centers of building boundaries as control points. To increase the robustness and accuracy of the registration, corner points of building boundaries are utilized to estimate the transformation model repeatedly. Two datasets of optical imagery and LiDAR data with different scene coverage are used to verify the performance of the proposed method. Experimental results show that the proposed method can achieve the registration accuracy of less than 2 pixels for both cases based on an assessment of check lines. This result can basically meet the requirements of data fusion.

Research on a rapid identification method for counting universal grain crops.

Thousand-grain weight is a key indicator of crop yield and an important parameter for evaluating cultivation measures. Existing methods based on image analysis are convenient but lack a counting algorithm that is suitable for multiple types of grains. This research develops an application program based on an Android device to quickly calculate the number of grains. We explore the short axis measurement method of the grains with morphological thought, and determine the relationship between the general corrosion threshold and the short axis. To solve the problem of calculating the number of grains in the connected area, the study proposes a corrosion algorithm based on the short axis and an improved corner point method. After testing a variety of crop grains and equipment, it was found that the method has high universality, supports grain counting with white paper as the background, and has high accuracy and calculation efficiency. The average accuracy rate is 97.9%, and the average time is less than 0.7 seconds. In addition, the difference between the average accuracy for various mobile phones and multiple crops is small. This research proposes a grain counting algorithm with a wide range of applications to meet the requirements of nonglare use in the field. The algorithm provides a fast, accurate, low-cost tool for counting grains of wheat, corn, mung bean, soybean, peanut, rapeseed, etc., which is less constrained by space and power conditions. The algorithm is highly adaptable and can provide a reference for the study of grain counting.

Point2Roof: End-to-end 3D building roof modeling from airborne LiDAR point clouds

Three-dimensional (3D) building roof reconstruction from airborne LiDAR point clouds is an important task in photogrammetry and computer vision. To automatically reconstruct the 3D building models at Level of Detail 2 (LoD-2) from airborne LiDAR point clouds, the data-driven approaches usually need to be performed in two steps: geometric primitive extraction and roof structure inference. Obviously, the traditional approaches are not end-to-end, the accumulated errors in different stages cannot be avoided and the final 3D roof models may not be optimal. In addition, the results of 3D roof models largely depend on the accuracy of geometric primitives (planes, lines, etc.). To solve these problems, we present a deep learning-based approach to directly reconstruct building roofs from airborne LiDAR point clouds, named Point2Roof. In our method, we start by extracting the deep features for each input point using PointNet++. Then, we identify a set of candidate corner points from the input point clouds using the extracted deep features. In addition, we also regress the offset for each candidate corner point to refine their locations. After that, these candidates are clustered into a set of initial vertices, and we further refine their locations to obtain the final accurate vertices. Finally, we propose a Paired Point Attention (PPA) module to predict the true model edges from an exhaustive set of candidate edges between the vertices. Unlike traditional roof modeling approaches, the proposed Point2Roof is end-to-end. However, due to the lack of a building reconstruction dataset, we construct a large-scale synthetic dataset to verify the effectiveness and robustness of the proposed Point2Roof. The experimental results conducted on the synthetic benchmark demonstrate that the proposed Point2Roof significantly outperforms the traditional roof modeling approaches. The experiments also show that the network trained on the synthetic dataset can be applied to the real point clouds after fine-tuning the trained model on a small real dataset. The large-scale synthetic dataset, the small real dataset and the source code of our approach are publicly available in https://github.com/Li-Li-Whu/Point2Roof.

IAMOT: Multi-object tracking with integrated heads and attention mechanism

Multi-object tracking is one of the most fundamental problems in computer vision with wide industrial applications. It involves the association of multiple targets across consecutive frames. The current trend has been gradually switched from tracking-by-detection paradigm to one-shot trackers. While special efforts have been made on anchor-free methods which turned out to be more suitable for the extraction of re-ID features (than anchor-based methods). However, most of the works used center points to represent the targets and regressed other properties, which lost global information and were unfavorable to ID switches. To address the above issues, we propose IAMOT, a simple yet effective network based on anchor-free architecture. Our network predicts the center point and derives the corner points of an object to obtain a refined bounding box through integrated heads. It also utilizes an additional attention module to weaken the occurrence of ID switches. Extensive experiments have been conducted on 4 mainstream datasets, where IAMOT exhibits superior performances by surpassing other state-of-the-art methods. Especially, we achieve 60.6% MOTA on MOT15, 75.8% MOTA on MOT16, 74.4% MOTA on MOT17 and 64.1% MOTA on MOT20, respectively.

Novel segmentation method and measurement system for various grains with complex touching

Thousand kernel weight is one of the essential indicators for evaluating grain quality. Manual work and mechanical counter are traditional means for thousand kernel weight measurement, but they cannot meet the requirements of modern applications in efficiency and intelligence. Machine vision is a powerful and promising technology that can replace traditional measurement means in many fields. Grain segmentation and counting are significant challenges in thousand kernel weight measurement relying on machine vision. This study proposed a novel segmentation method for complex-touching grains of different species, shapes, and sizes. In the method, three specialized algorithms were applied to segment non-touching, simple-touching, and complex-touching grains in the captured images to improve segmentation accuracy. The K-means clustering algorithm was used to extract non-touching grains, and a layered watershed algorithm was created to segment simple-touching grains. A splitting line detection algorithm was innovatively designed by the determination of concave corner point sets, line detection, and line extension to realize accurate segmentation of complex-touching grains. Error correction was carried out at the end of the proposed method to reduce under-segmentation and over-segmentation. Furthermore, a thousand kernel weight measurement system was developed based on the proposed segmentation method. The results for 1000 images containing 640 thousand grains showed that the developed system achieved an average measurement accuracy of 99.65 % and an average measurement time of 1.54 s. The measurement accuracy could be further improved to 100 % by the manual correction. The developed system was distinctly superior in accuracy, efficiency, and application scope for the thousand kernel weight measurement of various grains with complex touching.