Shape Extraction Research Articles

ACU-TransNet: Attention and convolution-augmented UNet-transformer network for polyp segmentation.

UNet has achieved great success in medical image segmentation. However, due to the inherent locality of convolution operations, UNet is deficient in capturing global features and long-range dependencies of polyps, resulting in less accurate polyp recognition for complex morphologies and backgrounds. Transformers, with their sequential operations, are better at perceiving global features but lack low-level details, leading to limited localization ability. If the advantages of both architectures can be effectively combined, the accuracy of polyp segmentation can be further improved. In this paper, we propose an attention and convolution-augmented UNet-Transformer Network (ACU-TransNet) for polyp segmentation. This network is composed of the comprehensive attention UNet and the Transformer head, sequentially connected by the bridge layer. On the one hand, the comprehensive attention UNet enhances specific feature extraction through deformable convolution and channel attention in the first layer of the encoder and achieves more accurate shape extraction through spatial attention and channel attention in the decoder. On the other hand, the Transformer head supplements fine-grained information through convolutional attention and acquires hierarchical global characteristics from the feature maps. mcU-TransNet could comprehensively learn dataset features and enhance colonoscopy interpretability for polyp detection. Experimental results on the CVC-ClinicDB and Kvasir-SEG datasets demonstrate that mcU-TransNet outperforms existing state-of-the-art methods, showcasing its robustness.

Harumanis Mango Classification and Grading System Based on Geometric Shape Extraction for Quality Assessment

In agricultural research, a fruit's look, which affects its market value, is the first and most crucial sensory evaluation. According to external criteria like shape, size, ripeness index, and flaws or internal criteria like sweetness, flavor, and nutrients, fruits can be categorized and rated. Due to the grading task's importance and difficulty in post-harvest management, there is a high standard for fruit quality that must constantly be met. Growing interest in various categorization and grading algorithms aims to use non-contact and novel measurements to lessen the subjectivity of visual examination. In this paper, the proposed system that uses fruit shape and size as feature parameters for evaluating fruit quality has been implemented. A morphological analysis, median filter, and multilevel threshold-based image processing technique were created to assess the geometric shape of the mango image, including its length, width, and area. These factors are required to compute the standard shape of the Harumanis mango's similarity index and divide its size into four mass grades: small (S), medium (M), large (L), and extra-large (XL). The test results show the suggested technique had an accuracy rate of 98% and 96% for grading the size based on mass estimation and was very successful for quality classification based on measurement of the similarity index for a standard shape.

ECLiPSE: a versatile classification technique for structural and morphological analysis of 2D and 3D single-molecule localization microscopy data

Single-molecule localization microscopy (SMLM) has gained widespread use for visualizing the morphology of subcellular organelles and structures with nanoscale spatial resolution. However, analysis tools for automatically quantifying and classifying SMLM images have lagged behind. Here we introduce Enhanced Classification of Localized Point clouds by Shape Extraction (ECLiPSE), an automated machine learning analysis pipeline specifically designed to classify cellular structures captured through two-dimensional or three-dimensional SMLM. ECLiPSE leverages a comprehensive set of shape descriptors, the majority of which are directly extracted from the localizations to minimize bias during the characterization of individual structures. ECLiPSE has been validated using both unsupervised and supervised classification on datasets, including various cellular structures, achieving near-perfect accuracy. We apply two-dimensional ECLiPSE to classify morphologically distinct protein aggregates relevant for neurodegenerative diseases. Additionally, we employ three-dimensional ECLiPSE to identify relevant biological differences between healthy and depolarized mitochondria. ECLiPSE will enhance the way we study cellular structures across various biological contexts.

Application of U-Net Network Utilizing Multiattention Gate for MRI Segmentation of Brain Tumors.

Studies have shown that the type of low-grade glioma is associated with its shape. The traditional diagnostic method involves extraction of the tumor shape from MRIs and diagnosing the type of glioma based on corresponding relationship between the glioma shape and type. This method is affected by the MRI background, tumor pixel size, and doctors' professional level, leading to misdiagnoses and missed diagnoses. With the help of deep learning algorithms, the shape of a glioma can be automatically segmented, thereby assisting doctors to focus more on the diagnosis of glioma and improving diagnostic efficiency. The segmentation of glioma MRIs using traditional deep learning algorithms exhibits limited accuracy, thereby impeding the effectiveness of assisting doctors in the diagnosis. The primary objective of our research is to facilitate the segmentation of low-grade glioma MRIs for medical practitioners through the utilization of deep learning algorithms. In this study, a UNet glioma segmentation network that incorporates multiattention gates was proposed to address this limitation. The UNet-based algorithm in the coding part integrated the attention gate into the hierarchical structure of the network to suppress the features of irrelevant regions and reduce the feature redundancy. In the decoding part, by adding attention gates in the fusion process of low- and high-level features, important feature information was highlighted, model parameters were reduced, and model sensitivity and accuracy were improved. The network model performed image segmentation on the glioma MRI dataset, and the accuracy and average intersection ratio (mIoU) of the algorithm segmentation reached 99.7%, 87.3%, 99.7%, and 87.6%. Compared with the UNet, PSPNet, and Attention UNet network models, this network model has obvious advantages in accuracy, mIoU, and loss convergence. It can serve as a standard for assisting doctors in diagnosis.

A morphology-Euclidean-linear recognition method for rebar point clouds of highway tunnel linings during the construction phase

PurposeLaser point clouds are a 3D reconstruction method with wide range, high accuracy and strong adaptability. Therefore, the purpose is to discover a construction point cloud extraction method that can obtain complete information about the construction of rebar, facilitating construction quality inspection and tunnel data archiving, to reduce the cost and complexity of construction management.Design/methodology/approachFirstly, this paper analyzes the point cloud data of the tunnel during the construction phase, extracts the main features of the rebar data and proposes an M-E-L recognition method. Secondly, based on the actual conditions of the tunnel and the specifications of Chinese tunnel engineering, a rebar model experiment is designed to obtain experimental data. Finally, the feasibility and accuracy of the M-E-L recognition method are analyzed and tested based on the experimental data from the model.FindingsBased on tunnel morphology characteristics, data preprocessing, Euclidean clustering and PCA shape extraction methods, a M-E-L identification algorithm is proposed for identifying secondary lining rebars in highway tunnel construction stages. The algorithm achieves 100% extraction of the first-layer rebars, allowing for the three-dimensional visualization of the on-site rebar situation. Subsequently, through data processing, rebar dimensions and spacings can be obtained. For the second-layer rebars, 55% extraction is achieved, providing information on the rebar skeleton and partial rebar details at the construction site. These extracted data can be further processed to verify compliance with construction requirements.Originality/valueThis paper introduces a laser point cloud method for double-layer rebar identification in tunnels. Current methods rely heavily on manual detection, lacking objectivity. Objective approaches for automatic rebar identification include image-based and LiDAR-based methods. Image-based methods are constrained by tunnel lighting conditions, while LiDAR focuses on straight rebar skeletons. Our research proposes a 3D point cloud recognition algorithm for tunnel lining rebar. This method can extract double-layer rebars and obtain construction rebar dimensions, enhancing management efficiency.

Learning Robust Shape Regularization for Generalizable Medical Image Segmentation.

Generalizable medical image segmentation enables models to generalize to unseen target domains under domain shift issues. Recent progress demonstrates that the shape of the segmentation objective, with its high consistency and robustness across domains, can serve as a reliable regularization to aid the model for better cross-domain performance, where existing methods typically seek a shared framework to render segmentation maps and shape prior concurrently. However, due to the inherent texture and style preference of modern deep neural networks, the edge or silhouette of the extracted shape will inevitably be undermined by those domain-specific texture and style interferences of medical images under domain shifts. To address this limitation, we devise a novel framework with a separation between the shape regularization and the segmentation map. Specifically, we first customize a novel whitening transform-based probabilistic shape regularization extractor namely WT-PSE to suppress undesirable domain-specific texture and style interferences, leading to more robust and high-quality shape representations. Second, we deliver a Wasserstein distance-guided knowledge distillation scheme to help the WT-PSE to achieve more flexible shape extraction during the inference phase. Finally, by incorporating domain knowledge of medical images, we propose a novel instance-domain whitening transform method to facilitate a more stable training process with improved performance. Experiments demonstrate the performance of our proposed method on both multi-domain and single-domain generalization.

Poster 354: Tibiofemoral Bone Shape Features Are Associated with Re-Tear after Anterior Cruciate Ligament Reconstruction

Introduction: Knee bony morphology is linked to anterior cruciate ligament (ACL) injury, ACL graft failure, and osteoarthritis (OA) risk, and employing tools such as statistical shape modeling could identify individuals prone to these joint conditions. Re-tear after ACL reconstruction remains a common and devastating complication, and therefore, recognizing patients at high risk of graft failure before primary ACL reconstruction is crucial for potential intervention to reduce failure rates. Magnetic resonance imaging (MRI) is most commonly obtained in the setting of possible ACL injury and before ACL reconstruction surgery, yet currently available MRI-based bone shape assessment methods require high-resolution sequences or manual segmentation, limiting feasibility of large-scale analysis. The development of automated tools for bone shape extraction from clinical MRI could allow for three-dimensional statistical shape modeling across large datasets. We established an automated pipeline for bone segmentation from proton density (PD)-weighted MRI scans, aiming to extract knee bony morphology from preoperative clinical MRIs in patients with ACL injuries. We hypothesized that our segmentation pipeline could uncover specific tibiofemoral joint bone shape features associated with ACL re-tear after reconstruction. Objective: To identify bone shape features prior to initial primary ACL reconstruction in patients with eventual ACL graft re-tear compared to those with a known, intact ACL graft. Methods: Patients who retrospectively experienced a subsequent re-tear after reconstruction within three years of initial injury were compared to those without a re-tear within three years (20 without subsequent re-tear, 22 with subsequent re-tear for distal femur; 22 without subsequent re-tear, 19 with subsequent re-tear for proximal tibia). The control group had MRI scans at 3 years after reconstruction that confirmed an intact ACL graft. An automatic femur and tibia segmentation framework was developed by training two deep convolutional neural networks using clinical, two-dimensional PD-weighted knee MRI sequences (3.5 mm slice thickness) from patients with ACL tears. Using the segmentation pipeline, bone was automatically segmented from preoperative clinical PD-weighted knee MRI sequence (3 mm slice thickness), three-dimensional triangulated meshes were generated, scaled to a reference, and aligned and matched to an atlas using Iterative Closest Point algorithm. Modes of shape variance were extracted from principal component analysis (PCA). The effect of each mode is modeled by three-dimensional surfaces for each specific shape feature. Modes were considered statistically significant if the mean principal component score between groups was p<0.05 using Students T-test. Results: Statistical shape modeling of the femoral condyles using 15 PCA modes reveals significant differences (p < 0.05) in Modes 1 and 4 at the initial injury in patients who subsequently experienced ACL re-tear after reconstruction compared to those without re-tear. Femur Mode 1 is characterized by a narrower intercondylar notch width, a wider medial condylar width, and an increased ratio of femoral condylar offset in patients with ACL re-tear. Femur Mode 4 illustrates a more prominent trochlear sulcus and increased surface area along the lateral femoral condyle relative to the medial condyle in patients with ACL re-tear. The initial 15 PCA modes of the distal femur collectively account for 75.04% of the normalized accumulated total variance. Statistical shape modeling of the tibial plateau with 15 PCA modes reveals that Mode 7 is significantly different (p < 0.05) at the initial injury in patients who experienced subsequent ACL re-tear after reconstruction compared to those without re-tear. Tibia Mode 7 indicates diminished ACL facet prominence, a squared lateral tibial plateau, and a broader, flatter tibial spine in patients with subsequent ACL re-tear. The first 15 PCA modes of the proximal tibia collectively explain 73.60% of the normalized accumulated total variance. Conclusions: Using an automatic bone segmentation pipeline on preoperative clinical MRIs, we identify specific shape features in the tibiofemoral joint associated with ACL re-tear after reconstruction. Patients with subsequent ACL re-tear exhibit a variety of distinct three-dimensional morphology differences, including reduced intercondylar notch width, wider medial condylar width, increased lateral tibial plateau convexity, and broader, lateralized tibial spine, uncovered using statistical shape modeling. Statistical shape modeling identifies three-dimensional morphology patterns, thus capturing differences otherwise unaccounted for by traditional two-dimensional measurements. This approach offers predictive potential for patients at risk of ACL re-tear, improving treatment decisions for those with high-risk morphologic features.

Multi-scale Knowledge Transfer Vision Transformer for 3D vessel shape segmentation

In order to facilitate the robust and precise 3D vessel shape extraction and quantification from in-vivo Magnetic Resonance Imaging (MRI), this paper presents a novel multi-scale Knowledge Transfer Vision Transformer (i.e., KT-ViT) for 3D vessel shape segmentation. First, it uniquely integrates convolutional embeddings with transformer in a U-net architecture, which simultaneously responds to local receptive fields with convolution layers and global contexts with transformer encoders in a multi-scale fashion. Therefore, it intrinsically enriches local vessel feature and simultaneously promotes global connectivity and continuity for a more accurate and reliable vessel shape segmentation. Furthermore, to enable using relatively low-resolution (LR) images to segment fine scale vessel shapes, a novel knowledge transfer network is designed to explore the inter-dependencies of data and automatically transfer the knowledge gained from high-resolution (HR) data to the low-resolution handling network at multiple levels, including the multi-scale feature levels and the decision level, through an integration of multi-level loss functions. The modeling capability of fine-scale vessel shape data distribution, possessed by the HR image transformer network, can be transferred to the LR image transformer to enhance its knowledge for fine vessel shape segmentation. Extensive experimental results on public image datasets have demonstrated that our method outperforms all other state-of-the-art deep learning methods.

Automated geometric reconstruction and cable force inference for cable-net structures using 3D point clouds

Laser scanning provides an efficient solution to digital twin construction in civil engineering. The complexity and redundancy of large-scale point clouds substantially prolong the labor-intensive model reconstruction process. This paper presents an automated and high-precision geometric reconstruction approach for cable-net structures with a complete workflow from raw points to the extraction of cable shapes and forces. The strategy involves the extraction of coordinates of key cable regions through the characterization of cable geometry and local registration methods, followed by the computation of cable forces and shapes using segmented catenary theory and nonlinear optimization. The approach is validated with a single-curvature cable-net structure, with the accuracy of cable shapes within ±5 mm and errors in cable forces <5%. The method contributes to health monitoring and rapid reconstruction of cable-net structures. The precise geometry and forces will facilitate the creation of the mechanical models reflecting the physical authenticity of structures.

Self-Supervised Lightweight Depth Estimation in Endoscopy Combining CNN and Transformer.

In recent years, an increasing number of medical engineering tasks, such as surgical navigation, pre-operative registration, and surgical robotics, rely on 3D reconstruction techniques. Self-supervised depth estimation has attracted interest in endoscopic scenarios because it does not require ground truth. Most existing methods depend on expanding the size of parameters to improve their performance. There, designing a lightweight self-supervised model that can obtain competitive results is a hot topic. We propose a lightweight network with a tight coupling of convolutional neural network (CNN) and Transformer for depth estimation. Unlike other methods that use CNN and Transformer to extract features separately and then fuse them on the deepest layer, we utilize the modules of CNN and Transformer to extract features at different scales in the encoder. This hierarchical structure leverages the advantages of CNN in texture perception and Transformer in shape extraction. In the same scale of feature extraction, the CNN is used to acquire local features while the Transformer encodes global information. Finally, we add multi-head attention modules to the pose network to improve the accuracy of predicted poses. Experiments demonstrate that our approach obtains comparable results while effectively compressing the model parameters on two datasets.

Frequency response function-based closed-form expression for multi-damage quantification and its application on shear buildings

Vibration-based damage identification techniques have gained popularity immensely in the field of structural health monitoring (SHM). Frequency response function (FRF)-based damage quantification is mainly performed based on iterative model updating techniques. Most of these methods are time-consuming and erroneous in higher modes. Spectral element method (SEM) can overcome some of these limitations and can determine FRF expression directly for a broad class of civil structures. However, a closed-form expression for multiple damage severity is not proposed to date. In this study, an FRF-based formulation for quantifying damage has been proposed with the help of SEM in order to bypass the mode shape extraction for damage identification. To determine the effectiveness of the proposed formulation both numerical and experimental studies have been performed. A numerical simulation has been performed on a 14-storey shear building. Damage has been introduced by reducing the storey stiffness at intermediate and adjacent storeys. It has been perceived that damage severity at any storey can be estimated with only 3 sensors, one at the top storey and two others at the adjacent floors of the desired storey. To verify the effectiveness of the proposed expression, an experimental study has been carried out on a 9-storey miniature model. It has been observed that the first resonating peak amplitudes of the responses recorded simultaneously at desired locations of intact and damaged structures are sufficient to estimate the damage intensity. Thus, the proposed formulation is efficient as an output measurement-based approach. The novelty of this study lies in the proposed closed-form FRF-based multi-damage severity expression involving minimum number of sensors and without any information on the input excitation. The results obtained from both numerical and experimental study showcase the robustness, simplicity and applicability of the proposed formulation.

Extraction of three-dimensional shapes in glaucoma patients in response to monocular depth cues.

To assess the impact of glaucoma on perceiving three-dimensional (3D) shapes based on monocular depth cues. Clinical observational study. Twenty glaucoma patients, subjected to binocular visual-field sensitivity (binocular-VFS) tests using a Humphrey Visual Field Analyzer, and 20 age-matched healthy volunteers, underwent two tasks: identifying the nearest vertex of a 3D shape using monocular shading (3D-SfS), texture (3D-SfT), or motion (3D-SfM) cues, and distinguishing elementary one-dimensional (1D) features of these cues. The association of the visual-field index (VFI) of binocular-VFS with 3D shape perception in glaucoma patients was also examined. Glaucoma patients demonstrated reduced accuracy in distinguishing 1D luminance brightness and a larger "error-in-depth" between the perceived and actual depths for 3D-SfM and 3D-SfS compared to healthy volunteers. Six glaucoma patients with a 100% VFI for binocular-VFS exhibited a similar error-in-depth to the other fourteen glaucoma patients; they had a larger error-in-depth for 3D-SfM compared to healthy volunteers. No correlation between the error-in-depth values and the VFI values of binocular-VFS was observed. The 3D shape perception in glaucoma patients varies based on the depth cue's characteristics. Impaired 1D discrimination and larger thresholds for 3D-SfM in glaucoma patients with a 100% VFI for binocular-VFS indicate more pronounced perceptual deficits of lower-level elementary features for 3D-SfS and higher-level visual processing of 3D shapes for 3D-SfM. The effects of the location and degree of binocular visual-field defects on 3D shape perception remain to be elucidated. Our research provides insights into the 3D shape extraction mechanism in glaucoma.

Expression of Concern: Oil painting color image enhancement recognition method based on artificial intelligence: applications of an AI model in environmental research

ABSTRACT Due to the pollution of the air and water environment and the problem of forgery, it is difficult to identify oil paintings. The reason is that air pollution and water pollution can lead to moisture, mold, and even water stains on the picture, which will seriously damage the integrity and color performance of the picture. At the same time, chemicals in the water may also have a corrosive effect on the oil painting, further destroying the color and detail of the picture. The problem of relying entirely on the conventional experience of experts is too subjective. Some controversial works are difficult to convince people with rational identification evidence, so it is necessary to explore a scientific and effective method to quantify the authenticity of oil paintings. This paper constructs an oil painting authenticity identification method based on multi-feature fusion based on the artistic style analysis and feature extraction of oil painting shape, color and texture. The recognition accuracy of the proposed method is compared with that of the existing neural network. The results show that the recognition rate of the proposed model is 73.0%, which is the best performance.

Extraction of Shape and Texture Features of Dermoscopy Image for Skin Cancer Identification

Skin diseases are increasing and becoming a very serious problem. Skin cancer in general there are 2, namely melanoma and non-melanoma. Cases that are often encountered are in non-melanoma types. A critical factor in the treatment of skin cancer is early diagnosis. Doctors usually use the biopsy method to detect skin cancer. Computer-based technology provides convenient, cheaper, and faster diagnosis of skin cancer symptoms. This study aims to identify the type of skin cancer. The data used in the study were 6 types of skin cancer, namely Basal Cell Carcinoma, Dermatofibroma, Melanoma, Nevus image, Pigmented Benign Keratosis image, or Vascular Lesion, with a total of 60 dermoscopy images obtained from the Kaggle site. Dermoscopy image processing begins with a pre-processing process, which converts RGB images to LAB. After that, segmentation is carried out to separate objects from the background. The method of extracting shape and texture features is used to obtain the characteristics of dermoscopy images. As many as 2 types of shape features, namely eccentricity and metric, and 4 types of texture features, namely contrast, correlation, energy, and homogeneity. The result of this study is that it can identify the type of skin cancer based on image features that have been extracted using a program from the Matlab application. The technique of extracting shape and texture features is proven to work well in identifying the type of skin cancer. In the future it is expected to use more data, and add color features in identifying dermoscopy images.

Diagnosis of Covid-19 from CT slices using Whale Optimization Algorithm, Support Vector Machine and Multi-Layer Perceptron.

The coronavirus disease 2019 is a serious and highly contagious disease caused by infection with a newly discovered virus, named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A Computer Aided Diagnosis (CAD) system to assist physicians to diagnose Covid-19 from chest Computed Tomography (CT) slices is modelled and experimented. The lung tissues are segmented using Otsu's thresholding method. The Covid-19 lesions have been annotated as the Regions of Interest (ROIs), which is followed by texture and shape extraction. The obtained features are stored as feature vectors and split into 80:20 train and test sets. To choose the optimal features, Whale Optimization Algorithm (WOA) with Support Vector Machine (SVM) classifier's accuracy is employed. A Multi-Layer Perceptron (MLP) classifier is trained to perform classification with the selected features. Comparative experimentations of the proposed system with existing eight benchmark Machine Learning classifiers using real-time dataset demonstrates that the proposed system with 88.94% accuracy outperforms the benchmark classifier's results. Statistical analysis namely, Friedman test, Mann Whitney U test and Kendall's Rank Correlation Coefficient Test has been performed which indicates that the proposed method has a significant impact on the novel dataset considered. The MLP classifier's accuracy without feature selection yielded 80.40%, whereas with feature selection using WOA, it yielded 88.94%.

Automated Grading of Angelica sinensis Using Computer Vision and Machine Learning Techniques

Angelica sinensis (Oliv.) Diels, a member of the Umbelliferae family, is commonly known as Danggui (Angelica sinensis, AS). AS has the functions of blood tonic, menstrual pain relief, and laxatives. Accurate classification of AS grades is crucial for efficient market management and consumer health. The commonly used method to classify AS grades depends on the evaluator’s observation and experience. However, this method has issues such as unquantifiable parameters and inconsistent identification results among different evaluators, resulting in a relatively chaotic classification of AS in the market. To address these issues, this study introduced a computer vision-based approach to intelligently grade AS. Images of AS at five grades were acquired, denoised, and segmented, followed by extraction of shape, color, and texture features. Thirteen feature parameters were selected based on difference and correlation analysis, including tail area, whole body area, head diameter, G average, B average, R variances, G variances, B variances, R skewness, G skewness, B skewness, S average, and V average, which exhibited significant differences and correlated with grades. These parameters were then used to train and test both the traditional back propagation neural network (BPNN) and the BPNN model improved with a growing optimizer (GOBPNN). Results showed that the GOBPNN model achieved significantly higher average testing precision, recall, F-score, and accuracy (97.1%, 95.9%, 96.5%, and 95.0%, respectively) compared to the BPNN model. The method combining machine vision technology with GOBPNN enabled efficient, objective, rapid, non-destructive, and cost effective AS grading.

Multi-Dimensional Extraction of Ice Shape and an Investigation of Its Aerodynamic Characteristics on Iced Wind Turbine Blades

The icing of wind turbine blades can cause changes in airfoil shape, which in turn significantly reduces the aerodynamic performance and affects the power generation efficiency of a wind turbine. In this paper, the iced airfoil shape of wind turbine blades with different positions, masses, and angles of attack icing was measured and modeled using 3D scanning technology, and changes in airfoil shape parameters under different icing conditions were obtained. The numerical simulations of icing blades were carried out to investigate the effect of blade icing on aerodynamic characteristics. The results show that ice accumulation thickness tends to increase nonlinearly along the spanwise direction and chord length for both windward and leeward icing. The airfoil angle of attack affects the trend of ice accumulation changes. As shown by the numerical simulation of the aerodynamic characteristic, blade icing changes the airfoil shape, which changes the pressure difference between the leading edge and trailing edge, affects the size and number of the wake vortex structures, and further changes the aerodynamic characteristics of the blade.

Mitigating Distractor Challenges in Video Object Segmentation through Shape and Motion Cues

The purpose of semi-supervised video object segmentation (VOS) is to predict and generate object masks in subsequent video frames after being provided with the initial frame’s object mask. Currently, mainstream methods leverage historical frame information for enhancing the network’s performance. However, this approach faces the following issues: (1) They often overlook important shape information, leading to decreased accuracy in segmenting object-edge areas. (2) They often use pixel-level motion estimation to guide the matching for addressing distractor objects. However, this brings heavy computation costs and struggle against occlusion or fast/blurry motion. For the first problem, this paper introduces an object shape extraction module that exploits both the high-level and low-level features to obtain object shape information, by which the shape information can be used to further refine the predicted masks. For the second problem, this paper introduces a novel object-level motion prediction module, in which it stores the representative motion features during the training stage, and predicts the object motion by retrieving them during the inference stage. We evaluate our method on benchmark datasets compared with recent state-of-the-art methods, and the results demonstrate the effectiveness of the proposed method.

Damage detection based on accelerometers and computer vision measurements of moving load-induced structural responses

Damage identification is an important and challenging issue in structural health monitoring to prevent sudden structural failures. This study introduces a novel damage detection method for beams associated with the cross-correlation function of the moving load-induced full-field displacements. Firstly, through theoretical derivation, the structural responses of a single-span beam under arbitrary boundary conditions subjected to a moving concentrated force were investigated, both before and after damage occurred. The changes in structural responses due to local stiffness damage were analyzed from a theoretical perspective. Then, a method is introduced to extract high-resolution mode shapes, which represent the true state of the structure, from the computer vision-based displacements and accelerations measured at limited locations. These mode shapes were then used for full-field displacement reconstruction, and the changes in the cross-correlation function of these reconstructed displacements were employed for damage detection. The proposed method can obtain the higher-order responses compared to the traditional vision-based methods, and the displacements at the acceleration-unmeasured locations can be improved compared to the conventional data fusion method. Finally, the proposed method was validated both in numerical simulations and experimental tests on a beam model with multiple damage scenarios. The results have demonstrated the capability of the proposed method in mode shape extraction, full-field displacement reconstruction, and damage detection.

Automatic vectorization of historical maps: A benchmark.

Shape vectorization is a key stage of the digitization of large-scale historical maps, especially city maps that exhibit complex and valuable details. Having access to digitized buildings, building blocks, street networks and other geographic content opens numerous new approaches for historical studies such as change tracking, morphological analysis and density estimations. In the context of the digitization of Paris atlases created in the 19th and early 20th centuries, we have designed a supervised pipeline that reliably extract closed shapes from historical maps. This pipeline is based on a supervised edge filtering stage using deep filters, and a closed shape extraction stage using a watershed transform. It relies on probable multiple suboptimal methodological choices that hamper the vectorization performances in terms of accuracy and completeness. Objectively investigating which solutions are the most adequate among the numerous possibilities is comprehensively addressed in this paper. The following contributions are subsequently introduced: (i) we propose an improved training protocol for map digitization; (ii) we introduce a joint optimization of the edge detection and shape extraction stages; (iii) we compare the performance of state-of-the-art deep edge filters with topology-preserving loss functions, including vision transformers; (iv) we evaluate the end-to-end deep learnable watershed against Meyer watershed. We subsequently design the critical path for a fully automatic extraction of key elements of historical maps. All the data, code, benchmark results are freely available at https://github.com/soduco/Benchmark_historical_map_vectorization.